Staking Router Module Proposal: Simple DVT

Thanks for your reply @Izzy.

I agree and as previously mentioned, I view it as a promising foundation for the switch to permissionless modules in order to finally achieve that ambitious goal of thousands of operators.

Kudos to Obol and SSV teams for their detailed and clarifying answers.

3 Likes

Hello, Lido community!

We are excited about the potential of DVT to bring further security and decentralization to Ethereum, and are glad to put forward this proposal to contribute to the security and adoption of the Simple DVT module.

Introducing Chainproof

Chainproof, a regulated primary insurance carrier backed by the world’s top reinsurer, Munich Re, is well positioned to offer its best-in-class slashing insurance product for the Lido DVT staking module.

  • Institutional Backing: Our insurance capacity is bolstered by Sompo, Japan’s second-largest insurance entity. We are fully regulatory compliant, with a class IGB insurance license in Bermuda, the world’s premier insurance jurisdiction.
  • Web3 Expertise: Chainproof isn’t just an insurer. We delve deep into the web3 ecosystem, insuring against risks like slashing and smart contract hacks, thanks to insights from our parent company, Quantstamp - a leader in web3 security.
  • Track Record: Chainproof proudly insures top validators like Kiln and Luganodes against slashing risks, among other crypto native entities and asset managers.

Our Differentiator

What makes Chainproof unique is our synergy of insurance and security:

  • Proactive Monitoring: Equipped with a security-centric DNA, we are able to vigilantly monitor DVT validators, consistently assessing DVT client risks.
  • Security Engagements: We’ve meticulously reviewed the validator infrastructure for top players. In addition,we’re gearing up for an audit of Obol’s Charon client. We developed deep familiarity with the security facets of SSV, thanks to our prior smart contract audits.

This fusion of security expertise and aligned incentives empowers Chainproof to accurately price and underwrite risks, presenting clients with competitive premiums, institutional grade insurance coverage, and unparalleled value.

Read More: Dive into our thoughts on slashing insurance via our blog.

The Chainproof Insurance Offer for Lido’s Simple DVT Staking Module

We are happy to put forward a simple and transparent proposal for the Lido community as follows:

  • Insurance Details: Chainproof is poised to protect LIDO’s Simple DVT Staking Module against slashing risks, with coverage up to 4 ETH per node and a 25% deductible.
  • Premium Calculation: At a total premium of 198 ETH, the DAO’s staking income comfortably covers the premiums for a span exceeding 28 years. That’s just 0.45% per ETH staked.
  • Understanding the Deductible: This refers to the initial portion borne by the DAO (or policy beneficiary) during a slashing event. Akin to traditional policies, a higher deductible brings down the cost of insurance, intertwining self-insurance benefits with our offering.

We are grateful for the consideration and opportunity to craft a safer, more secure staking environment for Lido in its goal to continue cementing its position as the leading choice for Ethereum liquid staking.

We are always available in this forum to answer any questions or concerns from the Lido community and stakeholders.

10 Likes

Nexus Mutual <> Lido DVT Proposal

The Current Position

The stake in Lido has grown rapidly, with billions of dollars of ETH now staked through the protocol. This growth in stake means that the DAO has been able to expand its team to do more to support the protocol. Critical to this is protecting users. Any loss of funds that’s experienced by users will hamper growth, which is why the DAO decided to provision funds to protect users against the impact of slashing events.

But these provisions have not kept pace with the growth in stake, meaning that they reduce each month in percentage terms. Currently, they only cover 0.08% of the stake, meaning that users would be impacted by events larger than this. In my opinion, this level of coverage is not sufficient, particularly as the stake continues to grow each year.

While the DAO has surplus funds that it could allocate to these slashing provisions, I still believe much deeper levels of coverage are necessary as Lido expands with new modules like DVT.

The core problem is the immense value of the stake. Even a small percentage loss is a huge absolute value and so to internally provision for these losses the DAO needs to be extremely well funded.

Self-coverage via provisioning is somewhat common among large corporations with massive excess reserves, unutilized cash flow and relatively small expected losses.

Lido DAO doesn’t currently match these criteria, which makes it challenging to expand the provisions inline with the rapid growth of the protocol. Given the DAO’s current runway and the size of the coverage gap, it might be preferable that the DAO consider implementing an additional fee to support the growth of these provisions, or to pay for premiums for external coverage.

DVT Module: Risks and Coverage

For the DVT module specifically, the risk profile is very different to the main staking module. Primarily, the risk is in unknown unknowns, due to the new codebase which hasn’t been battle tested like the current, trusted Lido staking module. With DVT the goal is to reduce the risk of trusting a given operator, trading off for more technical complexity. I believe that in the future it will serve to meaningfully reduce the risk of staking. However, we should accept some increase in the current risk profile because as with all smart contracts it will be most vulnerable to exploits when it’s first targetable by malicious actors.

The size of the DVT module makes it plausible that the DAO could utilize internal provisioning to protect against this risk. In the case of an exploit, these internal provisions would be sufficient to make users whole and the DAO would likely vote to replenish the reserves from the current surplus.

While I believe that this approach is sufficient for the purposes of protecting users from the risks of the DVT module, I don’t believe that the current provisions and the surplus together are sufficient to protect the entire stake at risk. Therefore, while I can see the logic of using internal provisions to protect the stake in the DVT module, I believe that a move to either significantly fund the staking provisions or to pursue external coverage more broadly is necessary. I see this DVT module as an opportunity to explore external coverage, with the goal of retaining the strength of the current provisions, and re-proving the benefits of external coverage to then expand this more broadly.

External Coverage

Lido DAO voted to not renew a previous coverage policy because this cost 25%+ of the DAOs internal revenue each month. Yet, in the past weeks we’ve seen a proposal from DAO members suggesting that the DAO looks to provision more than 25% of its funds each month for self-coverage.

My reading of this proposal is that it would require more than 100% of the revenue to the DAO to be allocated to self-coverage. At its core, this is the problem with self-coverage. The magnitude of the risks is so far above the current financial position of the DAO, that the DAO cannot provision enough funds from its revenue to match the growth of the Lido protocol.

Today, the provisions are 6,235 stETH to protect users against slashing. The protocol has 8,169,134 ETH staked, so the provisions are for 0.08% of the funds at risk.

Currently, this slashing provision is only sufficient to protect against relatively smaller events, with any sizable risks having expected losses far in excess of 6,235 stETH. A single operator experiencing slashing across their set, or a client bug causing downtime, could lead to events that either massively depletes this fund or wipes it entirely. If the goal is to mitigate the impact of catastrophic events, external coverage can provide deeper levels of protection.

However, relatively small events like downtime or non-function of the DVT module are coverable by the DAO slashing fund because the cost of penalties and missed rewards are relatively small.

The DVT module will allow for 0.5% of the Lido stake, meaning 40,845 ETH. Should the Lido DAO increase the stake allowed in the DVT module we would expect to re-quote a new policy for the higher stake amount.

Downtime for this entire cluster would mean 2.37 ETH in penalties and 3.8 ETH in missed rewards, per day. The provisions of 6,235 stETH would therefore allow for this DVT cluster to be offline for ~900 days before depleting. Clearly, this is easily sufficient…

Where the provisions are insufficient is in the case of an exploit or bug that leads to the loss of funds from the smart contracts, or slashing to the validators. DVT is brand new and so there are likely to be a host of unknown unknowns, which make this particularly risky.

With ~41k ETH potentially entering the DVT module it’s not feasible to get a policy that will protect this entire sum. Instead, we must focus on protecting as much as possible, while keeping the premiums at a level that the DAO can afford.

Nexus Mutual Coverage Proposal

Protecting 20% of the funds at risk in the DVT module, ~$13M, would protect against most of the day-to-day risks of operations.

Full terms of the policy will be shared should the DAO vote to pursue external coverage. We expect that this policy would include penalties and losses to slashing, with a minimal 2.5% deductible to prevent the submission of many small claims due to normal operations.

Per ETH deposited into the module, Nexus Mutual would charge 0.6% per year, offering up to 8,200 ETH of coverage and with coverage of up to 20% of the size of the module. Lido DAO would be free to select the coverage limit that’s right to them, in increments of 1 ETH, so long as that coverage total is less than 20% of the ETH in the DVT module. This structure will allow the policy to grow with the DVT module.

This policy is structured as an umbrella, rather than per validator, meaning that while this can be thought of as 20% per validator (6.4 ETH), it’s superior because it would allow for up to 32 ETH payout on any single validator with a policy-wide limit of 20% of the DVT module stake.

Examples:

DVT Module ETH Max Coverage Option Deductible Max Claims Payout Yearly Cost ETH
1000 200 5 195 6
10,000 2,000 50 1,950 60
30,000 6,000 150 5,850 180
41,000 8,200 205 7,995 246

Even at its peak of 41,000 ETH, the 0.5% limit imposed on this DVT module via Lido, the yearly cost of 246 ETH means that the current 6,235 ETH provisions could pay for 22-years of premiums.

Given that this 6,235 stETH in provisions is in stETH, at 4% per year it will earn 249 ETH in expected returns. That alone would pay for all of the premiums under even the most extreme policy.

To compare to the ChainProof policy which covers 4 ETH per validator, with a 25% deductible, meaning a maximum of 3 ETH payout per validator. The cost is 198 ETH, so the % cost is approximately:

198 ETH / (1281 validators * 3 ETH max payout per validator) = 5.15%

Our comparable policy looks like:

180 ETH / 5,850 ETH max payout = 3.08%

We are able to offer a deeper level of coverage, 5,850 ETH vs 3,843 ETH (3 ETH max payout per validator), at a lower cost of 180 ETH rather than 198 ETH. Should Lido DAO choose to opt for a larger policy, say 8,200 ETH, we’re able to offer this at the same 3.08% rate, only 246 ETH per year.

Smart Contract Coverage

The suggested policy above covers slashing and penalties. We would also be able to offer smart contract coverage for the module at 3% of the value covered. For example, if the DAO wanted to cover 10,000 ETH for one year against smart contract risk, the cost would be 300 ETH. This smart contract coverage is an optional choice outside of the primary slashing and penalties coverage policy quoted previously.

6 Likes

Comparison between RFC respondents

The baseline is to rely on today’s DAO slashing provision wallet. The below chart compares two insurance alternatives to cover DVT, namely Chainproof and Nexus Mutual.

To do this we model the size of the Simple DVT module up to a cap relative to today’s amount of stETH.

Inputs
Slashing provision 6235
Staked ETH 8,824,390
DVT Cap 0.50%

Nexus Mutual

% / cap DVT ETH Validators Max Coverage Coverage % Deductible Deductible % Max Payout Yearly cost ETH Cost % Coverage vs provisions Cost of incr. coverage
0.01% 1,000 31 200 20.0% 5 2.5% 195 6 3.00% (6,035.0) NM
0.11% 10,000 313 2,000 20.0% 50 2.5% 1,950 60 3.00% (4,235.0) NM
0.34% 30,000 938 6,000 20.0% 150 2.5% 5,850 180 3.00% (235.0) NM
0.46% 41,000 1,281 8,200 20.0% 205 2.5% 7,995 246 3.00% 1,965 12.52%
0.50% 44,122 1,379 8,824 20.0% 221 2.5% 8,604 265 3.00% 2,589 10.22%

nb. the last row is hypothetical and did not feature in the original proposal NM team please flag if you would like it removed

Chainproof

% / cap DVT ETH Validators Max Coverage Coverage % Deductible Deductible % Max Payout Yearly cost ETH Cost % Coverage vs provisions Cost of incr. coverage
0.01% 1,000 31 125 12.5% 31 25.0% 94 5 3.87% (6,110.0) NM
0.11% 10,000 313 1,250 12.5% 313 25.0% 938 48 3.87% (4,985.0) NM
0.34% 30,000 938 3,750 12.5% 938 25.0% 2,813 145 3.87% (2,485.0) NM
0.46% 41,000 1,281 5,125 12.5% 1,281 25.0% 3,844 198 3.87% (1,110.0) NM
0.50% 44,122 1,379 5,515 12.5% 1,379 25.0% 4,136 213 3.87% (719.8) NM

nb. the format is hypothetical and assumes a linear cost CP team please flag if you would like it modified. normalized so the quotes line up between both providers for the same level of coverage, please advise if this assumption is mistaken

Conclusions

The slashing provision of 6,235 ETH is sufficient to cover the maximum coverage prescriptions from either of these proposals except in the case of Nexus Mutual for 8,200 or a hypothetical 8,824. If framed as ‘incremental’ coverage relative to the existing slashing provision, as both these proposals do, this incremental coverage comes at a very high cost in most instances.

Evaluated side by side the Nexus Mutual offer is more economically attractive.

  • It is objectively cheaper
  • It covers more ETH in total
  • It has a lower deductible
  • It has shared coverage rather than per validator coverage

However, it’s not yet clear in our view that it is sufficiently economically interesting to merit wholehearted recommendation relative to relying on the slashing provision. To wit, allocating 2.5k stETH from the protocol surplus into the slashing provision wallet would effectively allocate self-insurance at a +10% ROI relative to the highest level of coverage offered by Nexus (nb. 10.22% cost of incremental coverage over the slashing provision).

Although the Nexus offer has a reasonable ask in terms of the deductible requested, we would like to see higher max coverage for the same premiums, even if it were a short-term solution for the Simple DVT module.

Look forward to hearing from the teams on the above comments.

Edit: small normalization on Chainproof

6 Likes

Hey there, Mol_Eliza from Analytics Workstream!
First of all, thank you @KimonSH for transparent and precise formulated proposal, all participants for discussion and @chainproof and @ccitizen for their proposals for mitigation mechanisms.
I’d like to provide my view on options for risk mitigation.

Risk decomposition

For baseline step I’ll dive deep in risks associated with performing validator duties, based on Risk assessment for community staking for 1 Validator (cluster)

All numbers would be calculated within a reasonable upper boundary assumptions (as risk decreases with overall APR) on CL and EL rewards based on recent observed data

  • Cl rewards: 3.2%
  • EL rewards: 1.2%
  1. Risks associated with slashings:
    1.1. Initial penalty: 1 ETH
    1.2. Correlation penalty: variable*
    1.3. Attestation penalties: 0.0757 ETH (for 36 days exit period)
    1.4. Missed rewards: 0.1389 ETH (for 36 days exit period)
  2. Risks associated with validator inactivity:
    2.1. Attestation penalties: 0.768 ETH (for 1 year of inactivity)
    2.2. Missed rewards: 1.408 ETH (for 1 year of inactivity)

Correlation penalty is incremental and depends on share of total network within ongoing slashing:

  • 0 ETH in case less than 1.04% (8988 validators for today) of total network within ongoing slashing
  • 1 ETH for 1.04% - 2.08% of total network within ongoing slashing
  • 2 ETH for 2.08%-3.13%
  • 3 ETH for 3.13% - 4.17%
  • <…>

Considering mitigation it seems important to explicitly specify risks covered from the list above: as for me it seems more clear that direct penalties (1.1-1.3) are within scope, but missed rewards associated with slashings (1.4) and risks associated with validator inactivity (2) may be not included within the proposals.
If possible, would really appreciate clarification @chainproof, @ccitizen

Risk coverage

Applying framework above i’d like to share my vision on different risk-scenarios and compare them with coverage provided within existing fund and proposals for external coverage.
As minor scenarios (~100 validators slashings) could be covered with existing fund, the focus would be on major incidents

All calculations would be done assuming 41 000 DVT ETH / 1281.25 Validators (non-integer value for stake of scalability)

Scenario 1 Only slashing: all validators are slashed and exited within 36 days, <1.04% of total network within ongoing slashing

Scenario 2 Only slashing + Correlation (1 ETH): all validators are slashed and exited within 36 days, >1.04% and <2.08% of total network within ongoing slashing

Scenario 3 Massive downtime + Slashing: all validators are inactive for one year, then slashed and exited within 36 days, <1.04% of total network within ongoing slashing

Scenario 4 Massive downtime + Slashing + Correlation (3 ETH): all validators are inactive for one year, then slashed and exited within 36 days, >3.13% and <4.17% of total network within ongoing slashing

Calculations details
Payout is calculated based on generous assumption of covering all risks

Key observations:

1. Inactivity risks

If inactivity (downtime) risk is not included in coverage - substantially lowering coverage should be considered, as even im most catastrophic reasonable scenario (all validators are slashed, no correlation penalty) total effect only from penalties would be ~1.3k ETH

2. Correlation penalty risks

Levels of coverage provided are sufficient enough to cover 1-2 ETH correlation penalty, but this risk is out of scope of simple DVT (and hence not affected by risks involving with using new technology).
It would require more than 1% of network slashing simultaneously with all validator slashing within module for correlation penalty to reach 1 ETH from 0.
Taking this risk should be considered rather than covering

3. Coverage providers comparison

For all major scenarios Nexus proposal demonstrates more coverage and effect in terms of payout / costs
While its worth to mention that for minor events (~100 Validator slashings) Chainproof deductible structure would lead to some levels of payments, while under Nexus police all effect (~120 ETH) would be under deductible amount

Given observations 1-2 so far proposals on extrenal coverage provided may be overestimating the impact to cover and, therefore, reasonable premium to cover those effect may be too high in comparison with self-covering

For minor incidents (<300 validator slashings) both options provides little to no coverage as most of the impact (>50%, up to 100% for ~150 validators) would be within deductable amount

3 Likes

Hi all

Thought it was worth highlighting a few of the key points that have been raised re the Nexus Mutual proposal. We’re also very much looking forward to working with the DAO to fine tune the coverage to suit your needs.

How it works:

Nexus Mutual exists on the Ethereum chain. Any wallet can KYC and join the membership of the mutual by paying a nominal fee via an on-chain transaction. Once approved, this Lido DAO wallet will be able to purchase the coverage on-chain.

Nexus Mutual will list the policy, with the agreed terms and pricing. The DAO will be able to purchase this via our UX with a single transaction, paying in ETH, with coverage denominated in ETH terms. The cover will be for an agreed upon duration, though we will allow you to purchase for as little as 30-day increments, preserving cashflow for the DAO and making it easier to scale the coverage as the module grows.

Once the DAO has purchased on-chain, their wallet will automatically receive an NFT, which represents the policy. Should a claim arise, the same wallet will use this NFT to submit a claim on our website.

Claims:

  • Claim Filing: In case of an incident, the Lido DAO would file a claim in the Nexus Mutual UI, describing the event and providing a proof of loss (e.g. the on-chain evidence that slashing occurred).

  • Claim Assessment: After filing the claim, Nexus Mutual’s claim assessors will vote on the claim. Claim assessors review the incident details, proof of loss, and other supporting information to determine a claim’s validity. Given the on-chain nature of slashing events and penalties, a claim under this policy would be very straightforward to assess.

  • Claim Payout: When a claim has been accepted, a cool-down period of one day needs to pass before the claim payout can be redeemed.

  • Time: The typical time it takes for a claim to be filed, assessed and paid out is 5-7 days.

  • History: Since inception, Nexus Mutual has paid out ~$18M in claims, the largest amount of any on-chain DeFi cover provider. Incidents that led to these claims included projects such as Euler Finance, FTX, Rari Capital, Sherlock, Bancor and Yearn. Because our system is on-chain, all claims are transparent and can be independently verified. Our DAO has created an overview of all claims with detailed descriptions (Nexus Mutual - Claims History)

Trusted Partner:

  • Lido: We have been a long-term partner to Lido, having staked ~30k ETH of our own capital pool since 2021.

  • Slashing Cover: Over the past two years, Nexus Mutual has worked with all major staking operators to understand all the risks, and we have been the preferred slashing cover provider for liquid staking protocols such as Liquid Collective, EtherFi and Stakewise.

What’s covered:

Our policy covers the direct loss of ETH to slashing (i.e. 1 ETH, or greater if correlated), and the penalties that are directly associated with the slashing event. It does not cover penalties that are unrelated to slashing, and it does not cover loss of rewards.

More Information:

4 Likes

Hi everyone. Below is an updated version of the full text of the proposal, following some suggestions from team members at Obol and SSV, as well as community members.

The new full text has been made available here (below), on hackmd, github, and IPFS.

The snapshot proposal will go live with the Summary from the text below and key excerpts, but please refer to the full text for all the details.

The following proposal is the final draft that will be put forth for Snapshot vote to the Lido DAO on October 26th. Please see the Snapshot vote linked here (PENDING) to participate in the decision as to whether the module should be deployed on mainnet.

A diff (comparison) from the original post can be found here.


Staking Router Module Proposal: Simple DVT

Proposal Summary

This proposal seeks DAO approval for the addition of a new module that will utilize Distributed Validator Technology (DVT) on mainnet through Obol Network and SSV Network implementations. This module will be referred to as the “Simple DVT” module, due to the manual coordination and curation required to adopt this early-stage technology, and to reflect that this module is not intended for indefinite and “at-scale” use.

DVT represents the fastest way to add many new Node Operators to the Lido Node Operator set with a more diverse profile of solo and community staker participants while benefiting from the technology’s inherent benefits such as increased resilience, distribution, and security. The Simple DVT module is intended to demonstrate that utilizing DVT on mainnet is possible, while furthering the diversification of the Lido Node Operator set on Ethereum and potentially setting the stage for more scalable and permissionless DVT-based modules in the near future.

Solo stakers, community stakers, existing node operators using Lido protocol, and other staking organizations would participate in the upcoming 3rd Lido DVT testnet utilizing Obol and SSV DVT solutions. Following a review of individual and cluster performance, the Lido Node Operator Subgovernance Group (LNOSG), with input from Obol and SSV, would be responsible for finalizing clusters to participate in a mainnet deployment.

The LNOSG would propose the cluster combinations for DAO discussion, and if no significant issues are identified, a committee known as the Simple DVT Module Committee would be responsible for adding these Node Operators to the mainnet registry and increasing validator limits per cluster.

The number of clusters and amount of validators operated per cluster would initially be minimal (e.g. 24 clusters running 5 validators each), leaving time for the Simple DVT Module Committee and DAO to monitor performance and assess the impact to the protocol. If the distributed validators demonstrate performance in-line with the broader Ethereum network, additional clusters could be added to mainnet and the number of validators per cluster would be increased.

After at least three months of mainnet performance comparable to the overall operator set, the LNOSG would also meet to discuss and potentially recommend increasing the number of validators operated by clusters to more meaningful levels (e.g. 100+), considering factors such as cluster performance and the make-up of participants.

The Staking Router technical documentation outlines two reward share components: the module fee and treasury fee. To incentivize Node Operator participation and provide continued support to DVT providers, it is proposed that the treasury fee for the Simple DVT Module is set to 2% and the module fee (paid to Node Operators and the DVT provider) is set to 8%. This reflects the limited economic attractiveness of running a small number of validators with multiple members and the desire to support continued development of Distributed Validator Technology, while taking into account the limited amount of stake the Simple DVT Module will contain and the intent to wind-down the module in the medium-term.

During the discussion period of this proposal, the DAO should also consider the choice of two risk mitigation approaches that will be included in the vote:

  1. DAO approval for the current (or future) cover fund to be utilized in the case of material impact to stETH stakers, with a maximum cover of up to 4 ETH per validator.
  2. Open an RFP process for interested cover providers to provide risk mitigation options with the subsequent DAO vote to choose between providers.

In summary, this proposal would greenlight the creation of a secondary module on mainnet (contingent upon a successful testnet deployment achievement of minimum success metrics) that would be used to allow Node Operators participating in DVT clusters using SSV Network and Obol Network technologies to use the Lido protocol to run validators. This module would be initially capped at 0.5% of Lido stake, which could be increased later by DAO vote, and be expected to be wound-down and replaced by more scalable modules over time (e.g. within approximately 2-3 years). The module would have the rewards share set at 10%, with 2% allocated to the treasury fee, and 8% to the module fee via participating Node Operators and DVT providers.

The proposal would also grant the Simple DVT Module Committee the approval to execute Easy Track governance motions (i.e. motions that can be objected to by DAO token holders) for aiding Simple DVT cluster operations (i.e. adding operators and setting validator limits) following LNOSG evaluations of upcoming DVT testnet trials.

Following the discussion period and any potential modifications incorporating community feedback, this proposal will be put forth for Snapshot vote to the DAO on October 26th, 2023.

Full proposal follows

Motivation

The Staking Router (introduced in Lido V2) adds significant optionality in terms of the ways Node Operators might participate in the protocol, however wholesale newly-designed modules are likely still 8 - 18 months away from mainnet.

In order to increase the scope of Node Operators who use the protocol and to support new technologies in the staking space such as DVT, this proposal seeks DAO consideration and approval for the creation of a secondary module on mainnet that utilizes both SSV and Obol DVT implementations to increase the number and type of Node Operators using the protocol.

This secondary module would set the groundwork for the continued expansion of the Lido Node Operator set while more permissionless modules are developed, in-line with the Lido community’s commitment to decentralization, accessibility, and censorship resistance, as well as the ability to lead the way in terms of new technologies adding to the robustness of Ethereum’s validator set.

Following two successful pilot integrations of Distributed Validator Technology based on implementations by Obol Network and SSV Network on the Goerli testnet, additional steps can be taken to advance the adoption of DVT for the Lido protocol.

While DVT is still in the early stages of adoption, this technology can assist in adding new Node Operator participants to the protocol in the near-term while benefiting from the increased resilience, security, and decentralization that DVs provide. The Simple DVT module would play a role in helping battle-test this new technology while limiting the scale and potential impact from any issues that may arise. The proposed module would initially be capped at 0.5% of Lido stake and potentially be covered by a Lido DAO cover fund or third-party cover so as to address potential sub-optimal performance (e.g. unexpected DVT software bugs) associated with this experimental technology.

These efforts would be encapsulated in the “Simple DVT” module, a module that leverages the existing design of the Curated Operator Module thereby keeping development scope manageable, and would incorporate the processes and mechanisms developed over the previous two DVT testnets as well as an upcoming third one. The “Simple DVT” module has the potential to see the first community stakers (what Lido DAO contributors use to refer to solo and small groups of stakers) participate in the Lido protocol as Node Operators on mainnet within early 2024.

This proposal suggests the new module would be:

  • Initially capped at 0.5% of total Lido stake (with the option to increase the cap via DAO vote),
    • The module cap should be re-assessed for possible increase by Q3/2024 and Q2/2025
  • require achievement of minimum success metrics on testnet (outlined below),
  • Not operated indefinitely, with the intention for the Simple DVT module to be superseded by more sophisticated DVT modules designed with scaling and permissionless participation in mind.

By moving forward with a Simple DVT implementation, the Lido DAO can advance its mission to keep Ethereum decentralized and democratize access to staking in the near-term while more complex and scalable modules are developed over the coming months.

Why DVT

DVT has the potential to provide significant benefits in terms of decentralization, liveness, and security. It also represents the single fastest way to increase the number of independent Node Operators participating in running validators using the Lido protocol in a secure manner.

By combining existing Lido on Ethereum Node Operators with solo and community stakers participating in testnets operated through the Lido Node Operator registry on Goerli and soon, Holesky, trust requirements for community participants can be further minimized as the threshold structure of a DVT cluster reduces the risks of single operator infrastructure issues and malicious behavior.

At the validator level, DVT enables high availability deployments, significantly improving liveness for the Ethereum validator set. It also enhances geographic, jurisdictional, client, and infrastructure diversity. Furthermore, recent DVT research seems to indicate significant benefits to validator resilience, including possible reduction of slashing risk, through an active-active cluster redundancy configuration and the distributed key-generation process.

While a reduction in slashing risk should reduce the overall technical risks to validators utilizing the protocol over the long-term, given the early stage nature of the technology additional mitigation should be considered for this initial deployment (as discussed in the “Associated Risks & Mitigation” section).

Proposed Process for Cluster Organization

Testnet Round Three

To date, Lido on Ethereum DVT testnet deployments have included cluster configurations consisting of 3/4 thresholds, 5/7 thresholds, 7/10 thresholds, and 9/13 thresholds.

The next round of testnet trials would be organized in a manner to replicate a potential mainnet deployments as closely as possible. The majority of clusters would be in a 5/7 threshold or greater, reflecting the benefits of increased resilience and redundancy larger clusters promote.

To move forward with a mainnet deployment, the following success characteristics should be considered over a 30 to 45 day testing period:

  1. Uptime of at least 95.00%
  2. Successful block proposal rate of at least 70% (some flexibility is needed due to the early stage of MEV-Boost integration and outstanding questions regarding Holesky performance)
  3. Attestation effectiveness per Rated Network comparable with that of the broader Holesky validator set

Given that two separate DVT infrastructures (Obol Network and SSV Network) would underpin the distinct clusters, the success characteristics should be grouped by infra provider and considered independently.

Achievement of these characteristics and a successful on-chain DAO vote to deploy the mainnet Simple DVT module would greenlight a candidate evaluation by the LNOSG for participation on mainnet and the subsequent deployment of Node Operator clusters.

In the event these performance metrics are not achieved, one of two options should be taken:

  1. Run another testnet round before deploying on mainnet
  2. Analysis of the testnet is presented to the DAO for discussion with an explanation of why specific criteria may not have been achieved, with a subsequent DAO vote to determine whether to proceed to mainnet or not

Mainnet

In the event of a successful DAO vote to move forward with Simple DVT, mainnet deployment would operate with the following characteristics:

Operator Additions to the Simple DVT Module

Participants for the mainnet clusters of the Simple DVT Module will be sourced through the third round of Lido DVT testing.

The proposal would grant a new multi-sig committee (known as the “Simple DVT Module Committee”) consisting of contributors from the Lido DAO, LNOSG, Obol Network, and SSV Network teams the ability to create and execute Easy Track motions that allow for the addition of new clusters, activation and deactivation of existing clusters, and changing of cluster properties for the Simple DVT Module.

The LNOSG will be responsible for ongoing evaluations of Simple DVT Node Operators and the cluster combinations that will be used to operate validators. Factors considered will include individual Node Operator performance, infrastructure type, and geographic location.

When the LNOSG reaches consensus, the proposed clusters will be transparently presented to the DAO via the Lido Research forums. If no major concerns are raised after a 7 day discussion period, the Simple DVT Module Committee will execute the addition of new Node Operators to the Simple DVT module.

Operator Selection for the Simple DVT Module

For the initial implementation, the LNOSG would propose e.g. 24 cluster combinations (12 using Obol and 12 using SSV) of potentially different threshold sizes (but most likely 5/7) based on prior testnet experience for onboarding as distributed validator participants.

The clusters would form multi-sigs that would form the basis of their entry in the Node Operator registry. For the initial stage of the deployment, each cluster could be approved for 5 validators and performance would be monitored in a transparent manner and reported to the DAO.

As an example, running this in a 5/7 scenario with 50%+ of the distributed nodes run by Node Operators from the Curated Module (not a requirement) would mean over 70 non-Lido Curated Set operators would be participating in running validators using the protocol (bringing the total number of Node Operators using the protocol to over 100). At the same time, with an initial limit of 5 validators per cluster, only 120 validators (0.04% of total Lido validators) would be initially used for testing.

If after 30 days the participants demonstrated strong performance on mainnet and both DVT providers complete up-to-date codebase audits, the limit of validators run by each cluster could be increased (e.g. to 20) and additional distributed validator clusters could be onboarded.

Assuming 70 clusters run in this format by Q2 of 2024, 210+ net-new Node Operators could be onboarded to the protocol. While the exact number of net-new Node Operators would depend on the cluster threshold combinations, the majority of clusters would be in a 5/7 threshold or greater.

These clusters would not only add to the number of Node Operators participating, but also seek to add additional client, geographic, and infrastructure diversity to the validators run as part of the protocol.

Further Increases to Validator Limits

While the primary intent behind the Simple DVT module is to test mainnet DVT and expand the Node Operator set, a drawback of clusters running a small number of validators is the economic reality of a group sharing rewards for only a minimal number of validators.

After sufficient battle-testing and performance monitoring of the DV clusters (at least three months after launch), the LNOSG will also consider for certain clusters to increase the number of validators operated to a higher threshold.

It is suggested that the LNOSG consider a classification such as “Advanced Node Operators”. Advanced Node Operators would be participants that demonstrate consistently strong performance and ecosystem alignment. These Node Operators would be sourced from the mainnet clusters, DVT testnet trials, DVT provider verified operator lists, and prior LNOSG assessments.

For clusters consisting of participants where Advanced Node Operators are 50%+ of the consensus threshold, validators limits would have the potential to be raised to operate a more meaningful number of validators (e.g. 100+).

The increasing of validators to these levels would move forward following sufficient performance monitoring during the initial stage of the Simple DVT Module and an extensive update would be provided to the DAO for discussion.

Simple DVT Module Economics

When considering the addition of the Simple DVT Module, the DAO should examine taking advantage of the ability to adjust rewards share parameters as described in the Staking Router documentation in order to increase the attractiveness for cluster members as well as to compensate the DVT providers for ongoing protocol development and services to the module. The Staking Router technical documentation outlines two reward share components: the module fee and treasury fee.

It is proposed that the total reward share for the Simple DVT module is 10%, with the DAO treasury fee set to 2% and the module fee (paid to Node Operators and the DVT providers) set to 8%. This differs from the reward share structure of the Curated Operator Module where the shares are split 5%/5%. Due to the economic differences in DVT clusters and the intent to run a smaller number of validators for a 2-3 year period, the Simple DVT module should share a higher percentage of rewards with Node Operators to reflect these differences.

Participating in a cluster of multiple members that split rewards for a small number of validators is of limited economic attractiveness. To promote the longer-term alignment of participating cluster members and to work towards profitability of the participating Node Operators, the module fee should be higher than that of the fee paid to members of the Curated Operator set, where they receive a higher individual fee over a larger number of validators.

In the case of DVT providers, as their respective DVT solutions are now coming to mainnet, the Simple DVT Module presents the opportunity to demonstrate alignment with the providers in supporting their continued services to the Simple DVT Module and long-term DVT protocol development.

Obol Network Economics

Obol Network will receive rewards via stETH as part of a splitter contract used for reward disbursement to Node Operators. Rewards received will represent 10% of the total share for Obol-based clusters (i.e. 1% of net Obol cluster rewards).

This disbursement method will not require any changes to the core module codebase, and will utilize an Obol Network developed splitter contract based on 0xsplits. This contract will be audited with results publicly shared before deployment on mainnet.

Node Operator participation on the Obol Network does not currently require service fees, so no additional tokens are involved in the operation of the relevant clusters.

SSV Network Economics

Usage of the SSV Network protocol requires Node Operators to pay SSV Network Fees and to deposit initial collateral denominated in the SSV token as outlined in the SSV Network technical documentation to operate validators using the protocol.

SSV Network will receive rewards denominated in the SSV token via SSV Network Fees. As outlined in the SSV [DIP-11] Mainnet Proposal, the SSV Network fee is currently 0.5% of Ethereum staking rewards. This will increase to 0.75% after 365 days pass from the launch of the initial configuration, and increase to 1% 730 days after the launch of the initial configuration.

Node Operators participating in SSV based clusters will receive an additional 1% of the staking rewards (8% total) that will be used to cover ongoing SSV Network Fees. In addition, pending the SSV DAO grants committee’s approval, a grant will be provided to a multi-sig of SSV and Lido DAO contributors that will be responsible for allocating SSV tokens to clusters for covering the costs of the liquidation collateral and SSV network fees required for the first year of module operation, not exceeding 2,500 SSV (equals an optimistic estimation of 50 clusters running 100 validators each). If approved by the SSV grants committee, after the first year of module operation, the combined multi-sig of SSV and Lido DAO contributors will return any excess SSV tokens to the SSV DAO treasury.

Members of this multi-sig would be determined pending the approval of the SSV grants committee to process the grant, following the processes described in the Lido DAO Ops Multisig Policy, i.e. it would include at least 3 signers with a 50%+ threshold, have a dedicated forum post containing a stated purpose and operating rules, include the wallet address and participant addresses, and each participants would “apply” by sharing proof of address ownership. This forum thread will be created by the Lido DAO Operations Workstream and be cross-linked to the Simple DVT Module proposal thread.

This disbursement method will not require any changes to the core module codebase.

Simple DVT Rewards Share
Obol SSV
Total stETH Reward Share 10% Total stETH Reward Share 10%
Treasury share 2% Treasury share 2%
Obol share 1% SSV share* 0%
Node Operators share 7% Node Operators share* 8%

Note: Cluster Node Operators pay for SSV collateral and Network Fees via their SSV Operator accounts. First year SSV fees may be covered by the SSV DAO grants committee subject to the grant committee’s approval. SSV will receive rewards via SSV Network Fees.*

Associated Technical Risks & Mitigation

While Obol and SSV have both demonstrated significant progress and advancements in their implementation of DVT solutions, the technology is not yet battle-tested.

Potential issues across areas such as networking, execution layer and consensus layer client compatibility, latency, DVT clients or middleware, etc. all present possible pitfalls of a DVT deployment.

In addition to these risks, the DAO should consider that Obol and SSV are still in the mid-to-early stages of the audit process. SSV has undergone an initial audit of the core contracts and client, and Obol is in the process of starting its second set of audits over the remainder of the code base (following a Sigma Prime audit of the Charon client). Both DVT providers are in the process of additional audits and have indicated that they will continue to share updates publicly.

While the numerous benefits of such a deployment have been outlined above, these technical risks are worth consideration.

As a possible mitigating factor to these risks, this proposal seeks DAO consideration of the following options: 1. The existing cover fund (or a subsequent cover provision) be used in the case of slashing or otherwise above-normal reduced rewards to Lido stakers or 2. Sourcing third-party cover via e.g. Chainproof or Nexus Mutual for the validators run as part of this module.

During the discussion period of this proposal, DAO members are encouraged to discuss their preference among these options.

Option 1: Use the existing cover fund to mitigate risk

With 6,200 stETH currently in the cover fund vault contract, the risk to Lido stakers can be mitigated in all but a catastrophic slashing event (that would have much further impact) or if all clusters were to go offline permanently (unlikely considering the curation process for this proposed module, and the expected rollout of triggerable exits (EIP 7002) within the next year).

Per research from the Lido Analytics Contributor Workstream and as an example assuming 1400 validators (70 clusters running 20 validators each), the cover fund currently holds more than enough stETH to compensate for potential losses under most conservatively realistic scenarios, even assuming intentional malicious behavior of all participating NOs: (i.e. no more than 4 ETH loss per Validator, assuming no correlation penalty and triggerable exits implemented within the next year).

Considering that a correlation penalty has to date never been assessed in any Ethereum network slashings and that clusters would be made up of Node Operators with demonstrated experience during testnet DVT trials, the scenarios that would exceed (or impact over 6,000 stETH) are extremely unlikely.

Of note, a discussion regarding a longer-term Surplus Management Framework has begun, which over time could provide additional cover via DAO revenue.

Option 2: Source third-party cover

An open RFP process is held to source cover via a third-party provider for up to e.g. 4 ETH per validator. This process includes public proposals for the DAO to consider from these providers, and if this option is chosen, a subsequent DAO vote to select the provider.

Two parties interested in providing third-party cover, Chainproof and Nexus Mutual, have responded to the forum thread sharing proposals to offer slashing cover for the Simple DVT Module.

The Lido Analytics & Finance contributor workstreams have also presented detailed analysis of the proposed options for the DAO to consider. Voters are encouraged to review the proposals and analysis within the forum thread before participating in the Snapshot vote.

Proposed Plan of Action

The proposed plan includes the following steps:

  1. DAO discussion of the proposal and consideration of the Risk Mitigation measure to utilize if the proposal is accepted
  2. Snapshot vote to signal DAO support for advancement of the Simple DVT module and choice between utilizing the Lido Cover fund or third-party cover
  3. Deployment of a secondary module & related tooling on Holesky testnet (not dependent on Snapshot vote)
  4. Third set of DVT testnet trials with Obol & SSV (not dependent on Snapshot vote)
  5. On-chain DAO vote for deployment of secondary mainnet module, role delegation for the Simple DVT Module Committee, and the deployment of related tooling
  6. LNOSG evaluation & review of testnet participants to propose for mainnet DV cluster creation
  7. Simple DVT Module Committee creation of mainnet DVT clusters up to 0.5% of total Lido validators (with option to increase this threshold of total Lido validators over time)

While subject to change, DAO contributors estimate Step 3 could take place by mid-to-late October, with the third round of DVT testnet trials being held in November/early December.

Continued community feedback on the proposal and potential risk mitigation methods is crucial. Please feel free to share any questions or comments below. This is the final version of the proposal and will be put forth for Snapshot vote on October 26th, 2023.

7 Likes

Snapshot vote started

The Staking Router Module Proposal: Simple DVT Snapshot has started! Please cast your votes before Thu, 02 Nov 2023 17:00:00 GMT :pray:

4 Likes

Hello LIDO Community,

Following our recent engagement and after taking into account the constructive feedback, we have fine-tuned our insurance offering. We aim to strike a harmonious balance that both represents the best interests of LIDO and resonates with our core principles at Chainproof.

The primary concern we gleaned from our discussions was the need for a structure that genuinely captures the risk-reward dynamics pertinent to LIDO. With this in mind, we revisited our strategy and crafted an updated offer tailored to LIDO’s unique requirements.

Our renewed offer

  • Our renewed proposal recommends an insurance coverage sum of 2.8m USD (approx. 1550 ETH) which covers the most likely scenario 1 mentioned in the excellent post by Mol_Eliza.

  • To render the insurance premium more economical, the revised offer incorporates a 10% Deductible. This implies that the initial 280k USD (approx. 155 ETH) of any potential losses would be borne by the pool.

  • The insurance premium for this policy would be 100k USD (approx. 55 ETH) regardless of the number of active validators. Also, the premium may be reduced on a proportional basis with respect to the sum insured, but it may not be increased.

Given the cost-benefit analysis presented previously, we believe this to be the most cost-effective option that the DAO can opt for, and covers the majority of risks that the DAO aims to cover. We believe that any additional coverage would be extraneous and have diminishing returns.

We also want to offer more transparency regarding important aspects related to this offer:

1. Ease of Use

At Chainproof, we understand that the rapidly evolving world of web3 requires streamlined processes. This is why we’ve simplified our insurance acquisition process for the Lido DAO.

Insurance Document: We provide insurance as a straightforward legal contract signed between Chainproof and the ultimate beneficiary (the DAO, or its representative(s)).

Signature Process: Onboarding is as effortless as signing a document using DocuSign. A sign-off by a KYC-verified representative of the Lido DAO, and you’re set.

2. Claims Assessment

We respect the governance structures of DAOs and offer a claims assessment process that’s both fair and transparent.

Assessment Period: Our claims assessment period aligns with industry standards (5-7 days). This ensures a thorough investigation and fair outcomes as the post-mortem report put together by experts at Quantstamp will be shared with the insured and publicly.

Dispute Resolution: Should the Lido DAO ever disagree with our assessment, legal recourse is available. We uphold the DAO’s right to legal action if needed.

3. Claims Payout

Quick response during challenging times is our assurance.

No Cooldown: We commit to no Cooldown period. If a claim is approved, the transfer begins immediately.

Direct Transfer: Payouts will be made directly in USDC to the wallet specified by Lido DAO in the policy document.

4. Trusted Partnership

Our reputation is built on the trust of our partners:

Regulated Entity: Chainproof is a regulated insurance entity, ensuring an additional layer of trust.

Strong Partnerships: Our associations with Sompo and reinsurance provided by Munich Re stand testimony to our commitment. We have marquee customers such as Kiln, who are a Lido node operator, as well as Luganodes who are a top 15 global validator.

Previous Collaborations: Our parent company, Quantstamp, was an early validator of Lido’s robustness in 2020. See the audit here.

Incident response capabilities: As we develop an intimate knowledge of the underlying protocols such as Obol and SSV, we are happy to extend incident response assistance in the event of a slashing incident. Chainproof will be able to go above and beyond the role of an insurer and help with fixing underlying technical issues to prevent the risk of future incidents.

5. Coverage Details

Just like Nexus, our policy covers the direct loss of ETH to slashing (i.e. 1 ETH, or greater if correlated), and the penalties that are directly associated with the slashing event. It does not cover penalties that are unrelated to slashing, and it does not cover loss of rewards.

Thank you for your continued trust and collaboration.

5 Likes

Hey there!
I really appretiate @chainproof for flexible approach and precisely formulated proposal.
Want to share my view on new option, expanding a bit approach i used in a comment above.

Coverage
In terms of coverage new option (in my opinion) is enough for most realistic catastrophic incidents: 1550 ETH is well enough to cover slashings of all validators (1378 ETH initial + attestation penalties)

For minor / medium incidents with 10% deductable risks are not covered.
With 1.075 ETH per slashed validator in initial + attestation penalties, incidents for 144 Validators or less are not covered

Premium

  • Risk-event mitigated: slashing of > 144 validators through a year
  • Impact covered: 155 - 1550 ETH
  • Premium: 55 ETH

Excepcted (statistically) impact could be valuated as
(probability of >144 slashings) X E(Impact | >144slashing), where
-First component is probability of risk-event
-Second is conditional expected value of random variable - Impact

Even in most extreme case: for second component valuated at it’s maximum value (1550 ETH)
Risk-neutral probability valuation of risk-event would be: 1550 / 55 = 3.55%

From module sustainability perspective, for 41k staked ETH and assumption on 4% APR total rewards
Total rewards for module (1 year) = 1 640 ETH
Cost of risk = 55/ 1640 = 3.35%

While relatively high premium value is perfectly reasonable in terms of high level of uncertainty, as DVT module is definitely

unknown unknowns, due to the new codebase which hasn’t been battle tested like the current, trusted Lido staking module

The level of uncertainty is expected to drastically drop: with extensive testnet trials and, finally, starting operation on mainnet.
Therefore, for me, at this point in time those probability estimations could be reasonable, but for a first starting months.
In terms of 1 year risk mitigation strategy it’s still preferable to use the existing cover fund.

Thank you for a constructive and insightful discussion.

5 Likes

agree with all of the points highlighted here, the insurance offer does not currently justify the bottom line expense and any other legal entanglements that could arise.

just to reiterate to the public (not a lawyer btw*) that Lido DAO is not a legal entity or have any common legal agency-- which to me could become a problem if there is an apocalyptic slashing scenario followed by a liquidity crunch and drawn-out grey-zone legal policy battle could make for having insurance ironically causing less peace of mind.

The deals appeal just doesn’t overcome the concerns. Possibly a go once DVT is a more tested technology and the externalities are better quantified for insurance companies to offer better rates. I appreciate all of the work the insurance providers put into compiling their offers!

3 Likes

Hi all,

We wanted to flag two DVT specific episodes from the Lido Community Staking Series that cover DVT Validator setup that should be helpful to community members that are interested in participating in the 4th round of DVT testnets (planned to go live in early Q1).

Community Lifeguard @Eridian hosted two workshops covering the process to set up both Obol and SSV Validators.

Check out the videos here:

As a reminder, the main requirement to participate in a Lido related DVT testnet is prior experience running an Obol or SSV based validator on testnet. If you’re interested in joining the next round, make sure to watch these videos and get a DV based validator setup before applying!

Lido DAO contributors will reach out in early 2024 to applicants and share next steps related to joining the 4th round of testnets.

Once your DVT based validator is running, fill out this form to apply to the next testnet: https://forms.gle/bB4khwE7hLgNpSN58

All solo stakers, community stakers, and professional Node Operators are more than welcome to apply!

In the coming days we will share an update regarding the status of the 3rd round of DVT testnets in this thread.

8 Likes

Anticipating the mainnet launch of the Simple DVT module, Lido contributors invite proposals from software providers specializing in node/validator setup to enhance their existing tools by incorporating a user-friendly pathway for community stakers to engage with the Simple DVT module. For more details follow the Request for Proposal | CSM and SDVTM integration.

5 Likes

Hello everyone, the following post contains an update on the status of the ongoing Simple DVT testnets with Obol and SSV Network. It also contains additional information related to the processes currently being used on the Holesky testnet, which are intended to be replicated on mainnet.

One recurring challenge faced, that will be mentioned in this update and discussed more extensively in future blogs posts, was an underappreciated lack of battle-tested community infrastructure for the Holesky testnet at the beginning of both the Lido Obol and SSV testnets. This includes key infrastructure such as SAFE, MEV-Boost, and relay infrastructure that, while well-established on the Goerli testnet, were only deployed on Holesky near the same time the trials began (some of which was supported by LEGO in order for it to happen), causing some issues.

Nevertheless, the overall testnet process to date for both Obol and SSV has gone smoothly (or as smoothly as coordinating with hundreds of participants running 448 nodes can be); with hundreds of individuals and organizations participating, there’s clear enthusiasm to move forward to mainnet.

The Simple DVT module represents the fastest way to enable hundreds of net-new Node Operators to use the protocol in 2024. Should the DAO vote to deploy the module to mainnet in Q1, the protocol could see 100+ net new Node Operators using the Lido protocol to run validators in less than three months.

As a reminder, the Simple DVT Module Proposal calls for a 30 - 45 day monitoring period. For both the Obol and SSV tesnets, the best consecutive performance of 30 - 45 days starting from the activation of validators or for the time period prior to the exit of validators will be used as the performance result and compared to the outlined benchmarks of Uptime of at least 95%, Block Proposal Success Rate of 70%, and Attestation Effectiveness per Rated Network comparable of that with the broader Holesky validator set.

Obol

The Obol testnet began on November 3rd comprising 32 clusters with a 5/7 threshold configuration, with a total of 214 total participants.

In some cases, cluster participants were replaced due to backing out or lack of adequate presence. After going through the validator setup process and running the nodes for ~8 weeks, a total of 196 individuals and organizations are expected to have participated in the full testnet process. This near final count of participants includes over 100 solo and community stakers.

Each of the 32 clusters initially started with 5 validators and, after a performance monitoring period, key limits were raised to 50 validators each. During the subsequent monitoring period, it was observed that while Uptime and Attestation Effectiveness were well above the network benchmarks, the Block Proposal success rate was lagging below the 70% threshold set during the original module proposal. At this point, analysis of the missed proposals identified four issues: 1. General NO misconfigurations, 2. Cluster latency, and two issues related to the early-stage nature of Holesky deployments, 3. Issues with Holesky relays, and 4. Lack of MEV-Boost support.

Shortly after the commencement of the testnet, the Obol team deployed an upgraded version of Charon, v0.18.0, that included support for MEV-boost (vs. prior direct connection to relays via the CL client) and began troubleshooting with the relay teams.

Concurrently, one of the clusters, Glacial Gull, had two members leave the testing round. It was determined at that time that the validators from the cluster should be exited and re-created via DKG with replacement members. The Crimson Coyote cluster also had a member leave the testnet, though the cluster continued to run with 6 participants.

Following a Charon upgrade which improved Block Proposals success rate, and further performance analysis, the validator limits for the vast majority of clusters were raised to 100 validators, while the Glacial Gull and Crimson Coyote clusters remained at 50 validators.

Over the course of December and early January, improvement was observed in the aggregate Block Proposal Success Rate, and for the 45 day monitoring period of November 28th - January 11, the metric stands at 71.2%. In the 7 days prior to this post going live, the daily aggregate Block Proposal Success Rate has remained at 85% or above for 6/7 days.

The first Obol cluster was exited on January 12th, and the remaining Obol testnet participants will begin exiting validators and testing the staking reward claiming process later this week.

An extensive update regarding the trial will be posted on https://blog.lido.fi/ in the following weeks.

SSV Network

The SSV testnet began on November 22nd and includes 32 clusters with a 5/7 threshold configuration, and a total of 192 participants.

During the end of November and early December, clusters coordinated to create Holesky SAFEs representing their clusters in the Simple DVT Node Operator Registry and participants set up their SSV Operators and DKG nodes. Unfortunately, due to infrastructure and tooling issues on Holesky due to its recent deployment, an issue was discovered between Holesky SAFE, Walletconnect on Holesky, and the SSV Webapp. Due to the timing of the holidays, the testnet was put on pause until January 3rd. As of today, 175 individuals and organizations are participating in the resumed testnet.

Over the break, SSV deployed an upgrade to their DKG tooling that significantly improves the efficiency of the process for participants. Over the following weeks, Node Operators updated their DKG nodes and clusters began the DKG process and key submission to the Simple DVT Node Operator registry for their initial 5 validators. So far 26 of 32 clusters have had 5 validators deposited to, and the remainder are expected to be running by the end of the week.

More information regarding the SSV testnet and cluster performance will be shared in the coming weeks, and an extensive blog post will be published at the conclusion of the testnet.

Simple DVT Process Updates

Simple DVT Module Committee

As described in the Simple DVT Module proposal, the “Simple DVT Module Committee” is a multi-sig committee responsible for creating and executing Easy Track motions specifically for Simple DVT that can create new clusters, activate and deactivate existing clusters, raise and lower cluster key limits, and change cluster manager and reward addresses.

The Simple DVT Module Committee will be made up of contributors from the Lido DAO, Lido Node Operator Sub-governance Group, Obol, and SSV Network DAO. The multi-sig is expected to consist of: two Lido DAO contributors, three LNOSG contributors, one Obol contributor, one SSV contributor. In the coming weeks, the participants for this multisig will be shared according to the processes outlined in the Lido DAO Ops Multisig Policy.

The Easy Track optimistic governance process allows for streamlined execution of processes related to the operations of what will potentially be 40-80 mainnet Simple DVT clusters in 2024. On mainnet, a 72 hour objection period exists for LDO voters to veto any active motions related to these cluster operations by using the Simple DVT EasyTrack smart contract, with an optional UI also available.

Simple DVT Reward Distribution

Each cluster participant submits their “Individual Manager Address” (1) and “Individual Reward Address” (2) during the cluster setup phase, from which (1) cluster specific multi-sigs representing their Cluster Node Operator entries are generated and (2) reward splitter contracts are deployed.

To simplify the reward claiming process for Simple DVT Node Operators, both the Obol and SSV Network clusters will utilize two sets of smart contracts: 1. a wrapper developed by Obol that wraps stETH rewards into wstETH, charges an (optional) fee, and sends the remainder of the wstETH to the next smart contract, 2. An 0xsplits set of smart contracts that allows for the distribution of rewards between participants. The Obol wrapper contract was audited by a solo auditor (obol-splits/audit at main · ObolNetwork/obol-splits · GitHub), and the Obol team plans to undergo an additional audit of the contract in the coming months.

A wrapper contract is created via the ObolLidoSplit factory and this contract is specified as each cluster’s reward address. This contract is responsible for wrapping stETH rewards to wstETH and when applicable, gives the DVT provider their reward share. This contract can be called by anyone, and will transfer wstETH to a split contract created via another factory (the main 0xsplit contract), from where each individual cluster participant can claim.

During the third round of testnets to date, the Obol team has deployed the splitter contracts for the Obol testnet, and Lido DAO contributors have deployed the contracts for the SSV testnet.

Additional updates regarding the status of the testnets, cluster performance, the Simple DVT Module Committee, and reward distribution will follow in the coming weeks.

Finally, the fourth round of Lido DVT Testnets for interested participants of the mainnet Simple DVT module is expected to commence in late February/March. If you are interested in participating, please fill out this form: https://forms.gle/Akq7poKXF7p7x6QU9

18 Likes

Since my last update, the SSV network has successfully transitioned to a permissionless network. To support this effort, the SSV smart contracts have undergone updates, most notably the removal of their previous permissioned restrictions, enhanced support for in-protocol validator exits and addressed bug fixes identified through the ongoing bug bounty program.

As a result, a new audit have been issued and completed, and full audit reports could be accessed here .

With this update, the SSV network renews their commitment to their outmost transparency to keep the Lido community informed by sharing each new audit report, not only for this recent change but also for any future updates throughout the life cycle of the Simple DVT module clusters.

Furthermore, there are additional updates in the work, primarily focused on enabling bulk operations for validators, such as registration, removal, and exits. Once these updates are finalized, SSV would conduct a new audit, and the results will be shared right here.

5 Likes

We’re pretty excited to share some thoughts following the recent conclusion of the first wave of the Obol SimpleDVT testnet. It’s been a rewarding journey, and we’re happy to see we hit our performance targets. We’ll be releasing an in-depth report soon highlighting some of the issues faced and here is a quick overview:

  • Node Configurations: Charon, as a middleware, offers unparalleled composability. We’re pleased to confirm that Charon itself exhibited no inherent flaws. However, the testnet highlighted challenges in node configurations, stressing the need for operators to tailor these settings instead of blindly relying on our template demo repo. Our commitment to decentralization means empowering operators with the knowledge to configure their nodes effectively, preventing Obol from becoming a centralizing force. We will intensify our efforts to educate the market on optimal configurations and invest in diversifying DV setups as much as feasible.
  • Latency Challenges: We observed latency issues, particularly for operators in Asia and Australia proposing blinded blocks. We plan to provide detailed guidelines on acceptable latency levels to optimize operator pairings based on geographical proximity. We also noticed that solo stakers sometimes faced additional latency challenges, possibly due to ISP limitations.
  • MEV-Boost and Relay Integration: The integration of MEV-Boost and relays, particularly with the new and evolving support on Holesky, was an area of active experimentation. While a detailed report will follow, preliminary findings indicate potential configuration optimizations to mitigate latency. The disparity in relay proximity and performance also offered insights, which will be crucial as we refine DVT’s synergy with MEV in a more developed and stable mainnet environment.
  • Hardware Considerations: Holesky’s high validator count revealed that some participants’ hardware (particularly those using virtual cloud servers and those using HDDs rather than SSDs) was not sufficient, contributing to syncing difficulties, latency and missed duties. This aspect will be an important consideration in future deployments.
  • Optimizing the Lido Exit Sidecar: The testnet was the first deployment of the Lido exit sidecar which yielded significant optimization learnings, which we intend to apply not only to this specific use case but also to generalize for broader pre-signed exit use cases.
8 Likes

Just bumping this as the interest form will close soon!

4 Likes

Hi everyone! The next step for the Simple DVT module proposal is on-chain voting.
A separate post provides a comprehensive overview of all the technical aspects related to this release:

On-chain vote to be announced soon, please stay tuned for updates :grinning:

7 Likes

The NOM contributor workstream on behalf of the Lido Node Operator Subgovernance Group (LNOSG) would like to provide an update regarding the results of the Obol Simple DVT testnet participant evaluations that took place on February 20th, 2024. This evaluation meeting followed the conclusion of the Lido x Obol testnet, wherein Obol’s DVT implementation surpassed all of the minimum performance requirements outlined in the Simple DVT Module Proposal.

The next round of Simple DVT testnets are expected to begin in late March. All solo stakers, community stakers, and professional node operators are welcome to join. See here for more information.

Obol Testnet Participant Evaluation

A total of 214 operators, including 40% of which were solo stakers, participated in the Lido x Obol Simple DVT testnet in November 2023. These participants were sorted into 32 clusters of 7 participants each, with various types of infrastructure configurations and geographic locations. Due to a lack of responsiveness or issues with node configuration, a total of 195 participants completed the testnet from start to finish and were subsequently evaluated by the LNOSG. For more information regarding the testnet process and the results, see the Lido blog or Obol blog posts.

Evaluation Results

During the evaluation discussion, the LNOSG together with testnet coordinators from the Lido DAO NOM Workstream and Obol Labs suggested 161 individual participants across 26 clusters with a 5/7 threshold configuration be onboarded to mainnet Obol clusters in the Simple DVT Module over two Cohorts. The proposed participants include 70 solo stakers, 28 community stakers, 41 professional node operators, and 22 members of the Lido on Ethereum Curated node operator set. Of these participants, 21 were deemed to have shown exceptional performance and are proposed to participate across two clusters.

The candidates proposed by the LNOSG were considered to have shown strong performance, responsiveness, and reliability over the course of the 69 day Lido x Obol testnet. These candidates operate nodes out of 36 countries, utilize diverse infrastructure hosting setups including on-premise, bare metal in a datacenter, and public cloud, and support a wide range of both EL & CL clients.

The compositions of the 26 proposed clusters are designed to minimize latency and support infrastructure diversity. Additionally, all clusters have participants that support a range of EL & CL clients.

The proposed participants have been asked to confirm their desire to join the Simple DVT Operator Set via an Obol cluster. In case they do not reply affirmatively in a timely manner, or for some other reason cannot participate, the LNOSG will make an adjustment to the proposed list and share an update in this thread.

During the final meeting on February 20, 2024, the LNOSG suggested to propose the following Obol based clusters for inclusion into the Lido Simple DVT Module Ethereum protocol:

You can find the full PDF with summary statistics and the LNOSG meeting recordings of the introduction, as well as the audio (video unavailable) for the final conclusion and cluster proposal in the public drive below. Due to the privileged/confidential nature of some of the information provided in some of the applications, the portion of the call where applications were discussed in detail was not recorded.

https://drive.google.com/drive/folders/1I1bEvJXn2b4FPIi1PBrH3UInbM8wb3eU

All participants in the testnet have been emailed notifying them of the results and are welcome to reach out via email to [email protected] for further clarification.

Path to Mainnet

The Simple DVT Module is expected to be deployed to mainnet this week as the on-chain vote has reached quorum (objection period still pending).

Pace of Node Operator Additions

The Simple DVT Module Proposal outlines an initial mainnet stage where 12 clusters for both Obol and SSV Network undergo a 30 day monitoring period with 5 validators.

Clusters 1-12 of the proposed list constitute Cohort 1, while Clusters 13-26 constitute Cohort 2. After the initial 30 day monitoring period, a performance report will be presented to the DAO. If the performance is comparable to other mainnet operators, a proposal will be made to begin the process of adding clusters from Cohort 2 to mainnet and to potentially raise key limits for the existing clusters. The addition of Cohort 2 clusters are not all guaranteed to start at the exact same time, as the pacing may require additional monitoring time.

DAO Discussion Period

A 7 day discussion period is now open for the DAO to discuss the proposed participants for the initial Obol clusters within the Simple DVT module.

Should the DAO disagree with any of the proposed applicants, request additional information, or otherwise request additional input for the onboarding, the LNOSG will reconvene to propose a next course of action.

If after 7 days, no major changes are required by the DAO, the Simple DVT coordinators from the Lido DAO NOM Workstream and Obol Labs will begin the coordination process with cluster participants.

Next Steps

If no adjustments are required after the week discussion period, the Obol clusters from Cohort 1 will be added to coordination threads and begin the process of verifying addresses and setting up cluster multisigs. The Lido Simple DVT Module Committee will create their entries in the Simple DVT Node Operator Registry via Easy Track, however key limits will not be raised above 0 until Cohort 1 SSV clusters are also prepared to run validators.

The Lido x SSV Network Simple DVT testnet is currently underway and expected to be complete in the next two weeks. Upon the conclusion of the testnet, a follow up blog post presenting cluster performance will be published, and an LNOSG evaluation for SSV participants will take place.

An update will be posted to the research forums before validator key limits are raised for any clusters.

Future Simple DVT Testnets

Another round of Simple DVT testnets are expected to begin in late March. All participants that were not proposed to participate in Cohort 1 or 2 of the Obol trial are welcome to participate, in addition to any other solo stakers, community stakers, or professional node operators. The testnets will be followed by another LNOSG evaluation, whereafter additional participants can be proposed to move forward to mainnet.

Sign up here if you are interested in participating!

7 Likes

Congrats to all participants. We are happy to be part of shortlisted candidates in cohort 2. I hope everything will go as planned, and cannot wait to get started. Shout out to all testnet participants, and especially to our Brisk Beavers cluster :saluting_face:

4 Likes