Proposal: Curated Module Intra-Operator DVT Guidelines

Summary

With the increasing adoption of Distributed Validator Technology (DVT) on Ethereum, discussion has increased within the Lido Node Operator set regarding how Curated Node Operators might start to utilize the technology.

In light of the adoption of DVT across Ethereum, its strong performance both in SDVT and CSM, as well as in other staking protocols, NOM workstream contributors would like to propose that:

  1. Node Operators in the Curated Set may opt-in to intra-operator DVT setups (utilizing either Obol or SSV)
  2. Each Node Operator in the Curated Set can utilize key splitting for up to the lesser of 1,000 or 20% of existing active validator keys, after following a testnet process via the Lido Hoodi instance
  3. Node Operators in the Curated Set may add any number of additional DVT-based keys to their depositable keys in the Curated Module, provided that these keys are generated via a verified Distributed Key Generation (DKG) process, and the process for which key shards are created, stored, and used has been vetted by NOM workstream contributors
  4. Regarding DVT provider incentives for any validators operated in this manner in the Curated Module, Node Operators would receive 20% of DVT provider incentives pertaining to said validators, with the other 80% to be directed to the Mellow Decentralized Validator Vault.
  5. Participants in the Community Staking Module with the Identified Community Staker Node Operator type would have their DVT provider incentive share increased to 25% from 20% for validators operated via CSM, with the other 75% to be directed to the Mellow Decentralized Validator Vault.

This post also contains a detailed update on the status of DVT by Node Operators using the Lido protocol and across the network, as well as a performance report of larger non-Lido scaled clusters, and a more detailed proposal on the above, which, should the community be generally supportive of this initiative, would serve as the basis for a Snapshot for the DAO to decide on the matter.

DVT Performance Overview

Today, over 135,000 validators active on Ethereum mainnet are utilizing DVT, with 120,000+ utilizing SSV, and another ~ 16,000 utilizing Obol’s technology. Both providers are growing quickly, while at the same time demonstrating strong performance and many of the inherent benefits that DVT provides (e.g. infrastructure resilience and decentralization).

Within Lido, the Simple DVT Module is the highest performing active Staking Router Module, with an all time Rated Validator Effectiveness Rating (RAVER) of 97.5% vs. the overall network at 96.72%. In addition, the module has a 99% Block Proposal Success rate over the last 30 days, with over 1,000 proposed blocks. Notably, this includes 82 active clusters and 323 distinct Node Operators consisting of over 180 home stakers, in addition to professional operators.

The Simple DVT Module has 9,635 active validators, representing 3.40% of all Lido stake, and when it reaches its current stake share limit of 4%, will represent over 1% of all staked Ether. The SDVTM currently has 41 active Obol clusters (including 5 Super Clusters with a limit of 500 validators) and 41 active SSV Clusters (also including 5 Super Clusters).

In addition, within the Community Staking Module, nearly 1,300 validators are currently utilizing DVT, with over 1,100 SSV and 100 Obol based validators active via the module.

In order to examine the performance of distributed validators at scale, analysis was performed on the performance of eight clusters operating outside of the Lido protocol: four utilizing Obol’s and four utilizing SSV’s DVT. Of the Obol clusters, three currently operate over 2,000 validators, and one with 1,000. For the examined SSV clusters, they operate a range of 640 - 1000 validators. Currently SSV clusters can operate no more than 1,000 keys, after a recent technical update by SSV that increased their limit from 500.

Obol Clusters

The following clusters demonstrate strong performance over the last 30 days, with improving trends for Block Proposals after adjustments earlier in the period due to 1. a client issue post Pectra and 2. an external MEV relay issue (missed block rewards were reimbursed). The first two clusters consist of 4 operators in a 3/4 threshold setup, with the second two consisting of 7 operators in a 5/7 setup. These clusters generally are more centralized than the clusters currently in Simple DVT, utilizing only professional operators and more strictly optimizing for performance via infrastructure and geographic location vs. decentralization.

With an average uptime above 99% as well as an above network average effectiveness rating, these clusters clearly demonstrate that at scale Obol clusters perform well.

From a rewards perspective, the 30 day APR was analyzed via Rated, including both Execution Layer and Consensus Layer rewards for clusters with a full 30 days since activation. Note that given the inclusion of EL rewards and recent market volatility, per entity APR can be impacted by large MEV blocks and make comparison difficult, however the aggregate APR for these four clusters are in a similar range to the Simple DVT Module at 3.04% while about in-line with the 30D network average at 3.10%.

Obol Clusters Validator Count Avg. Uptime Avg. Effectiveness 30D APR Successfully
Proposed
Total Proposed % Proposed
Obol Cluster One 2500 99.92% 97.99% 3.11% 520 520 100.00%
Obol Cluster Two 2500 99.80% 97.65% 3.00% 443 494 89.68%
Obol Cluster Three 1000 98.64% 98.01% 3.11% 205 218 94.04%
Obol Cluster Four 2287 99.97% 97.73% 3.15% 442 466 94.85%
Grand Total 8287 99.58% 97.84% 3.09% 1610 1698 94.82%

SSV Clusters

Similar to the Obol clusters, the SSV clusters also demonstrated extremely strong performance with all metrics exceeding the network averages over the last 30 days. The examined SSV clusters all operate in an intra-operator format, something more common for SSV vs. Obol given more usage of key splitting to date. Given the reduced latency in most intra-operator formats, it is to be expected these clusters would show outperformance vs. the more distributed Obol clusters above. In addition, it is worth noting all of these operators are professionals.

All of the four clusters achieved a 100% Block Proposal Success Rate, with these results again clearly demonstrating SSV clusters can perform well at scale, with metrics far exceeding the network average.

From a rewards perspective, it is again worth noting that the inclusion of EL rewards and recent market volatility can have an outsized impact on rewards rates. The aggregate APR for the SSV clusters held at 3.09% for the 30D period.

SSV Clusters Validator Count Avg. Uptime Avg. Effectiveness 30D APR Successfully
Proposed
Total Proposed % Proposed
SSV Cluster One 1000 99.99% 98.03% 3.05% 204 204 100.00%
SSV Cluster Two 800 99.99% 97.72% 2.98% 142 142 100.00%
SSV Cluster Three 986 99.98% 97.77% 3.11% 212 212 100.00%
SSV Cluster Four 640 99.99% 98.00% 3.22% 146 146 100.00%
Grand Total 3786 99.99% 97.88% 3.09% 704 704 100.00%

Proposal

Based on the above performance data from both Simple DVT running smaller and more decentralized clusters that have consistently outperformed the Lido and network averages, as well as the larger more centralized clusters outside of Lido with exceptional performance data, it is clear that there is an opportunity to drive higher performance for Lido based validators with the utilization of Distributed Validator Technology via the Curated Set, while continuing to provide strong rewards rates.

As such, it is proposed that as an initial step (and until module developments such as SSVLM occur) for those Node Operators in the Curated Module willing to adopt Distributed Validator Technology, they may do so in an intra-operator format (e.g. as the sole operator in the cluster) configuring their infrastructure as they see fit, while following the established policies in place (e.g. the Block Proposer Rewards Policy).

As consolidations are currently not yet supported by the Lido protocol (though will be in the future), it is proposed that Node Operators may utilize intra-operator DVT for depositable validators in the queue that have utilized a Distributed Key Generation ceremony, or for the lesser of up to 1,000 validator keys or 20% of active Lido validators via a key splitting ceremony following SSV and Obol’s respectively audited processes. Node Operators in the Curated Set may add any number of additional DVT-based keys to their depositable keys in the Curated Module, provided that these keys are generated via a verified Distributed Key Generation (DKG) process, and the process for which key shards are created, stored, and used has been vetted by NOM workstream contributors.

For Node Operators planning to utilize a key split, they must communicate with Lido Contributors and test the process via the Lido Hoodi testnet with sufficient time to monitor performance. Node Operators should also note, that if they decide to stop running keys that have undergone a splitting ceremony via either of the DVT providers, they must either exit the validator keys or utilize consolidations (once supported in the protocol). Node Operators should not re-use the previously utilized full private key after a key splitting ceremony.

Node Operators may also, if interested, coordinate cycling of existing validators to DVT setups with Lido contributors (however they would not be guaranteed to receive those keys as the ETH would follow the normal deposit and withdrawal cycle of the protocol).

Node Operators planning to utilize DVT are encouraged to use the opportunity to increase the resilience of their infrastructure by adopting new EL & CL client pairs, which would also benefit the decentralization of the network.

Incentive Reward Share

Given the ongoing incentives programs from Obol and SSV, there is an opportunity to drive growth to protocol via the Mellow Decentralized Validator Vault (DVV), in a similar configuration to how incentives are treated for the Community Staking and Simple DVT Modules.

Today, 90% of DVT incentives from the Simple DVT Module and 80% of incentives from the Community Staking Module are directed to the DVV. Similar to CSM, Curated Module (CM) Node Operators would be responsible for paying the Obol or SSV Network fees. As such, it is proposed that Node Operators would receive a 20% share of DVT provider incentives pertaining to validators run via the Curated Module, with the remaining 80% directed to the Decentralized Validator Vault.

In addition, with the introduction of the proposed Community Stakers Identification Framework, it is proposed that the DVT incentive share for Identified Community Stakers (ICS) be increased to 25% from 20%, incentivizing home stakers to utilize DVT while providing an additional benefit of utilizing the Identification Framework.

The Decentralized Validator Vault incentivizes the decentralization of the Lido protocol with deposits flowing to the Community Staking and Simple DVT Modules. In addition, because of the deposit flow, these newly created validators have a higher chance of being DVT based, also benefiting the decentralization and resilience of validators utilizing Lido. The incentives from the vault drive net-new growth to the Lido protocol as the DVV only accepts ETH deposits, rewarding users for the excess stake compared to their withdrawals from the protocol.

Mellow Points SSV Incentives Obol Incentives
1. DVV user share of incentives (SDVT validators) 100% 90% 90%
1. SDVT Cluster share of incentives (respective to number of clusters operated using specific DVT infra) 0% 10% 10%
2. DVV user share of incentives (DVT CSM validators) 100% 80% 80%
2. CSM Cluster share of incentives (respective to validators operated using specific DVT infra) 0% 20% 20%
3. ICS CSM Cluster share of incentives (respective to validators operated using specific DVT infra) 100% 75% 75%
3. ICS DVV user share of incentives (DVT CM validators) 0% 25% 25%
4. DVV user share of incentives (DVT CM validators) 100% 80% 80%
4. CM NO share of incentives (respective to validators operated using specific DVT infra) 0% 20% 20%

Risk Mitigation

While DVT has now shown to be appropriately battle-tested on mainnet given its extensive use, there are slight incremental risks during the key splitting process. However, this process has also been used widely on mainnet, with the vast majority of SSV’s 120,000+ mainnet validators utilizing this process, as well as an increasing number of Obol’s 16,000+.

As such, this proposal calls for no more than the lesser of 1,000 keys or 20% of active validators from a given Node Operator to undergo a key splitting process via the Lido protocol.

Currently, the Lido DAO cover fund contract holds over 6,500 stETH. In the event of a slashing incident for a single Node Operator as a result of key splitting, research from the Lido Analytics workstreams indicates that it is estimated that 279 ETH would be at risk.

While it is extremely unlikely that multiple Node Operators would face a simultaneous slashing incident due to key splitting given their diverse key management practices, however even in that scenario up to 7,500 validators could be slashed before exceeding the balance of the cover fund.

Summary

Given strong performance from mainnet DVT usage within Lido and across the network, as well as the opportunity to drive growth to the protocol with DVT incentives via the Mellow Decentralized Validator vault, it is proposed that:

  1. Node Operators in the Curated Set may opt-in to intra-operator DVT setups (utilizing either Obol or SSV)
  2. Each Node Operator in the Curated Set can utilize key splitting for up to the lesser of 1,000 or 20% of existing active validator keys, after following a testnet process via the Lido Hoodi instance
  3. Node Operators in the Curated Set may add any number of additional DVT-based keys to their depositable keys in the Curated Module, provided that these keys are generated via a verified Distributed Key Generation (DKG) process, and the process for which key shards are created, stored, and used has been vetted by NOM workstream contributors
  4. Regarding DVT provider incentives for any validators operated in this manner in the Curated Module, Node Operators would receive 20% of DVT provider incentives pertaining to said validators, with the other 80% to be directed to the Mellow Decentralized Validator Vault.
  5. Participants in the Community Staking Module with the Identified Community Staker Node Operator type would have their DVT provider incentive share increased to 25% from 20% for validators operated via CSM, with the other 75% to be directed to the Mellow Decentralized Validator Vault.

Node Operators in the Curated Module interested in utilizing DVT in an intra-operator setup are encouraged to respond to this thread, noting experience with DVT and how they would plan to incorporate it into their existing infrastructure setups.

A discussion period is now open for the community to consider this proposal, and following feedback and any potential interactions, is suggested to move forward to a Snapshot vote.

14 Likes

The success of the Simple DVT module in bringing hundreds of net new node operators to the Lido protocol, and ushering the first widespread use of DKG to significantly reduce slashing and operational on mainnet is something to be lauded.

In general, I think it makes sense to allow Curated NOs to utilize DVT in a manner that also reduces risk and adds diversity and robustness to their setups. Given that 0x02 consolidations aren’t yet available, using the existing-key-sharding approach seems like a relatively low risk approach to allow this to happen in the short term, especially since it has already been widely used by other staking protocols that (unfortunately) didn’t really care about utilizing DKG. By maintaining a relatively safe amount of active keys separate between “sharded” DVT and non-DVT, the potential risk is quite low, and certainly more robust than in other staking providers who haven’t considered such safeguards.

7 Likes

Thanks for sharing this detailed overview of DVT adoption within Lido and the broader Ethereum network. We are pleased to see the strong performance data for Obol-powered clusters and support the proposed guidelines for DVT adoption in the Curated Module.

The performance analysis confirms what we have been seeing across our network - distributed validators not only match but often exceed network average metrics. We are also pleased to see these results across different configurations. The professional deployments shown in this forum post comfortably run 2,500+ validators across 7 operators, setting a record for validator count per DV cluster. We have also seen great success with decentralized clusters in the Simple DVT Module that include home stakers. This demonstrates the versatility of the technology in various operational environments. Notably, throughout this time, Obol DV clusters in the Simple DVT module continue to perform excellently and rank at the top of the Simple DVT module per RAVER score.

Prior to Pectra, our Obol Clusters consistently ranked among the top performers in the EtherFi Permissioned Set. The 2,500-key, 7-node cluster notably outperformed many clusters operated as single operators, demonstrating our capability when systems operate as designed. The temporary performance decrease observed during the Pectra upgrade was caused by Ethereum’s protocol changes to attestation data structures, which created key collisions in Charon’s aggregation database when multiple validators aggregated in the same slot and affected large clusters with >fault tolerance threshold of Lodestar VCs. This incident has been resolved and led to enhanced testing protocols, improved monitoring systems, and client diversification strategies that have strengthened our operational resilience going forward.

The performance data clearly demonstrates that DVT technology has matured to the point where it can be confidently deployed within Lido’s Curated Module. We are ready to support Node Operators in their DVT adoption journey.

The Obol Team

6 Likes