ZK Lido Oracle powered by Succinct

Proposals
1

By: John Guibas (Succinct)

TLDR; We’re excited to share with the Lido community that we have a working POC of correctly calculating total Lido reserves, number of validators, and number of exited validators in ZK across all ~1M validators on Goerli based on Mainnet data. The code is open-source here and we have succesfully verified the proof on Goerli for ~300k gas.

Using the Succinct SDK, we were able to develop the core logic in around ~130 lines of rust and ~100 lines of solidity and quickly deploy and test the system onchain.

In this proposal, we outline a roadmap for the Succinct team to become another ZK oracle provider as part of Lido’s initiative to augment/harden the security of the core protocol.

Context

Currently, the Lido Accounting Oracle contract is used for getting key consensus and execution layer information to the Lido protocol, such as the balances of Lido-participating validators. The oracle relies on a third-party quorum set, which is a 5/9 multisig, to attest to this information.

The Lido Community has already funded two similar proposals to explore the feasibility of partially replacing the oracle with more trustless methods:

Multiprover

Because ZK technology is new, a ZK-based oracle should be used to augment the security of the existing oracle, rather than completely replacing it. As a result, it makes sense to add several layers of redundancy by having several ZK oracles based on different implementations reporting in parallel.

With this type of approach, if there is a bug in a single implementation, it is unlikely for the same attack to work on the other approaches. Because of this, we think it is in Lido’s interest to continually support work that aims to build multiple different implementations of a ZK Lido Oracle.

Succinct SDK

Our solution is implemented on top of the Succinct SDK, an open-source and modular toolkit for rapidly building and deploying production-grade ZK applications for teams that are not ZK experts.

The SDK addresses following problems when using ZK in production systems:

  • Reading inputs and writing outputs to and from Ethereum
  • Support for EVM types and data structures
  • Deploying verifiers on the EVM
  • Versioned upgrades and previews on testnets
  • Running off-chain actors such as watchers, provers and relayers
  • Cloud proving infrastructure
  • Optimized proof verification onchain

The support for these features allows developers with limited knowledge of ZK to build end-to-end POCs using ZK, similar to the one we developed for Lido.

With this SDK, we hope that existing contributors to Lido can quickly ramp up to leveraging ZK when building new systems and applications for Lido.

Technical Details

Circuits.

In the current POC, we calculate the following three fields using the plonky2x proving system with the same logic in the official Lido Oracle implementation.

  • clBalanceGwei: cumulative balance of all Lido validators on the consensus layer
  • numValidators: the number of Lido-participating validators on consensus layer that ever appeared (deposited + activated)
  • numExitedValidators: the number of Lido-participating exited validators on the consensus layer that ever appeared

In terms of how the circuit is implemented:

  • Core Logic
    • First, we use a SSZ proof to prove that the validators root exists within the provided block at a specific gindex.
    • Then, we witness all validators and recalculate the validators root in the consensus layer and assert that the roots match.
    • After, we iterate over all the validators and compute the aggregate statistics.
  • To handle all (~1M) validators in a single proof, we utilize a “map-reduce” style computational model where we shard proving by batches of validators.
  • To know whether the validator is associated, we filter by a withdrawal credential.
  • Using cloud proving infrastructure, we parallelize the generation of the proofs.
  • The proofs are designed to be verified directly inside the EVM for ~300K gas.

Contracts.

Our contracts store the three calculated statistics in storage.

  • uint256 clBalancesGwei
  • uint256 numValidators
  • uint256 numExitedValidators

To request and verify proofs they use the FunctionGateway contract inside the Succinct SDK, which abstracts away the logic of how to communication information from the contract and the circuit and vice-versa.

Demo

We have a fully-working POC of correctly calculating the total Lido reserves, number of active validators, and number of exited validators in ZK across all ~1M validators on Goerli.

Event: Etherscan

The report was delivered for slot 6984000 and the results were verified against the forked Lido Oracle repo from Nil Foundation. The proof took around 40 minutes to generate, but is highly parallelizable so the latency can be reduce with better utilization of elastic compute in the cloud.

Roadmap & Deliverables

We propose that the Succinct team work in collaboration with Lido to develop an MVP on testnet that can demonstrate how the ZK oracle can augment the security of the existing oracle.

Already Completed:

  • Report Lido’s total value locked, active and exited validator counts to a smart contract on Goerli with 2^20 (~1M) validators.
  • On-chain verification for ~300k gas
  • On-chain API to request new update.

Phase 1 (Productionization, December 31st, 2023):

  • Conform the current smart contract to the desired interface of the ZK Lido oracle.
  • Deliver an oracle on Goerli with multiple reports with the new interface.
  • Create a testing suite for the oracle to ensure soundness / correctness.
    • End-to-end tests
  • Create cost-effective and highly available offchain actors that can deliver the proof every 225 epochs
    • Efficient use of cloud compute
    • Monitoring / logging
  • Make the system easer to administer
    • Documentation on how to run the system and how it works
    • Open-source
  • Start necessary audits for contracts/circuits (may complete in Q1)

Phase 2 (Testing, Q1 2024):

  • Latency requirements: less than 30 minutes for 1M validators
  • The Succinct team will provide infrastructure to run the oracle on Holesky testnet to emulate Mainnet conditions for up to 3 months.
    • Includes generating proofs up to every 225 epochs (~1 day)

We request that Lido provide us $50K to reward our existing work and to continue our ongoing efforts in this direction. This grant will be used to cover:

  • Development: 1 full time engineer from the Succinct team to complete this project
  • Compute: Covering prover/cloud costs during development and testing
  • Audits: The cost of auditing the contracts/circuits involved in this system

If this project succeeds, we hope to continue working with Lido on expanding the scope of this project and maintaining a long-term relationship between both teams.

FAQ

Who is the team behind this proposal?

Succinct is a company based in San Francisco, CA, working on applications of zero-knoweldge proofs on Ethereum. Our team has backgrounds from Celo, Google Brain, MIT, Stanford, UC Berkeley, and Caltech. In the past, Succinct built Telepathy, a zkSNARK-based light client for the Ethereum Beacon chain. The light client is one of the few non zkEVM applications of ZK used in production and to this day is used to secure over $40M in TVL with multiple integrations.

How do we get the beacon block root?

Once EIP 4788, is passed, the beacon block root will be trustlessly available on the EVM. Until then, either they must come from a trusted source or they can rely on another ZK oracle (such as Telepathy) to get the block root.

How does it compare to the other proposals?

Our solution is built upon open-source software built/contributed to by Succinct over the past year. In particular, the smart contracts and proving infrastructure is managed with the Succinct SDK and the circuit itself is built using our open-source Plonky2 verifier, our frontend for Plonky2 known as Plonky2x, and our frontend for writing STARK-based accelerators known as Curta.

The zkLLVM and DendrEth teams also take a circuit-based approach, although in different ways. The zkLLVM team uses LLVM to define the circuit while the DendrEth team uses a frontend library more similar to our approach, also using the Plonky2 proving system. The biggest difference between our approach and the DendrEth team is that we are exploring the usage of STARK-based accelerators built using Curta which dominates most of the proving time.

How will the work be maintained after delivery?

In the short-term, we are happy to run the infrastructure to run the oracle for at least a year. We may need another proposal in the future to cover longer-term proving costs.

In the long-term, our goal is to equip Lido contributors with the tools and infrastructure to maintain the system themselves and add functionality as the oracle requires changes.

11 Likes
2

I support this proposal! The Proof of Concept (POC) for calculating Lido reserves and validator data in ZK on Goerli is a promising step. We’ve discussed internally with other contributors and believe that using a multiprover architecture like 2/3 would be a good way to test ZK technology. Considering Succinct as a third leg alongside Metacraft Labs and =nil; Foundation would be a valuable addition.

5 Likes
3

I agree with the points, and it’s good to have in Görli, though Holešky is way better :+1:

5 Likes
4

Lego person here. The committee voted to support the work with a $50k grant, yay! To make this actually happen, need your address for dai transfer =))

5 Likes
5

This is an important initiative for reducing trust assumptions in critical aspects of the protocol + a great team to tackle the problem.

3 Likes
6

Hi everyone, thank you for the support on the proposal. We will perform the testnet on Holesky as @TheDZhon suggested.

In terms of our address for the DAI transfer, please find the details below @kadmil:

Mainnet Address: 0x4cFb28161717b976Cfe12C99eDe4311F1f9F6659

5 Likes
7

Thank you! Will look to perform the payment early next week =)

3 Likes
8

Hi all,

We have completed a working implementation of the Lido zkOracle, which could serve as a trust-minimized second opinion source for negative rebases of stETH, as outlined in LIP-23. Though delayed from our initial timeline, this allowed us to leverage the latest zkVM advancements, strengthening the implementation.

Initial Plonky2x Implementation

The initial prototype described in the first post was built using Plonky2x—a circuit framework. At the time this design was state of art and performant. However, there were key limitations:

  1. The circuit design has a computational constraint that limits the maximum validator set size to 2^21 (approximately 2 million validators). This limit could be reached through two mechanisms:

  2. Natural growth in the number of active validators from increased staking.

  3. The continuous increase in validator indices is due to each validator (including those that have exited) being permanently assigned a unique index. With validator indices currently approaching 1.7 million, this computational ceiling of 2 million becomes a practical concern that needs to be addressed.

  4. Circuit programs are difficult to maintain due to a lack of readability.

Final SP1 Base Implementation

Following the release of SP1, Succinct’s general purpose zkVM, we decided in April-May to rewrite the entire oracle implementation in Rust for SP1. Not only was this much faster to implement than the circuit-based design, but the codebase is much more readable and auditable. It also does not suffer from the 2^21 validator limit, enabling a future proof-design.

Demo

The implementation provides end-to-end functionality: from requesting slot-specific metrics to generating proofs and updating the Lido AccountingOracle contract onchain.

  1. Here is a report for slot 10339199, the clBalanceGwei value is 9744648818963059, which corresponds exactly to the Lido Oracle committee report.
  2. This proof is verified in this mainnet contract, where onchain proof verification is 366k gas.
  3. Proof generation took takes around 30 minutes and consumes around ~4B RISC-V cycles (note this is on the v3 version of SP1 and computational times vary as a public shared cluster is used). Proof generation times will likely be greatly improved with SP1 V4, our latest release.

SP1 ZK Lido Oracle Audit

The greatly improved solution based on SP1 took significantly more time and resources than it was expected. While the code was preliminary reviewed by two independent 3rd-parties (a16z research and Pashov audit group) with a helping hand of the Lido audits committee, we propose to put the full-fledge audit expenses out of scope for the grant and make them against the latest Pectra-hardfork enabled specifications.

Audit scope

Timeline and Next Steps

  1. We suggest considering this grant as completed and open for community feedback.
  2. The suggested timeframe for the final audits and escalating a governance vote to plug a second opinion provider is the post-Pectra hardfork period to accommodate any potential Beacon chain last-minute updates and the V4 release of SP1 being battle-tested.
  3. Further audit expenses will be handled separately from the original grant due to the extended scope of work and development timeline.

SP1 Based Implementation: GitHub - color-typea/sp1-lido-accounting-zk
Original Forum Post: ZK Lido Oracle powered by Succinct

7 Likes
9

It was a long thorny road and it’s great to see the fruits of this impressive work. Congrats.

4 Likes
10

Congratulations @jtguibas on successfully completing the grant! It was a long journey and required enormous effort to create the first-of-its-kind ZK Oracle to help secure the Lido Protocol. Many thanks to you and your team for your dedication! There will be next steps for plugging this Lido ZK Oracle to the protocol after a trial period but it’s out of scope for this grant. The most uncertain and tough part with ZK Oracle itself is done. Yay!

4 Likes
11

Given both your update and acks from folks above, lego is good on the grant as well, congrats!

5 Likes