Sushi RouteProcessor2 Post-Exploit Request For Comment

[jiro]

Dear Lido Community,

Over the past weekend, we upgraded our protocol to introduce V3, which included a new router to facilitate swaps and future aggregation plans. Unfortunately, the RouteProcessor2 contract contained a critical risk level approval bug, and users who approved the contract within the approximately 12 hours that it was live were at risk of being exploited from the contract. While we were working on mitigations to prevent as much damage as possible, a whitehat made a mistake while attempting to secure the funds using a public rpc. As a result, several other parties were able to replay the transaction, resulting in approximately 1800 WETH being drained from a single wallet. Some of these transactions were built by independent block builders, where in one case a substantial amount of ETH had been transferred as a MEV reward to the block builder that then redirected it to Lido Execution Rewards Vault.

We reached out over the weekend to several Lido contributors to discuss options for recouping these funds and were advised to continue the discussion within the community. We understand that Lido is a fully permissionless protocol, and therefore there may be no immediate levers that can be pulled to help capture these exploited funds. However, we wanted to initiate a conversation about possible options for recovering funds in this situation, which is unprecedented and could happen again in the future with block builders being bribed to build malicious transactions.

From our initial conversations, it appears that approximately 78 ETH was sent to the Lido Treasury, which could be an easy starting point for recovering some of the funds. For the rest of the funds, the majority of them have been or will be re-staked, and we are open to all ideas and suggestions for mitigating the situation and helping with the recovery process.

We apologize for any inconvenience this may have caused and are fully focused on doing everything we can to make everyone involved in the exploit whole. The Sushi community has much respect for Lido, and we thank you for the help we have received so far.

edit: We’ve updated this proposal to include additional granular information below to help the Lido community understand the exact nature of the funds disbursement.

Within blocks 17007839 and 17007842 in total of 5 transactions related to Sushi RouteProcessor2 bug sifuvision.eth sent a total of 795.9761955 ETH to Lido: Execution Layer Reward within beaverbuild (within block 17007842) or directly (within block 17007839).

Transactions and ETH Amounts

91.9961955 ETH
10 ETH
5.1 ETH
678.88 ETH
10 ETH

Also as a correction to our 78 eth sent to the treasury statement. Those rewards were combined with rewards on CL and other rewards on EL and defined stETH rebase that reflects ETH inflow in protocol within the Oracle report on 2023-09-04. As a result of the Oracle report, according to Lido protocol specification, 5% of total rewards were transferred to treasury and 5% to node operators. With total rewards for 2023-09-04 being ~1,564.7 ETH, with 5% at ~ 78.23 ETH.

Therefore, the exact proportions applied to the total rewards gained as a consequence of the bug (795.9761955 ETH) as it’s a part of total rewards result in:

1. 5% (~39.8 ETH) going to the treasury
2. 5% (~39.8 ETH) were sent to node operators
3. 90% (~716.3 ETH) remained within the protocol and were staked according to the protocol specification combined with other rewards, resulting in 1056 ETH in total deposited from stETH token contract to beacon chain to different validators operated by different operators, 33 validators total

[satBalwyn]

@jiro

78 ETH was sent to the Lido Treasury

can you provide the transaction id here? why the fund went to the treasury?

btw, to make it more clear to the community, better to attach all the related transactions.

[Jared_Grey]

Thanks for your feedback, satBalwyn. We’ve updated the proposal to reflect your input.

[horses]

Hi Lido team, I’d like to note that Sushi is in a hostile state with Jared Grey locking the extremely fair proposal (Remove Jared Grey as Head Chef - Proposals - Sushi Swap) to have him removed. That silencing is just the tip of the iceberg. There is a lot more and this “exploit” is looking very suspicious as days go on. I’d suggest not returning funds until he has been removed since no one is able to guarantee the recovered amount will make it back to users. Thanks for your consideration.

[Misha_Statemind]

On surface level proposal makes sense, however, if passed, it might set a dangerous precedent as there is no framework to govern this activity, yet, we all know, that cases like this are plentiful. Without a proper research ramifications to the protocol are simply unknowable. I see several possible risks here:

1. Without a clear framework Lido DAO can be heavily throttled by inflow of hack reimbursement proposals.
2. In case of reimbursement DAO needs to be an arbitrary judge of what is legal/illegals activity for other protocols which is way beyond it’s usual capacity and might bring an unpredictable legal risks.
3. What about other MEV manipulations and their status?

[k06a]

I believe that any residual dust or fragments of stolen assets should still be considered as stolen property and must be returned to the appropriate entity responsible for hack/debt resolution.

[Hasu]

I’m very sorry to hear that Sushiswap became victim of a hack. Thank you also for this proposal, which opens an important discussion in Lido DAO. In this post, I will address two separate things:

1. How should Lido governance approach problems like this in general (meta governance); and
2. My thoughts on what a concrete policy, in this case, should be.

Metagovernance

I believe that DAOs, in general, and Lido DAO, in particular, can’t succeed while micromanaging the day-to-day choices of a protocol. Instead, they should be voting on very important and relatively infrequent decisions. In particular, on five things:

1. a constitution (basically the operating manual for a DAO, or “protocol for the social layer”) that is decided once and hard to change afterward
2. key personnel, time-bounded
3. budget, time-bounded
4. important policies, time-bounded
5. important one-time decisions (e.g., token issuance, buybacks, M&A deals, etc.)

The decision at hand falls firmly into “important policies.” The difference between a policy and a one-time decision is that policies are guidelines or templates for how all decisions of a particular type should be decided in the future.

One could argue that it’s an “important one-time decision” (and Sushi will certainly try to do that). But I will argue in the rest of the post that this decision will have a big impact on the future decisions that Lido has to make, which calls for creating a policy instead.

So if we assume that Lido token holders should discuss and vote on policies that the Lido protocol and Lido DAO service providers will execute, what would a good policy be in this case?

Should third parties be allowed to seek arbitration from Lido DAO?

First of all, I want to repeat that we are dealing with a very important decision here. Whatever Lido DAO governors decide will not affect the case of Sushi primarily. Instead, it will create a precedent that will henceforth act as a policy, whether we want it or not. Hence we gotta be deliberate that we are creating a policy here and treat it with the necessary weight.

Of the funds in question, 5% was sent to Lido DAO, 5% to node operators, and 90% automatically deposited to stakers. We are hence not predominantly talking about whether Lido DAO should return any funds but whether Lido DAO should seek to arbitrate a dispute between Sushiswap and node operators and stakers.

My concrete policy proposal is for Lido never to act as such an arbitrator between stakers and node operators and third parties. Three main arguments speak in favor of that:

1 – An MEV arbitration policy opens a significant attack surface on Lido.

The first argument is that arbitrating Sushi’s MEV transaction is no different than arbitrating any MEV transaction where another party lost money. If Lido DAO were to arbitrate any money lost by Sushiswap, that would create the expectation (and policy) that anyone can get their MEV arbitrated by Lido.

Whether it’s a Uniswap trader that got sandwiched, an NFT aficionado who lost the opportunity to mint because of bots sniping a launch, or a liquidity provider who lost money to arbitrage – everyone can come to Lido and claim that whatever happened to them was illegitimate and hence Lido should refund them.

It should be clear that such a policy represents A) a big drain on Lido governance and B) would put Lido at a competitive disadvantage to all other stakers and staking pools with no such policy, and C) make Lido the arbiter of what transactions are allowed to be included in Ethereum or not.

2 – Lido is neutral middleware.

Lido aspires to be a neutral middleware on top of Ethereum. This type of neutrality is extremely important for us to minimize our ability to harm stakers and the Ethereum community and scale to a much bigger size than we otherwise could. Becoming a neutral middleware requires us to ossify Lido’s technical and governance layers as soon as possible and be unopinionated about as many things as possible.

As we saw in the first argument, it becomes very hard to draw the line when third parties could seek recourse on individual transactions from Lido. Doing this effectively requires Lido DAO to become an arbitration and censorship layer on top of the Lido protocol and, therefore, on top of Ethereum. This starkly violates Lido’s goal of neutrality – in the same way that Ethereum does not freeze the accounts of anyone, Lido DAO should not decide what transactions are legitimate or not for Ethereum node operators who use Lido to include.

3 – Lido should minimize the role of governance.

Finally, there is the question of how much governance we should seek to allow in Lido DAO. This is a valid question since governance brings flexibility, and ossification brings inflexibility.

One of the core tenets of having an ossified governance layer is to have no more rules than are necessary. So when we are faced with the choice to A) adopt a policy of doing something and B) adopt a policy of doing nothing, we should always favor the latter today. That is because when it becomes time to ossify our governance layer, ossifying a policy that does nothing is much easier than the policy that does something.

Conclusion

In this post, I argued that Lido DAO should be governed through delegation and high level policy, not individual decisions. In the case of Sushiswap, we are dealing with such a policy decision that has wide effects on the neutrality and governance ossification of Lido in general. I recommend for Lido to adopt the policy of not arbitrating between third parties and Lido stakers and node operators for the reasons laid out above.

[matthewlilley]

Appriciate your thoughts being shared here, it’s a utopian view though and ignores real-world concequences of not taking action…

There is not only an ethical responsibility to return stolen funds to users, but to protect the many contriburtors of Lido, stakers, etc… handling stolen funds affects every one of them.

[will_harborne]

Approach:

I agree with Hasu’s general statements:

1. Lido governance should approach this and other questions by setting a constitution, and in this case “important policies” which can be followed consistently
2. In this specific case a concrete policy must be set going forward which covers the handling of MEV collected from exploits
3. We must make sure that this policy does not open Lido to having every case of MEV (whether from known exploit or not) disputed, which would create an admin burden that Lido DAO is not equipped to deal with, and an expectation that any MEV is potentially recoverable

The specific policy:

However I disagree with the actually proposed policy (i.e. that Lido should never act as an arbitrator and should just put its hands up and say “not our problem” when clearly hacked or stolen funds pass to the DAO / node operators or stakers).

A goal of neutrality for Lido is honourable, but using the excuse of neutrality to avoid difficult problems is not.

A policy taking a more nuanced approach will be less simple, but can still be crafted to avoid unnecessary burden on the DAO and remain clear. An example would be:

• Exploits must be known and clearly defined and then reported via a specifically designed channel (i.e. a thread set up in the forum)
• Lido treasury, operators, and stakers must be able to be shown to have received more than 50 ETH from MEV related to this exploit (to filter out smaller requests)
• Lido will then arbitrate to recover funds, and all its operators (accepting its constitution) will opt into agree to do this also

Competitive environment

A final point is that it is worth considering the overall direction of travel for the MEV ecosystem and competition for LSDs. With staking withdrawals now open, the liquidity moat that stETH has may shrink in the future. This will make competition over other factors more important than it has been in the past, and mean that competitors with smaller market share are less disadvantaged in DeFi than they have been up to this point.

We know that there is for example competition in the MEV space for more MEV to be returned to users and distributed in ways that are more aligned with ecosystem. Examples are mevblocker . io compared with i.e. Flashbots.

If Lido takes a policy of neutrality and refuses to ever return funds collected as MEV from major hacks, then I would expect some other liquid staking providers may use this as a differentiator to gain market share, signalling that they are more aligned with the DeFi users, projects and ecosystems.

Teams such as Sushi, and other DeFi projects, may feel align themselves with LSDs that have policies to help return stolen funds, instead of with Lido which does not. However this is clearly speculative at this point…

[matthewlilley]

@will_harborne

However I disagree with the actually proposed policy (i.e. that Lido should never act as an arbitrator and should just put its hands up and say “not our problem” when clearly hacked or stolen funds pass to the DAO / node operators or stakers).

Thanks for you post, an most importantly disaggreeing here.

Putting it simply, illegitamite funds were received which enriched Lido. Lido didn’t ask for this, and we’re sorry for our mistakes which put you in this position, but ignoring it is doesn’t seem wise. We don’t want to get dragged into politics of utopian goverence, we want to make users are made whole.

[sambacha]

How does adopting a constitution not violate your 3rd preamble?

The payout isnt MEV, its proceeds of a hack. I think your counter arguments are fairly week to be honest. The counter point should be more so related to how fee payments are treated ex post facto. Also the claimant themselves should come and make their case, not a 3rd party whom may or w be speaking on their behalf.

Node Operators may have less legal shield than stakers. A John Doe warrant served to the DAO against node operators would be more successful than against stakers per se.

[Hasu]

How does adopting a constitution not violate your 3rd preamble?

Glad that you ask! Basically, constitution is to the social layer what an immutable smart contract is to the protocol layer. It establishes a set of guidelines and rules between different actors in the system, to the degree that they happen outside of what can be enshrined with smart contracts.

[Neiltbe]

Personally, I think the idea of stolen property that is “traceable”, “identifiable”, & “attributable” but “not recoverable” is a significant risk to long term viability.

Likely this comes with huge legal headaches, practical headaches (validators or addresses getting co-labeled as recipients of stolen funds) + regulatory exposure & headaches. Really this does boil down to participants in the protocol (including the treasury) being compensated with stolen funds. I don’t believe now is the time to open this Pandora’s box.

I am hoping this matter can pave the way toward thinking about how protocols can solve these challenges before the trial by fire. My sense is the “ can’t do anything about it” is not the correct path.

[Hasu]

Would you say the same about Ethereum?

[Neiltbe]

Yes, great example. That’s why there’s Eth & Classic. They didn’t open Pandora’s box prematurely.

[Cortina]

I agree with most of your longer post above but this post seems to raise a contradiction.

Wouldn’t you say that Ethereum has both technically and socially ossified such that similar requests/expectations don’t exist? One could argue that until Lido “can’t do anything about it” Lido should do something about it (which is why Lido must ossify.)

Another question that should be asked if a reimbursement plan is pursued: In the process of reimbursing Sushi victims, is it possible to only take from stakeholders exactly as much as they were unduly rewarded or is it inevitable that any solution will create another set of undeserving victims. (It is always easy to take small amounts from many people over long periods of time to remedy victims with more acute pain but that is called insurance and a premium must be paid.)

[sacha]

@Cortina

Wouldn’t you say that Ethereum has both technically and socially ossified such that similar requests/expectations don’t exist?

Danny Ryan’s recent thoughts on the matter are insightful here:

Or maybe it isn’t that the protocol can truly ossify but that instead the best we can do is always be-in-dialogue-with ossification. If the ethos is to attempt to ossify or to be in a much more ossified state rather than to always expect, desire, or need change, then the dialogue of ossification will bias towards change-skepticism. Enough skepticism here might be sufficient to protect the protocol even as the protocol (slowly) evolves.

As I reflect this year, I begin to believe that this dialogue with ossification is not only the best we can do but in a world in which we cannot predict tomorrow, ensures that Ethereum does not reach a local maximum that is insufficient for what is to come. But I also believe that the ossificationers – those that bias toward slowing down, that bias towards only changing certain components if absolutely necessary, that fear ever mounting complexity and the impact of change on the layers above – are too small a cohort in both the core L1 process and the greater community today. This camp and ethos is not yet strong enough to be the requisite immune system of the protocol.

By any measure, Ethereum is still very far away from technical ossification. The recent discussions on social slashing show it is quite far from ossifying socially too (fwiw, I think the distinction we between the two is largely overrated).

[sacha]

@Neiltbe

Yes, great example. That’s why there’s Eth & Classic. They didn’t open Pandora’s box prematurely.

Apart from the large differences in number of people affected and the magnitude of the funds at risk, I think an important part of what made this socially acceptable is that we were able to run both possible futures by forking, and give community members the freedom to decide which future they would like to belong and contribute to. Lido DAO does not have that superpower.

Note that Vitalik has expressed mixed feelings on this too. Here are some of his reflections from an interview with Naval:

Least proud moment? Definitely the handling of the DAO fork situation. A lot of people did feel betrayed as a result of the DAO fork. A lot of people did feel like their expectations got violated. And a lot of people felt like their opinion was disrespected, especially people who opposed the fork.

A lot of them did feel like there was this social environment where if you oppose the fork, then you’re evil because you’re pro hackers stealing millions of dollars.

That environment ended up turning a lot of people off, and there was a lot more that we could have done to not create that environment and still make people feel welcome despite the disagreement.

[kadmil]

As the discussion has multiple possible directions outlined, seems like a good next step would be to summarise main options for the DAO Snapshot vote.
The summary would be sent as a comment to this thread on Fri, 21st, will be up for feedback and go to the snapshot (with feedback incorporated) by Thu, 27th.

[sacha]

@will_harborne

A goal of neutrality for Lido is honourable, but using the excuse of neutrality to avoid difficult problems is not.

With all due respect, avoiding difficult problems is not what we’re about, nor what Hasu is getting at. Quite the opposite in fact. The path he’s suggesting is a much longer and gruelling one. One that does not buy us any friends in the short term.

Nor is neutrality simply an honourable goal. It is essential to a world in which agreements can be enforced without violence, the pursuit of which is a core part of Lido’s purpose.

For me, this is about staying true to the DAO’s core ethos and guiding principles, which is arguably the much harder thing to do in the face of public pressure to compromise on them.

The guiding principle that is perhaps most relevant to this request is the following:

Self-regulate through technology and incentives, not laws and promises.

Even if it is by far the more gruelling path, I believe that Lido DAO should favor self-regulating through cryptography and incentives (market-forces), rather than laws and promises.

To paraphrase Nikolai Mushegian, we should respect incentives as natural law. For in a system that is open for the whole world to interact with, incentives are not just a suggestion; they are more akin to physical laws, like gravity or entropy. If there is even one part of the system that is not incentive-compatible, it is only a matter of time until it is exploited. In this sense, fixes that don’t serve to align the relevant incentives that led to the issue in the first place are a strategic error.

With respect to your speculative claims:

If Lido takes a policy of neutrality… I would expect some other liquid staking providers may use this as a differentiator to gain market share

My perspective here is that credibly neutral protocols are so incredibly difficult to get right that those that do make it will always have a competitive advantage vs the rest, and so will always be in demand. This is especially true in this current geopolitical climate, where increasing political polarization and instrumentalization of the rule of law has resulted in a scarcity of neutrality across all countries and the systems they depend on.

In Nikolai’s words again:

Credible neutrality is a competitive advantage. Even if it takes longer to scale, a sound and credibly neutral system will ultimately win out… Teams that try fiddling with incentives and see short-term results fail to realize just how much capital is waiting on the sidelines, unwilling to commit to a mechanism where the developers still have so much control.