The slashing incident of April 13, 2023 has got us all thinking about its causes and how to remedy them - and perhaps even more importantly, about how to prevent incidents like this happening to Ethereum and its contributors in the future.
Whether you call this particular incident a mistake, a reluctant reaction, human error, system failure or anything else, there will always be minor and major incidents that are unforeseeable; we need a proper strategy to deal with these.
Since mistakes provide a valuable opportunity to learn, grow and make things better, we want to start an open discussion within the Lido DAO community on whether or not we should establish a jointly accepted, well-proven and commonly known procedure for how to treat potentially harmful incidents. And if so, what such a procedure should look like.
The slashing incident that occurred to RockLogic GmbH in April 2023 and triggered this initiative is one amongst a greater variety of incidents that have happened before across the community. As we understand it, all these incidents have been handled professionally by those involved - but a common procedure has been distinctly lacking.
Which bears these questions:
- Would it be better to jointly develop a structured, universal procedure for how to treat things in case of emergency?
- Should we have a common “recipe” for how to deal with dangerous incidents? Should we develop open guides or emergency protocols that can be easily executed BEFORE disaster strikes?
- Should we adapt procedures, generalize them, and make them mandatory?
- Is there more we should think of, beyond monitoring & alerting, automated tests, doppelgänger protection, node configuration changes, key handling, release precautions & prior testing, …?
In short, should we have a universal emergency plan?
Think of it as a fire extinguisher: it will not help you with every problem that might occur on the way - but if something happens, it is easily accessible, and you know how to use it. It will kill minor flames in an instant and help mitigate, or in the best case even prevent, large fires.
We are keen to hear your opinions on the subject, and hope this post kicks off a healthy and fruitful discussion on the matter!