Figment - Misdirected Execution Layer Rewards - Incident Report

Node Operators
1

Root Cause

On November 6, 2024, around 11 PM UTC, a code change was made by a Figment developer, which led to a misconfiguration, causing execution layer rewards for some validators run by Figment using the Lido Protocol to be misdirected.

As a result, execution rewards (but not consensus rewards) were impacted for a brief period.

Impact

  • The bug was identified on November 6th and fixed within about 9 hours.
  • Only execution rewards were impacted; consensus rewards were not affected.
  • The issue persisted from approximately 11 PM UTC on November 6 to 8:30 AM UTC on November 7, 2024.

Impact on Lido

  • Approximately 1.152 ETH was misdirected for about 9 hours and has since been redirected (here) to the Lido Execution Layer rewards vault.

Our Response
We quickly identified and fixed the bug, restoring all validators to their correct configurations. We’ve also implemented measures to prevent similar issues in the future.

  • The bug has been fixed, and no further rewards will be misdirected.

Remediation and Prevention
Figment has acknowledged the impact of this incident and is implementing additional safeguards to prevent similar events in the future.

  • Enhanced QA code reviews and automated testnet deployments for sensitive code changes
  • Time-based role escalation with permission security requests for sensitive code changes
  • Additional detection and alerting systems from existing and new log sources

Next Steps
We will continue to monitor the system and prevent similar issues in the future.

2 Likes
2

Hey Ben. Thanks for reporting the infrastructure misconfiguration and a recap of the root cause and timeline.

I can confirm that the amount matches up with the blocks that were identified by NOM contributor monitoring as having incorrectly assigned fee recipient.

1 Like