Turned out there are a lot of good teams (we’re currently at 5 or 6 and they keep coming) who want to build the censorability meter for Ethereum. We can’t practically work with all of them on that separately and fund them all - that’s beyond our capacity. So we’re doing an open RFP instead. We ask teams to submit the links to their works and methodology to this thread, and in 3 months Lido will evaluate them and give out up to 7 grants:
1st grant: 20k DAI - most useful meter for Lido’s purposes: tracking the effects of Lido policies on Ethereum censorability)
2nd grant: 10k DAI - most useful meter for tracking the overall state of Ethereum’s censorability, can’t be the same participant as the first prize
up to 5x 2k DAI grants - best five working solutions out of the rest of the participants
We will separately work with select teams to fund their operational expenses at Lego’s discretion.
Criteria for the first grant, in order of importance:
tracks the current and historical metrics of service degradation for censorable transactions
tracks the impact of Lido and Lido’s node operators on service degradation
provides API, or, preferably, an open dataset to work with
is likely to be maintained long term (low complexity, low cost of maintenance, other sources of funding are a plus)
tracks the impact of Lido and Lido’s node operators to service degradation compared to the other staking pools and operators in staking pools, where applicable
is open-source
Criteria for the second grant and third grants, in order of importance:
tracks the current and historical metrics of service degradation for censorable transactions
is open-source
speed of delivery of minimally viable version in public (earlier is better)
provides API, or, preferably, an open dataset to work with
is likely to be maintained long-term (other sources of funding, low complexity, and low cost of maintenance are a plus)
tracks the impact of staking pools and select node operators to service degradation compared to the each other
We’re also interested in how the tx’s “value” (priority fee and extractable MEV) impacts these metrics - are more expensive censorable txs included faster? But if there’s no correlation in practice, including it in the main project isn’t important.
Only projects that a) have a open public version by the time RFP ends and b) posted a short freeform application in this thread will be considered. The end of the RFP is the 28th of February 2023 - so you’ll have about three months to deliver. After that, we’ll take a week or two to evaluate all the submissions.
Hello, very interesting project. Can you detail more what you mean by “service degradation for censorable transactions”? Besides “slow” tornado cash transaction (waiting for a non OFAC compliant validator) I don’t see what protocols could be censored/measured for service degradation.
Thank you in advance.
Blocknative is open to collaborating with anyone working on this RFP to illuminate core infrastructure issues including potential censorship. We have a comprehensive historical dataset of mempool data on Ethereum which we consider a public good. In aggregate, the database consists of >5 billion unique, detailed transaction events (over 14.6 TB of compressed data) growing by more than 150 million events each month.
If you are working on this RFP and interested in the dataset, feel free to reach out to us in this thread or on our discord in the #general channel.
That’s a really cool! Depending on what the submitted RFPs (eg if one is just a back in time analysis vs constant monitoring), I can definitely see that dataset and API access to that mempool data be immensely useful for evolving a submission into a forward-looking solution!
We’re excited to participate in this RFP for building the censorability meter for Ethereum.
Our code and methodology can be found on our GitHub. We will also have our site up and running at eth.neutralitywatch.com by the end of February.
We have proposed several metrics for the censorability meter, you can find information on these metrics on our GitHub repository (currently in develop branch). We have also developed an API to access our dataset and metrics, which is currently closed but we can open access to it in March by request.
We welcome any inquiries and look forward to possible future collaborations.
Thanks Vasiliy and LEGO for this grant initiative and RFP!
Here’s @fredo’s and my proposal - the Ethereum Censorship Monitor. You can find a hosted frontend with visualizations and a bit of educational content at https://valitraitors.info (sorry about the name ) and the source code here. There’s also an API and a low-level Grafana Dashboard.
Our proposal fulfills all criteria for the first grant:
Tracks the current and historical metrics of service degradation for censorable transactions
The metrics of service degradation are tracked in real-time and historical data is available back to Dec 7th, 2022. [Please note that slight modifications to the methodology have been made since and there were brief periods of outage.]
The dashboard shows live censorship data in the past week.
Tracks the impact of Lido and Lido’s node operators on service degradation
Among others, the dashboard compares the biggest staking pools. Specifically for Lido, there is a breakdown by node operator available.
Provides API, or, preferably, an open dataset to work with
We provide an easy to use REST API that makes the censorship data publicly accessible. Check the documentation with examples in the readme on GitHub. We are also down to sharing the raw data with interested parties for their own analysis.
Is likely to be maintained long term (low complexity, low cost of maintenance, other sources of funding are a plus)
Required maintenance capacity is low and can initially be supported by Brainbot technologies AG who also funded the initial development of this project. Long-term maintenance (6m+) as well as future improvements, optimizations, and additional features may require additional funding.
Tracks the impact of Lido and Lido’s node operators to service degradation compared to the other staking pools and operators in staking pools, where applicable
The dashboard compares the major staking pools.
Is open-source
Please find the code on Github.
The methodology and more details are further explained in the Readme.
It’s exciting to see submissions already. Just to make sure we don’t miss any that may be just close to the finish line – LEGO shall extend the submission deadline until EOW (March 3).
Here’s the P2P Team and our proposal - the Ethereum Censorship Monitor.
Our proposal satisfies all criteria for the first grant:
Tracks the current and historical metrics of service degradation for censorable transactions
We are monitoring the main indicators and metrics since February 10th, 2022: time delay, block delay, censoring addresses, mev-relays, mev-builders, Lido validators etc.
Tracks the impact of Lido and Lido’s node operators on service degradation
In our research and dashboard we consider Lido validators and compare these with others. Also in our paper we made statistical analysis impact of Lido on service degradation.
Provides API, or, preferably, an open dataset to work with
We provide Google Bigquery Datasets (p2p-data-warehouse.p2p_public.eth_mev_censored, p2p-data-warehouse.p2p_public.eth_missed_blocks_metrics) which contains all main indicators and metrics to analyse influencing of censorship on service degradation in Ethereum.
Is likely to be maintained long term (low complexity, low cost of maintenance, other sources of funding are a plus)
Our DWH system is automated and scalable and increasing the dataset is not a problem for our solution.
Tracks the impact of Lido and Lido’s node operators to service degradation compared to the other staking pools and operators in staking pools, where applicable
We marked only Lido validators so far. So, we analysed it in the subgroup Lido / Non-Lido. But we are planning to mark all other validators to deeply research.
Is open-source
We released open-source:
Google Bigquery Datasetp2p-data-warehouse.p2p_public.eth_mev_censored, p2p-data-warehouse.p2p_public.eth_missed_blocks_metrics
The ultra sound team is pleased to make a submission to this RFP! Two new dashboards titled “sanctions censorship” and “inclusion delay” will be live on relay.ultrasound.money in a couple hours. A few notes:
experimental: This is an experimental beta release. There are known bugs and rough edges—please bear with us. The team will rapidly improve the dashboards next week. We are also open to improvement suggestions from Lido and the wider community.
open-source: The code is split across two MIT-licensed repos for the backend and the frontend. There is an additional MIT-licensed repo with OFAC Ethereum addresses. Figma designs can be found here—feel free to reuse them! We can also share ad hoc read access to the DB—please contact a team member.
holistic: The “sanctions censorship” dashboard aims to provide a birds-eye view of Ethereum censorship, diving into the three key stages of the MEV pipeline: building, relaying, proposing. There is a second “inclusion delay” dashboard to shine light into the various reasons Ethereum transactions may be delayed beyond censorship—these include congestion, insufficient base fee, and low tip.
Lido-focused: We currently focus on Lido operators for proposer censorship. We aim to expand to non-Lido operators and validators in a future release.
definitions:
A builder or relay is said to perform sanctions censorship if none of its blocks contain transactions touching addresses from a sanctions list. We know of two sanctions lists with Ethereum addresses: the US OFAC SDN list and the UK sanctions list. The UK sanctions list only has one Ethereum address with no transactions since the merge, hence our focus on the OFAC SDN list.
A validator operator is said to perform sanctions censorship if its validators only connect to censoring relays and all its self-built blocks also censor. In this initial release we ignore self-built blocks for Lido operators—a more fine-grained analysis taking into account self-built blocks will come in a future release.
data sources: We have taken care to build a comprehensive data set that covers the full lifetime of MEV-boost from the merge onwards. Special thanks to zeromev.org for sharing their mempool timestamps as observed from nodes in three separate geographies: America, Asia, Europe. We will be enriching this dataset using Blocknative mempool timestamps, once publicly available. Relay block inclusion data is gathered directly at the source by querying relays. A mapping from builder pubkeys to builder names was manually collated as part of the ultra sound relay operations.
Figma screenshots: Below are screenshots of the Figma designs. These are mockups with fake data. Please see relay.ultrasound.money for a live version with real data.
Dashboard 1: The above “sanctions censorship” dashboard focuses on OFAC censorship at three different stages of the MEV pipeline: builders, relays, proposers.
Wow, overall I’m really happy with the submissions. I think the soft censorship landscape in Ethereum is illuminated a lot better than it used to be. We’ll take a week or two to evaluate all of that - it’s clear that all the submissions are pretty great and it’ll be a tough choice. We’ll be asking questions so please keep in contact with us.
If you open a link, there will be censorability RFP Q&A threads. LEGO members, helpers and community will ask you a few questions over the course of a next week and we’d like you to answer. It’s not supposed to be real time but please check it out every couple days or so. We will use your answers to rate the submissions and give out the prizes.
We had four awesome submissions, and it was tough to choose the prize winners. But, after much deliberation, we have made our decision.
The first prize of 20k DAI goes to Neutrality Watch! They did an amazing job of tracking the current and historical metrics of service degradation for censorable transactions, and they best tracked the impact of Lido and Lido’s node operators on service degradation. Their submission provided an API, and we will find a way to make sure the project is maintained long-term.
The second prize of 10k DAI goes to Ultra Sound Relay! They came up with awesome dashboards that highlight the current situation in Ethereum very well, are live on the well-known website, and, in general, scratch the itches of the Ethereum community best. It also showed a strong likelihood of being maintained long-term.
Both other submissions receive 2k DAI prizes because they are awesome in their own way. We also want to give out two special prizes! The first special prize of 3k DAI goes to Valitraitors for having the best visualization of censorship and the best staking pool comparison. And a controversial monicker, that too.
The second special prize of 3k DAI goes to P2P for having the best open dataset. Their submission, the Ethereum Censorship Monitor, is the most comprehensive in terms of raw data (accessible to all via bigquery) and shows the dedication to Ethereum ethos.
We want to thank everyone who submitted their works and methodology for the RFP! Your contributions were amazing, and we appreciate the time and effort you put into your projects.
Please post the addresses to receive prizes below or reach out to a LEGO committee member.
So, that’s it! Thanks again to all the participants, and we can’t wait to see what you all come up with next!
We’re super excited to find out that Neutrality Watch has has won the first prize!
A big thank you to Lido for recognizing our efforts, and congratulations to Ultra Sound Relay, Valitraitors and P2P - your projects were outstanding, and we’re proud to be part of this innovative community.
Looking forward to contributing more to the Ethereum ecosystem!
We have a comprehensive historical dataset of mempool data on Ethereum which we consider a public good. In aggregate, the database consists of >5 billion unique, detailed transaction events (over 14.6 TB of compressed data) growing by more than 150 million events each month.
An update to our original post: You can now use this link to submit a request for access to the Blocknative Data Program to receive the data needed for your research. Feel free to reach out if you have any questions!
) and the source code 
