EIP-4844: An Optimistic Bet on Rollup Scalability
EIP-4844 (protodanksharding) is on the rise. With its development, memes, and community support, it's anticipated as the next big upgrade to Ethereum. But what is it really? In a nutshell, it's a new transaction type that makes it easy to write cheap but ephemeral data on L1. The data, represented as blobs, relies on some nifty new cryptography to ensure that it is available long enough for L2 use. There are plenty of resources on that topic. Here, we’ll cover the overall effort to push EIP-4844.
A History of EIP-4844 Development
EIP-4844 implementation started as a hackathon idea during ETHDenver 2022 where our own protolambda, terencechain from Prysm, and a couple others hacked an early prototype. This involved a geth fork that adds the blob transaction type and implements new crypto needed to secure the blobs.
The real development of EIP-4844 began later that spring with the immediate goal of fully implementing consensus and execution client prototypes. Early on we knew that getting an EIP of this magnitude to mainnet couldn't happen without the full support of the Ethereum community. While we still had at least nine months until the targeted hard fork, that time was critical to identify any major issues in the specification well before testnet. Our first milestone was achieved in early summer with Prysm and geth forks for EIP-4844. This was the first time we had fully spec-compliant execution and consensus clients.
We learned a lot from this exercise, including where the spec needed more detail, how to reduce implementation complexities, and the causes of performance issues. One noteworthy problem we’re still improving on is the speed of verifying blobs (transactions). Thanks to Ethereum Foundation researchers, we came up with a novel application of almost forgotten 70s math to update the spec and thus improve the efficiency of blob verification. Vitalik writes about it here. In a nutshell, we are able to amortize the cost of expensive blob verification using the following formula:
A Community Rallies
Developing a key upgrade like EIP-4844 (and seeing it included in an upcoming hard fork) requires the skills and efforts of a broad community. We joined forces with developers from Coinbase, Prysm, and Worldcoin to test, implement, and ship EIP-4844. The first fully implemented EIP-4844 devnet was presented on an EIP-4844 community call, which gave us a test environment for researchers and developers to hack what we built.
While community calls were helpful, we were still missing key feedback from client devs - especially given 4844’s development in parallel with The Merge. Devcon VI in Bogota was a pivotal moment for EIP-4844, giving us an opportunity to present, instruct, and discuss the upgrade with Ethereum core devs at the EF, as well as client teams. We released devnet v2, which implemented the latest specification of EIP-4844, giving the client devs an opportunity to gain more familiarity with the proposal. Devnet v2 was also used to build a Blobs Explorer during the ETHBogota hackathon.
Thanks to several research and development workshops hosted by the Ethereum Foundation, we were not only able to code together in person, but we also onboarded new core developers onto EIP-4844. As a result, we came up with a roadmap to getting EIP-4844 included for the next mainnet upgrade.
Development Process
For a long time, the development process went something like:
- Implement specification
- Test specification
- Update specification based on test results
- Repeat
As the spec was constantly in flux, devnets were our primary means of testing EIP-4844 because they were easy to spin up. Now that the spec is almost complete, we're productionalizing the implementation which includes hive testing, transaction fuzzing, providing test vectors for client interoperability, and testnets. Our goal is to make it easy for client devs to implement EIP-4844 and be confident that their implementation is according to specification.
The scope of EIP-4844 covers both consensus and execution, and thus its complexity is somewhere between EIP-1559 and The Merge. Furthermore, we want (need!) to ship EIP-4844 with the Shanghai HF. This gives us roughly a month or so from now to have a fully functional EIP-4844 testnet ready. To top it off, this is happening while client devs are busy implementing withdrawals for Shanghai. Since The Merge, client devs have been focused on fixing client bugs, cleaning up the code and working on withdrawals. We now have two non-trivial features: withdrawals and EIP-4844, concurrently being developed for the next hard fork. Executing this will be a true test of the efficiency and resourcefulness of the Ethereum community.
The EIP-4844 effort is the largest (other than maybe The Merge) community-led effort. With so many l33t devs now working on EIP-4844, we can pull this off.
A Blobspace World
Once EIP-4844 is deployed to mainnet, and data availability - dubbed blobspace - is unlocked for L2 use, we expect the cost of rollup L1 transactions to be reduced by at least 20x. We expect all rollups to take advantage of blobspace to reduce transaction costs for their users. It’s exciting to see what sort of applications the Ethereum community will come up with for Blob Transactions. From "blob splitters" to NFT metadata, EIP-4844 will create a new set of applications that need access to inexpensive and available data without compromising on security.
.oO