The Beacon Chain and Ethereum 2.0 — where it came from and where it’s going. Missed “State of the Ethereum Protocol #1”? Check it out here.Ethereum 2.0 isn’t a new idea. Back in 2014, Vitalik said of Ethereum 2.0 that, “We will either solve the scalability and consensus problems or die trying.” Well, we’re still very much alive, and his updated view from just a couple of weeks ago is that “There is no significant unsolved theoretical problem left for Ethereum 2.0.”Now it’s time for what he calls the “software development slog,” and the Beacon Chain is the first component of Ethereum 2.0 in the delivery plan. In this article we’ll discuss what it does, why it does it, and how it’s being developed.Introducing the Beacon ChainIn my previous article, introducing Ethereum 2.0, I showed Hsiao-Wei Wang’s now famous architecture diagram of the Ethereum 2.0 system.Ethereum 2.0 overall architecture. Original diagram by Hsiao-Wei Wang.This image also serves as a step-by-step roadmap for the development and delivery of Ethereum 2.0. Reading from top to bottom:The PoW Main Chain is the part that exists today: the current Ethereum Mainnet. In the Ethereum 2.0 system, this continues to operate pretty much as it does today. Everything below this is new.The Beacon Chain is currently under development and will be the first component to be delivered.The Shard Chains are next and are where the scalability will come from. Initially, the shard chains will simply aggregate transactions and come to consensus on their ordering, without executing them. This will be a good test of the system’s infrastructure and security.The VM layer is the final big component of the Ethereum 2.0 system and will provide for the execution of contracts and transactions.Why do we need a “Beacon Chain”?The Beacon Chain is a brand-new, Proof-of-Stake blockchain. It is the spine that supports the whole of the new Ethereum 2.0 system. It is the heartbeat that keeps the system alive. It is the conductor coordinating all the players.The Beacon Chain coordinates the whole system (image source)The key function of the Beacon Chain is to manage the proof-of-stake protocol for itself and all of the shard chains. There are a number of aspects to this: managing validators and their stakes; nominating the chosen block proposer for each shard at each step; organising validators into committees to vote on the proposed blocks; applying the consensus rules; applying rewards and penalties to validators; and, being an anchor point on which the shards register their states to facilitate cross-shard transactions.Before we look at these functions in more depth, a note on terminology. The name Beacon Chain has its origins in the idea of a “random beacon” — such as NIST’s — that provides a source of randomness to the rest of the system, and the random beacon concept was adopted in a blockchain context by Dfinity. Although the name “beacon” suggests a central point, broadcasting to the rest of the system, of course it’s not like that on the blockchain: everything is decentralised. Each participating node maintains its own local Beacon Chain, striving to stay in lockstep with the other nodes. Perhaps the image with the conductor above is misleading. While the Beacon Chain does conduct the rest of the system, the conductor is himself decentralised, like each musician’s own internal sense of the rhythm.Introducing the Beacon ChainLet’s look at some of the functions of the Beacon Chain.Managing ValidatorsA major part of the work of the Beacon Chain is maintaining the set of validators, which is the set of nodes that have placed the required stake of 32 Ether, who are responsible for running the Ethereum 2.0 system. There are several statuses that a validator can have, but only those marked as “active” take part in the Ethereum 2.0 protocol.Nodes join the validator set by sending their stake to a contract on the Proof-of-Work chain (the current Mainnet). After some validity checks, the stake is locked up and the contract emits a log entry (an “event” in Solidity) which can be picked up by Beacon Chain clients. The node is then inducted into the validator set on the Beacon Chain.Once active, validators participate in the protocol by proposing blocks, when chosen to do so, on both the Beacon Chain and, once they have been implemented, the shard chains. They also join committees that vote for blocks, as described below.Validators can signal that they wish to exit the system and cease being involved. After a period of time (currently 97 days, but may be made more flexible), their stakes, plus rewards, minus penalties, will be returned into one of the shard chains. It will not be possible to unlock the initial stake on the PoW Mainnet, unless, perhaps, the whole system were to fail and the community agrees to a fork that refunds stakers.All of this is managed by the Beacon Chain.Providing RandomnessGood quality randomness is really hard to generate in blockchain systems, but a key requirement of the proof-of-stake protocol is a source of randomness that is distributed, verifiable, unpredictable, and (reasonably) unbiasable. The Beacon Chain is responsible for providing this randomness to the rest of the system: several of the protocol features described below depend on it.Randomness is tricky on blockchains (image source)The current approach to random number generation is a RANDAO construction with validators providing a “hash onion”. A RANDAO is simply a way to combine contributions (individual random numbers) provided by many participants into a single output number. To prevent any one participant manipulating the randomness significantly, a commit–reveal scheme is used. When a validator registers, it provides a commitment value that is its chosen original number hashed many times. Each time the validator is selected to be the proposer, it peels off one or more layers of the onion by providing a pre-image of the last number revealed. Everyone else can check that this was done correctly, so the proposer cannot cheat the system by changing its contribution.Although this scheme is not completely unbiasable — a proposer could skip its turn if it didn’t like the random number — it is believed to be sufficiently robust for the current protocol design.Block proposersThe Beacon Chain manages both its own proof-of-stake protocol and that of each shard chain. In a proof-of-work system, the node that gets to choose the next block, the block’s miner, is the first to solve the mining challenge. In proof-of-stake there is no mining, so block proposers are chosen randomly, based on the in-protocol randomness described above.Another property of a PoW system is that block times are irregular, although they average to around 15 seconds on Ethereum. In contrast, I described the Beacon Chain as being like a heartbeat. Ethereum 2.0 blocks are produced regularly every 16 seconds (although there is a desire to reduce that to 8 seconds if testing demonstrates it to be feasible). These 16 second periods are called “slots”.At each slot, the chosen proposer for the Beacon Chain collects up all the protocol votes (attestations) from the Beacon Chain validator set for previous blocks and forms them into a block that it publishes.Once the shard chains are in place, each shard will have its own chosen proposer at each slot that will gather up the transactions for that shard and form them into a block to be voted on by the shard’s committee.CommitteesVoting by committees of validators plays a key role in securing Ethereum 2.0 (image source, cropped)A critical source of security in the proof-of-stake blockchain is the committees that vote on which blocks form the true history of the chain. The Beacon Chain relies on counting votes, known as “attestations”, from its own committee in order to agree and enshrine (finalise) its history. This committee could, ideally, be all the active validators in the system if their attestations can be collected quickly enough.In addition, the Beacon Chain will appoint smaller sub-committees for each shard at random that will, in due course, be responsible for confirming that the shard’s proposers are behaving correctly.Rewards and penaltiesYet another administrative role for the Beacon Chain is the tracking and updating of validators’ deposits.Validators receive rewards for behaving well and playing their part: this is the incentive for participating. But if validators break the rules, they are penalised by having some of their 32 Ether deposit reduced (slashed), and being ejected from the system. There is also a small penalty for being absent (not showing up to vote on blocks) called the “quadratic leak”. The reasons for this are subtle, and are about allowing the system to keep processing even if huge numbers of validators go offline, such as in the event of a catastrophe.If a validator’s deposit falls below 16 Ether, the Beacon Chain will remove it from the validator set.CrosslinksFinally, the Beacon Chain performs the processing of crosslinks. Crosslinks tie the whole sharded system together, anchoring each shard to the spine that is the Beacon Chain.Periodically, the current state (the “combined data root”) of each shard gets recorded in a Beacon Chain block as a crosslink. When the Beacon Chain block has been finalised, the corresponding shard block is considered finalised, and other shards know that they can rely on it for cross-shard transactions.A visualisation of the Beacon Chain (blue) with 8 shard chains (aquamarine) and crosslinks (light blue lines). Finalised blocks on all chains are yellow. Time increases from left to right. Simulator by Casey Detrio.Buidling the Beacon ChainThus concludes our lightning tour of the soon-to-be Beacon Chain! On its own, the Beacon Chain might not seem particularly useful. It cannot process arbitrary transactions: it has no smart contracts; it has no EVM. You cannot do anything with it. But, as the first component of Ethereum 2.0 to be delivered, it is the foundation of the building. The whole spectacular architecture to come is to be built on this. It has to be solid.If you want to get down into the detail, there is a work-in-progress Beacon Chain specification. All the creation and maintenance of this document is being done openly: anyone with useful input is welcome to join in. There’s all sorts of useful discussion to be had in the issues and the pull requests. If you just want the headlines, I’ve started consolidating the main specification updates and other news in a weekly bulletin.In order to run the Beacon Chain you’re going to need a Beacon Chain client. These are currently being developed separately from the familiar suite of standard Ethereum clients (Geth, Parity, Pantheon, et al.) by a number of teams. You can see a list of the ones that I know about here with links to their GitHub repositories. Prysmatic and Lighthouse are putting out periodic updates on their client development progress, and some of the teams are offering bounties to contributors.Teams working on the Beacon Chain. Slide with links here.As for timescales… the technical specification declares itself to be 60% complete at the time of writing this, and there remains a fairly chunky todo list. Nonetheless, a finger in the air suggests that the spec ought to be reasonably complete by the end of the year, and that perhaps by the end of Q1 2019 we’ll be running a multi-client Beacon Chain testnet. Momentum has been building rapidly over recent weeks and the long discussed Ethereum 2.0 is really beginning to happen!https://medium.com/media/eb70db0a21c4e8715b29a5306af17d39/hrefState of Ethereum Protocol #2: The Beacon Chain was originally published in ConsenSys Media on Medium, where people are continuing the conversation by highlighting and responding to this story.