Introduction
The Beacon chain is organised in epochs of 32 slots, where every 12 seconds (a slot), a randomly selected validator has the chance to propose a new block. Other validators are in charge of saying whether the block that has been proposed is valid. They do it by broadcasting a vote, which we call Attestations.
Inside these attestations, the validators have to state their three main perceptions of the chain:
- The Source: the last justified checkpoint that they see at that moment in the chain.
- The Target: the hash of the first block at the current epoch.
- The Head: the hash of the last block they have seen (the just proposed one).
Validators produce these attestations once per epoch, meaning around 880,000 votes (at the time of writing) every 6,4 minutes. This is where the p2p network plays an essential role. All these attestation messages are distributed over the 32 slots of the epoch, roughly 27,000 attestations per slot. These are divided into 64 validator committees, approximating 433 validators per committee.
Each of these committees is assigned an index, and they are expected to advertise their votes on a dedicated GossipSub topic, a.k.a. a subnet, for that particular index. This way, validators don’t need to waste resources validating and propagating attestations they are not interested in.
The Problem and Proposed Solution
With the premise that you know the duties of your validators two epochs in advance, one could expect not to have issues finding and connecting peers on the attestation subnets where the validator needs to send its attestation. However, 12 minutes and 48 seconds might not always guarantee a direct connection with the eight desired peers that ensure the best message propagation in the network. The whole problem is described in this Issue of the Consensus specs. Beacon nodes had to subscribe to the same number of subnets as they had validators connected. This means nodes with 64 or more validators are subscribed to all subnets and those without are subscribed to none. The more subnets a node subscribes to, the more resources that node requires. The load on clients was being distributed unevenly across the network. This means that nodes with no (or few) validators are likely less useful to the other nodes, and it also makes nodes with many validators very easy to identify. Researchers of the EF and core developers of the main clients realised that this had the significant drawback of making the peer discovery of nodes subscribed to a particular topic more complex and slower, and the network as a whole could be consuming more resources than it needed to.
Their proposed solution, approved on this Pull Request, was to make all the nodes subscribe to at least two subnets. The specific subnets should be decided based on their node_ID. These mandatory subscriptions, also called long-lived subnets, will slightly increase the bandwidth of those nodes that don’t host validators. At the same time, this will make looking for peers subscribed to a specific subnet much easier, as one can predict which nodes participate in a particular attestation subnet. If a given peer does not subscribe to its subnet duties, other nodes could downscore it, and therefore, the non-cooperative node would be penalised.
Observations in the Wild
One of the cool things about having a feature-full crawler is that one can observe these changes in real-time. Here is where the tools developed by our team come in handy.
All CL core developer teams have already implemented these changes on their clients. Most of them have released it on their respective versions, i.e., Lighthouse, Teku, Lodestar, and Nimbus, while the Prysm team is still testing the changes before merging it.
Thanks to the network overview provided by the network crawler Armiarma, we have observed the evolution of the number of nodes subscribed to X number of attestation subnets.
Figure 1 shows the number of nodes subscribed to 0, 1, 2, and 64 attestation subnets from the beginning of May 2023. As we can see, the number of nodes subscribed to two attestation subnets has increased substantially. It went from barely 2,000 nodes at the beginning of May to almost 6,000 in November. In other words, the number of nodes subscribed to two attestation subnets has multiplied by three in the last six months. We can also observe a slight drop between peers subscribed to zero attestation subnets and peers subscribed to only one. This clearly shows that nodes, by default, are starting to be subscribed to at least two of the GossipSub topics (subnets) as the change in the spec proposed.
 |
|---|
| Figure 1. Evolution of the number of active nodes’ subnets subscription. |
As Lighthouse is one of the two more present nodes in the Ethereum network, we tried to correlate the sudden spike of nodes from June to the release of the first Lighthouse version, including the changes, v4.3.0.
Figure 2 aggregates those Lighthouse versions, from v4.3.0 to v4.5.444, with the distribution of the attesting network subscriptions (0, 1, 2, and 64). As we can see, the rising slope of nodes subscribed to two attestation topics accelerates its growth near the Lighthouse’s release at the beginning of July. The large utilisation of Lighthouse nodes in the network clearly correlates to this event observed by our crawler.
 |
|---|
| Figure 2: Correlation between Lighthouse releases and the attestation subnet subscriptions. |
We also notice that the small increase in nodes subscribed to two subnets starts slightly before the Lighthouse v4.3 release. In addition to the normal increase due to new nodes joining the network with two subnets, we suspected this bump could be related to the Lighthouse team running nodes to test their code on Mainnet before the official release. We discussed this with the Lighthouse team, and they confirmed our suspicions. A similar thing happens with Teku v23.9.0. Figure 3 shows that the second spike of nodes is closely related to the changes in Teku’s latest releases.
 |
|---|
| Figure 3: Correlation between Teku releases and the attestation subnet subscriptions. |
The case of Lodestar is slightly different because the number of Lodestar nodes implementing the changes is low to make an observable difference. Barely 30 nodes updated to the new version. However, it is good to see that the development team quickly adapts to the latest changes in specs.
In the case of Nimbus, there are some technical difficulties in measuring its impact. The Nimbus nodes have never shown their client version on their network user_agent. All other teams use that common libp2p field to tag their nodes in the network with the client and version. Armiarma crawls the network, looking for this kind of information. Not explicitly showing the client and/or version can slightly improve the privacy and security of the node, in particular, if a significant exploit is found for a given version of the client. However, not doing it prevents us from getting more detailed data that can help us understand the status of the network, the adoption of newer versions, and the interoperability across clients.
Conclusion
Figure 4 shows the global picture of adopting Lighthouse, Teku, and Lodestar versions that include deterministic subnets and the number of nodes subscribed to 0, 1, 2, and 64 attesnets. Overall, we can see that adopting the changes has been very noticeable from the network’s perspective, particularly since the first release of the changes in Lighthouse. These types of observations help us validate the data provided by the tools we build, such as the Armiarma crawler in this context.
 |
|---|
| Figure 4: Correlation between clients supporting the changes on the specs and the attestation subnet subscriptions. |
This study also shows that many users running non-validator nodes seem to run Lighthouse and Teku clients. This can be related to these clients' excellent API performance, making them a great option for collecting and indexing Ethereum on-chain data.
Finally, we hope the remaining nodes will update to a version supporting this network change in the spec. This will help with message propagation by increasing the number of peers in each subnet, which is essential in the long term, particularly for data availability sampling.
