testing layer4 techniques
Over the past 100 days, Jormungandr has used a P2P technique called poldercast to enable communication between nodes.
If you want to know more about the overlays, rings, vicinity and cyclons, here is an excellent summary written by Pal Dorogi: The Hitchhikers Guide To The Shelley – The Network
But somehow something was missing, with which the operator of a node could influence directly and manually which other nodes he wanted to connect to in any case. And this is where layer 4 could come into play. It was part of the network specs for a long time. With the help of community member M.Fazio a first attempt to implement and test such techniques became reality.
In short, the point is that a pool operator adds the “preferred” marker to some of its trusted peers. That’s all. In practice, the node will still use the usual poldercast layers to connect to hundreds of other nodes and not block anything. But it will also try to establish direct connections with these preferred trusted peers.
Now it needs several pool operators to perform the effective test of how something like this reacts and whether it really leads to improved communication between the block-generating pools. The pools often found it difficult to distribute and receive blocks of the short 2-second slots quickly enough across multiple poldercast stations to reliably create their own block. The many passive wallet nodes were not really helpful for this.
Starting around Era 100, some pool operators have decided to try configuring each other as trusted peers and observe how the network reacts.
The following figure shows epoch 100 and a node that works with the standard poldercast still without layer4.
Every 10 seconds a measured value was recorded and compared with several other nodes. If the own node was behind by one or a few blocks, the graph shows with the blue bars. The red lines show a moving average which makes the course and the typical level more visible. The average level of all recorded values in epoch 100 is 0.4
In epoch 101, the preferred trusted peers was activated with about 30 direct connections to other block-generating nodes (Stake Pools)
In the second graph we can see that the level of backlog is not constant but always lower. Partly even much lower, so that the average of all measured values in epoch 101 with 0.22 is almost half as without layer4.
This is a significant improvement!
The third graph shows the transition and the difference between the two epochs without and with layer 4 side by side in one graph. The lower the values, the better the situation of the node to successfully create its block at the right moment. This leads to fewer misunderstandings, forks and thus to a more stable and efficient network. Probably also for all other pools that do not (yet) use this technology.
The developers involved in the implementation were also very pleased with this result:
this is an encouraging results. This show that there is a real benefit from improving stake pools connectivities and with the final step of the 4th layer (the bottle in the sea protocol) we hope to provide this improved connectivity to all stake pools.
Nicolas Di Prima (Team Leader)
After a few more epochs we will have collected enough data to be able to evaluate and report how this affects the network quality and reliability of Block production over time.
Stay tuned!