Benchmarking Lisk Core v3.0.0 against Lisk Core v2.1.6

Our upcoming Lisk Core v3.0.0 (referred to as “Lisk Core v3” below), introduces a large number of improvements on the code level as well as multiple new features on the protocol level.

The purpose of this blog post is to show the overall results of our analyses with regard to the performance improvements, and benchmark them against the performance of Lisk Core v2.1.6 (referred to as “Lisk Core v2” below), which is currently used to operate the Lisk network. To achieve this we focused on four specific test cases, namely query performance, block size growth, block processing, and transactions per second (TPS).

By Lisk

09 Jun 2021

Lisk-Core-3.0.0-Benchmark-SOCIAL@2x.png

Query Response Performance

A query can be defined as a request for information, therefore, in our case, the query response performance defines the time required to get an answer from an API request. In all tests, the time was measured in milliseconds (ms). In addition, in Lisk Core v3 we now support HTTP and WebSocket endpoints. Therefore, both are compared to the HTTP endpoints in Lisk Core v2.

Query response performance is especially important for user interfaces to rapidly retrieve all information from a blockchain. The addition of WebSockets further supports the development of real-time user interfaces.

62x_fa088a1f23.png

On average, the query response performance for HTTP endpoints has improved up to 2.5 times and for WebSocket endpoints up to 8 times. These improvements now make it possible to receive important information such as an account, a block, or a transaction via the WebSocket endpoints within 1 ms on average.

More information can be extracted from the tables below.

Get Account by Address

Lisk Core v2.1.6 HTTPLisk Core v3.0.0 HTTPLisk Core v3.0.0 WS
Min (ms)3.322.240.49
Max (ms)30.8219.9114.29
Avg (ms)4.973.431.03

Get Block by ID

Lisk Core v2.1.6 HTTPLisk Core v3.0.0 HTTPLisk Core v3.0.0 WS
Min (ms)4.962.410.68
Max (ms)35.2614.8021.40
Avg (ms)7.553.721.77

Get Block by Height

Lisk Core v2.1.6 HTTPLisk Core v3.0.0 HTTPLisk Core v3.0.0 WS
Min (ms)5.232.240.68
Max (ms)52.9519.916.88
Avg (ms)8.743.431.10

Get Transaction by ID

Lisk Core v2.1.6 HTTPLisk Core v3.0.0 HTTPLisk Core v3.0.0 WS
Min (ms)3.541.730.41
Max (ms)29.4113.837.13
Avg (ms)5.262.800.65

Block Size Growth

Every blockchain is using a database to store its information. Lisk Core v2 is using the object-relational database PostgreSQL, while Lisk Core v3 is using the key-value database RocksDB.

The block size growth defines how much space a block is using in the database. This is an important metric as it defines how easy it is to maintain a full node and fully synchronize with the network. Both actions are crucial in keeping the network decentralized. It is however also in direct contrast to the scalability of the network, as bigger blocks result in more transactions being able to be processed.

42x_293da257be.png

Therefore, it is important to design the system as efficiently as possible to be able to store as many transactions as possible whilst simultaneously maintaining a relatively well-balanced blockchain growth.

In these tests performed here the size was measured in kilobytes (kB). From analyzing the results it is evident that a large decrease in the block size for empty blocks and a tremendous decrease for blocks containing transactions occurred.

To compare the blockchain size to the current production blockchain operated with Lisk Core v2, we have created a very similar blockchain with Lisk Core v3. With around 3.3M transactions, Lisk Core v2 has a blockchain size of 10GB, and Lisk Core v3 has a blockchain size of 1GB. Therefore the required blockchain size has diminished and is now around 10 times smaller.

Block Sizes in Different Situations

Lisk Core v2.1.6Lisk Core v3.0.0
5 empty blocks1616160
70 empty blocks512400
100 empty blocks560476
5 blocks (64 txs. each)4832328
12 blocks (64 txs. each)10848492
19 blocks (64 txs. each)16080656
30 blocks (64 txs. each)26664924

Block Processing

It is necessary for each full node on the network to process every single block to validate that each transaction in it is valid. This ensures that each network participant is always in possession of the latest up to date and correct status of the blockchain.

In case the full node is also a validator in the DPoS consensus algorithm of Lisk, this further ensures that all transactions sent on the Lisk network are valid and can be confirmed. Hence, this results in the blockchain continuing to progress and ensures that all transactions are being correctly handled.

With a block time of only 10 seconds, it is of key importance that a full node is able to receive, fully process, and forward the entire block with all its transactions within that time frame. Lisk currently supports multiple transaction types, therefore we conducted the test with blocks containing no transactions, regular transfer transactions, and delegate vote transactions. All measurements were performed in milliseconds.

The results clearly show that block processing is now significantly faster. In fact, it has decreased on average by 3 times for empty blocks and over 33 times for blocks containing transactions.

Empty Blocks

22x_3a3944928c.png
Lisk Core v2.1.6Lisk Core v3.0.0
Min (ms)6.5762.777
Max (ms)806.74634.435
Average (ms)18.4074.815

Blocks with Transfer Transactions

12x_0bd44754d8.png
Lisk Core v2.1.6Lisk Core v3.0.0
Min (ms)1999.46751.263
Max (ms)6862.420135.075
Average (ms)2267.96667.456

Blocks with Delegate Vote Transactions

32x_71352ebc04.png
Lisk Core v2.1.6Lisk Core v3.0.0
Min (ms)2207.93364.058
Max (ms)5110.120142.892
Average (ms)2445.14978.625

Transactions Per Second (TPS)

The most widespread metric to compare various blockchain networks is the number of transactions they can process per second. For example, Bitcoin can currently process around 7 TPS and Ethereum is hovering around 18 TPS.

Thanks to the performance improvements of block processing more blocks and transactions can now be validated in less time. Furthermore, thanks to the block size growth improvements more blocks and transactions can now be stored with less space required.

Subsequently, that results in a higher number of transactions being able to be put into a single block, as each transaction has a specific size, and therefore the number of transactions inside a single block is defined by the block size.

With Lisk Core v3, we have defined an upper limit of 15kB per block at a 10s block time to keep the yearly blockchain size growth to under 50GB. This results in 11.25 TPS or 972,000 transactions per day for regular transfer transactions. This ensures that in today’s world all users can easily run a full node to validate all blocks and transactions themselves. Hence, this is a significant improvement as that is 5 times more than Lisk Core v2 is able to process.

52x_abaef3cfa0.png

Conclusion

The upcoming Mainnet v3 migration will not only come with an immense amount of new protocol features but also with key performance improvements to increase stability and scalability. This will result in ensuring that the Lisk blockchain is ready to handle the increased workload with Lisk’s interoperability in the future.