HackOnLisk is open for submissions. Participate now!

Benchmarking Lisk Core v3.0.0 against Lisk Core v2.1.6

Our upcoming Lisk Core v3.0.0 (referred to as “Lisk Core v3” below),  introduces a large number of improvements on the code level as well as multiple new features on the protocol level. The purpose of this blog post is to show the overall results of our analyses with regard to the performance improvements, and benchmark them against the performance of Lisk Core v2.1.6 (referred to as “Lisk Core v2” below), which is currently used to operate the Lisk network. To achieve this we focussed on four specific test cases, namely query performance, block size growth, block processing, and transactions per second (TPS).

 

Please note that Lisk Core v2 is built with the Lisk SDK v2.3.8 and Lisk Core v3 is built with the Lisk SDK v5.0.0, therefore most of the benchmark results presented below are also applicable to blockchain applications built with the Lisk SDK v5.0.0.

 

Furthermore, all tests were conducted on a server from DigitalOcean which consisted of 4 CPUs and 8GB of RAM for both Lisk Core versions. To ensure a high level of accuracy was achieved, we performed all tests 500 times to form lower and upper boundary levels, as well as calculating the average values.

 

Query Response Performance

A query can be defined as a request for information, therefore, in our case, the query response performance defines the time required to get an answer from an API request. In all tests, the time was measured in milliseconds (ms). In addition, in Lisk Core v3 we now support HTTP and WebSocket endpoints. Therefore, both are compared to the HTTP endpoints in Lisk Core v2.

 

Query response performance is especially important for user interfaces to rapidly retrieve all information from a blockchain. The addition of WebSockets further supports the development of real-time user interfaces.

Query Response Times

On average, the query response performance for HTTP endpoints has improved up to 2.5 times and for WebSocket endpoints up to 8 times. These improvements now make it possible to receive important information such as an account, a block, or a transaction via the WebSocket endpoints within 1 ms on average.

 

More information can be extracted from the tables below.

 

Get Account by Address

 

Lisk Core v2.1.6 HTTP

Lisk Core v3.0.0 HTTP

Lisk Core v3.0.0 WS

Min (ms)

3.32

2.24

0.49

Max (ms)

30.82

19.91

14.29

Avg (ms)

4.97

3.43

1.03

 

Get Block by ID

 

Lisk Core v2.1.6 HTTP

Lisk Core v3.0.0 HTTP

Lisk Core v3.0.0 WS

Min (ms)

4.96

2.41

0.68

Max (ms)

35.26

14.80

21.40

Avg (ms)

7.55

3.72

1.77

 

Get Block by Height

 

Lisk Core v2.1.6 HTTP

Lisk Core v3.0.0 HTTP

Lisk Core v3.0.0 WS

Min (ms)

5.23

2.24

0.68

Max (ms)

52.95

19.91

6.88

Avg (ms)

8.74

3.43

1.10

 

Get Transaction by ID

 

Lisk Core v2.1.6 HTTP

Lisk Core v3.0.0 HTTP

Lisk Core v3.0.0 WS

Min (ms)

3.54

1.73

0.41

Max (ms)

29.41

13.83

7.13

Avg (ms)

5.26

2.80

0.65

 

Block Size Growth

Every blockchain is using a database to store its information. Lisk Core v2 is using the object-relational database PostgreSQL, while Lisk Core v3 is using the key-value database RocksDB.

 

The block size growth defines how much space a block is using in the database. This is an important metric as it defines how easy it is to maintain a full node and fully synchronize with the network. Both actions are crucial in keeping the network decentralized. It is however also in direct contrast to the scalability of the network, as bigger blocks result in more transactions being able to be processed. Therefore, it is important to design the system as efficiently as possible to be able to store as many transactions as possible whilst simultaneously maintaining a relatively well-balanced blockchain growth.

Block Size Growth

In these tests performed here the size was measured in kilobytes (kB). From analyzing the results it is evident that a large decrease in the block size for empty blocks and a tremendous decrease for blocks containing transactions occurred. To compare the blockchain size to the current production blockchain operated with Lisk Core v2, we have created a very similar blockchain with Lisk Core v3. With around 3.3M transactions, Lisk Core v2 has a blockchain size of 10GB, and Lisk Core v3 has a blockchain size of 1GB. Therefore the required blockchain size has diminished and is now around 10 times smaller.

 

Block Sizes in Different Situations

 

Lisk Core v2.1.6

Lisk Core v3.0.0

5 empty blocks

1616

160

70 empty blocks

512

400

100 empty blocks

560

476

5 blocks (64 txs. each)

4832

328

12 blocks (64 txs. each)

10848

492

19 blocks (64 txs. each)

16080

656

30 blocks (64 txs. each)

26664

924

 

Block Processing

It is necessary for each full node on the network to process every single block to validate that each transaction in it is valid. This ensures that each network participant is always in possession of the latest up to date and correct status of the blockchain. In case the full node is also a validator in the DPoS consensus algorithm of Lisk, this further ensures that all transactions sent on the Lisk network are valid and can be confirmed. Hence, this results in the blockchain continuing to progress and ensures that all transactions are being correctly handled.

 

With a block time of only 10 seconds, it is of key importance that a full node is able to receive, fully process, and forward the entire block with all its transactions within that time frame. Lisk currently supports multiple transaction types, therefore we conducted the test with blocks containing no transactions, regular transfer transactions, and delegate vote transactions. All measurements were performed in milliseconds.

 

The results clearly show that block processing is now significantly faster. In fact, it has decreased on average by 3 times for empty blocks and over 33 times for blocks containing transactions.

 

Empty Blocks

Empty Block

 

Lisk Core v2.1.6

Lisk Core v3.0.0

Min (ms)

6.576

2.777

Max (ms)

806.746

34.435

Average (ms)

18.407

4.815

 

Blocks with Transfer Transactions

Blocks with Transfer Transactions

 

Lisk Core v2.1.6

Lisk Core v3.0.0

Min (ms)

1999.467

51.263

Max (ms)

6862.420

135.075

Average (ms)

2267.966

67.456

 

Blocks with Delegate Vote Transactions

Blocks with Delegate Vote Transactions

 

Lisk Core v2.1.6

Lisk Core v3.0.0

Min (ms)

2207.933

64.058

Max (ms)

5110.120

142.892

Average (ms)

2445.149

78.625

 

Transactions Per Second (TPS)

The most widespread metric to compare various blockchain networks is the number of transactions they can process per second. For example, Bitcoin can currently process around 7 TPS and Ethereum is hovering around 18 TPS.

 

Thanks to the performance improvements of block processing more blocks and transactions can now be validated in less time. Furthermore, thanks to the block size growth improvements more blocks and transactions can now be stored with less space required.

 

Subsequently, that results in a higher number of transactions being able to be put into a single block, as each transaction has a specific size, and therefore the number of transactions inside a single block is defined by the block size.

 

With Lisk Core v3 we have defined an upper limit of 15kB per block at a 10s block time to keep the yearly blockchain size growth to under 50GB. This results in 11.25 TPS or 972,000 transactions per day for regular transfer transactions. This ensures that in today’s world all users can easily run a full node to validate all blocks and transactions themselves. Hence, this is a significant improvement as that is 5 times more than Lisk Core v2 is able to process.

Maximal Transactions Per Second

Conclusion

The upcoming Mainnet v3 migration will not only come with an immense amount of new protocol features but also with key performance improvements to increase stability and scalability. This will result in ensuring that the Lisk blockchain is ready to handle the increased workload with Lisk’s interoperability in the future.

 

For further information, you can find our Lisk.js 2021 presentation of Benchmarking Lisk Core 3.0.0 below.

 

Lisk

Blockchain application platform