AMA Recap: Research
In this blog post we recap the most interesting Research related questions from the community members during the live AMAs (Ask me anything) hosted on Lisk.chat this year, as well as the answers of our Research team members: Maxime, Alessandro, Iker, and Jan.
The research blog posts that these AMA sessions covered are listed below:
- Multisignature Accounts
- New Lisk ID System
- Introducing Lisk Codec
- Introducing Lisk Tree
- Lisk Mainnet Regenesis
Multisignature Accounts AMA
Crypto202006: Hi, what would be the use for my second passphrase after implementing the new proposal? Do I need to do anything to protect my Lisk account?
Maxime: No, you don't need to do anything. Your second passphrase is linked to a "second public key" and this key will remain a mandatory key to send transactions from your account.
Jurre | Moosty: What were the main reasons to change the multisig? Or phrased differently: why would you change the way it works?
Maxime: I would say the main reason is flexibility. IMHO the two main features that are included are:
- arbitrarily split between optional and mandatory. - option to not include the original public key.
If you want to increase the security of your account, those features are not necessarily needed. But if you want more complex uses, like an account for a local club, or even for a family account, then it becomes useful. For example, I could create a new account with my partner and we could both use our (already existing) personal key to register this account as multisig. This way we each only have one key and nobody needs to remember the original key used to create the common account.
przemer: Are there any disadvantages with the new way of signing multisig transactions compared to the old one? For example the need of internal coordination before sending a transaction to the network.
Maxime: In terms of "multisignature account" possibilities, the new way is more general and flexible. In terms of the network, it does require the transaction to be fully signed before being sent to the network.. From the UX perspective, there might be other tools that facilitate this.
Jurre | Moosty: “Securing enough self-votes to be an active delegate might be easier to achieve for a group than for a single individual”. I am not sure how to read the above exactly. Do the related accounts count as "vote weight for the self-votes" or do you mean that you need to transfer the tokens to the multisig, but it would be easier with more people together?
Maxime: With the new voting system, you will need some self votes. So possessing enough LSK could be challenging for one person. So together, the participants create a new multisig delegate account and each sends some LSK to that account. This way, this account can vote for itself and have a lot of self-votes.
Jurre | Moosty: Could you provide a quick comparison between lisk multisig and other platforms? Or is this not easy to do in a few lines?
Maxime: Food for thought, I can give you some differences between our multisig and an "aggregated signature" type of multisig. In our proposal, the keys are registered on the network and then all the signatures have to accompany the transaction for it to be valid.
You could however also try to create a multi signature account by using the fact that Schnorr signatures can be aggregated (with some care). In this paradigm, the account itself only has one public key (so from the outside it looks like a regular account), but nobody has the private key to the account and the only way to sign a transaction is to create an aggregated signature with the other participants.
This solution does not really support optional keys though, and it requires that the participants create the multisig account in a particular way (to avoid some attacks). Different types of signature also allow aggregation, for example, in Cosmos, the signature of multiple delegates are used to sign the block headers. And the block is only valid with enough signatures.
New Lisk ID System AMA
carbonara: I have a question regarding the reclaim TX (https://lisk.io/documentation/lisk-protocol/mainnet.html#%5Freclaim). How does that actually work? From where does it have to be sent?
Iker: So first you have to know this only applies to those accounts that have no outgoing transaction yet (therefore no public key is associated). I assume that nobody in this chat is in this situation. Assuming you are a user with an old unused account, without outgoing transactions, and then the new ID system is released on the Mainnet and Lisk platform migrates to the new address format.
Then in this case, to be able to claim your funds in that old account (no registered public key, only an old address registered in it), you have to send this special Reclaim Transaction into the network. This is just a simple transaction where you will need to specify the amount that you have in that old account. With that, the Lisk protocol will be able to merge the old account to the new address,and this way the user will be able to have access to those old funds.
przemer: The invalidation of a wrongly typed address is done on the client side right? if I broadcast it directly to the nodes API, would it go through?
Iker: Yes, the error detection capabilities of the new address are at the user interface level, i.e., in the Base32 format address. The nodes will not directly deal with this format, but with the compact hexadecimal format, which is what will be inserted in the receiver address property the balance transfer transaction.
As long as you are handling "raw" data in hex format, the address is just the first 160 bits of the hash digest of the public key, that is all. However at this level, I would say that"mistyping" errors in an address should not apply and copy-pasting mistakes should also be very limited.
Introducing Lisk Codec AMA
dav1: How will the calculation be done of the payload and payloadLength of a genesis block on a sidechain if I decide to have different initialization transactions there? Will it be possible that Lisk Codec will offer all methods for it?
Maxime: If I understood, you asked about the size of the genesis block. With the new serialization, you will provide a schema for all objects that your blockchain uses. This schema is used by Lisk Codec and the result can be measured in bytes.
Sgdias: From a db schema perspective, how will you keep retrocompatibility/consistency with previous blocks not using the encoding when this goes live?
Maxime: That is a good question. The basic rule is: "everything before block N is serialized according to the old rules, and everything after is serialized according to the new rules". In essence, if you want to recheck old signatures, you will need the old serialization method.
Jong: I don't fully understand the benefits of this codec change. In the past, Lisk had used binary serialization for blocks and found that it did not affect the size of blocks at all. Seehttps://github.com/LiskHQ/lisk-sdk/issues/1621#issuecomment-369325568 Another counterpoint is that protobuf messages after compression are typically only about 9% smaller in size compared to JSON (which is not a significant difference). A supporting point can be made that protobuf is faster than JSON at encoding/decoding. However, based on my understanding of the Lisk code, the serialization/deserialization represents a very small portion of the total workload of a Lisk node (it's not a bottleneck), so I don't fully understand the benefits of this codec change. Are there other benefits aside from size (bandwidth usage), and encoding/decoding performance? Does some other LIP depend on it?
Maxime: thanks for your question. Firstly, yes, the encoding time represents a rather small part of the whole processing and it is not the main motivation to use Lisk Codec. The need for the codec is three fold, (you have a more detailed explanation in the blogpost and the LIP):
1) Uniqueness: If we rely on other tools to generate the binaries and that the output for those is affected by anything (for example the node version), then you might create different binaries and reject signatures that others might accept. This would be a "worst case scenario".
2) Size: It makes a significant difference (much larger than 9%). If you look for example at the signature in a JSON version of a transaction, it is currently displayed as a hexadecimal string of length 128. With Lisk Codec this goes down to 64 bytes. We could have changed that by displaying signatures in JSON in another format than hexadecimal, but this is not as "human friendly" and would have needed a change anyways (so might as well reap the other benefits of Lisk Codec).
3) Developer experience: You can put all the burden of writing a getBytes function on the shoulders of the SDK user, but this would go against the idea of making the SDK as easy and as user friendly as possible.
Introducing Lisk Tree AMA
Corbifex | Moosty: For someone who is not technical and knowledgeable in the different Merkle Tree implementations on other projects, is Lisk implementing Merkle Trees in a different way than everyone else?
Alessandro: Different projects employ Merkle trees for different reasons. In general, you can identify two "flavours" of Merkle trees: regular ones and sparse Merkle trees. Lisk-tree is an implementation of the regular Merkle trees. In simple terms, the main difference is that regular Merkle trees are created from an underlying dataset (like a set of elements) that get progressively hashed together up to the final root.
In some implementations, like ours, you can also append new elements to the tree efficiently. A similar construction is used, e.g. by bitcoin. You can find more details here. The other flavour are sparse Merkle trees. In sparse Merkle trees all possible elements are present in the underlying dataset, and they are identified by their index.
An example is worth a thousand words: Let's build a sparse Merkle tree for the Lisk account state. Imagine that all possible Lisk accounts are present in this dataset, where their address corresponds to their index in the dataset. Many (the vast majority) of these accounts will just be empty, giving empty leaf nodes in the tree, but those that correspond to an existing account will instead have the serialized account stored in the leaf node. Now, this obviously comes with a problem.
The resulting tree is enormous and, well, sparse (nomen omen). Therefore you can use some tricks to reduce the tree size. Ethereum uses a specific implementation of a sparse Merkle tree, called patricia trie. Details are not super important here.
To add on the previous point, notice that the initial Merkle tree implementation in bitcoin had some flaws. In particular, it was possible to mutate the block payload without mutating the block ID, while at the same time making the block invalid. Because of some specific choices in the P2P network layer, this could have allowed an attacker to isolate a target node.
To conclude, sparse Merkle trees are more complex than regular Merkle trees, but also more powerful. In a sense, it is easier to update them, because the order of insertion of data in the tree does not change the resulting Merkle root (since all data is indexed, and therefore its position in the tree is fixed). This is not the case for a regular Merkle tree.
Because of this, if you want to calculate something like an account state root, sparse Merkle trees are better suited. As mentioned earlier, lisk-tree is the best way to solve the concrete problems we wanted to solve. In the future, we may enhance it with more functionalities.
Corbifex | Moosty: Sparse tree sounds to me more like a database.
Alessandro: Yes! It is in a sense a key-value store with extra structure on top of it. But that's exactly the point of Merkle trees (and in general one-way accumulators). You want to condense a lot of data into a single hash, with the extra feature of having efficient proofs-of-inclusion. Actually your one-liner is a much better way of explaining them, I might steal it.
Jurre | Moosty: What will be your next topic of research?
Alessandro: The whole research team is working on interoperability right now.
superIPFS: how many approximate lines of code does it take to implement lisk-tree into lisk core? & does introducing this require a HF?
Alessandro: I'm not sure about the first question, but it's not too large. Lisk-tree per se does not introduce a hardfork, but for example introducing a transaction root instead of the payload hash in block headers would be a hardfork, however note that all of this will be implemented after re-genesis with the new SDK version.
Lisk Mainnet Regenesis AMA
udi: Hi, is there an intended date for the hard fork?
Jan: No, we don't have a date yet. We finished development and internal QA for Lisk Core 3.0, but we first will have a betanet and testnet phase for the community to test the implementation as well. Note that Lisk Core 3.0 and Lisk SDK 5.0 are also undergoing an external security audit at the moment.
So only after the security audit and also a sufficient time on the public testnet we would announce a date for the Lisk Mainnet hardfork. A mainnet hardfork requires good planning and is always complex, not just software-wise, but also coordination-wise with all delegates and exchanges, for instance.
udi: Will Testnet have a similar hard fork?
Jan: We haven't decided if we also do the regenesis approach for the testnet or start fresh. But migrating testnet could be a good way to test the whole hardfork.
udi: How do you expect transaction fees to change?
Jan: The new dynamic fees should determine a price depending on supply and demand. Currently, on Lisk Mainnet there is a lot more supply (block space) than demand (transactions). Therefore, I expect that transaction will only pay the minimum fee for a while.
superIPFS: “To verify the correctness of the whole Lisk blockchain from the genesis block onwards, it is now necessary to run two implementations, first Lisk Core 2.x and then Lisk Core 3.0” (quoted from blog)
- will HQ host a "snapshot block"? - if we obtain a "snapshot block" from HQ, will we still need to run Core 2.x to setup a Core 3.x node?
Jan: We will also host a snapshot block in the future so that it would be possible to obtain it from us. It would be more convenient, but not as trustless compared to you creating it yourself with a Lisk Core 2.x node. At some point we can also fix the hash of the snapshot block in the Lisk Core 3.0 implementation so you could obtain the block from anyone in the network.
As a node operator joining Lisk Mainnet after the hard fork, you have the option to compute the snapshot block yourself using Lisk Core 2.x or to obtain it from some trusted source allowing for faster synchronization with the network. So you don't have to run Lisk Core 2.x if you have the snapshot block.
MrV: I'm assuming HQ will do everything they can to communicate these changes with exchanges and with enough time to not cause issues withdrawing/depositing” and added: If enough delegates are running 2.x and 3.x nodes, I'm assuming there will be 2 networks running then and exchanges that don't update in time will be using the 2.x fork that really doesn't have value anymore, unless it becomes lisk classic but people could try and sell these useless 2.x tokens.
Jan: Yes, good observation. That is always an issue with a hard fork. The previous protocol version could continue running so that there are two networks. Exchanges need to be aware and typically stop deposits/withdrawals for a while before the hardfork until some time after the hard fork.
przemer: So how will the migration process look like exactly from a delegate point of view? Let's say it's scheduled for block height 20.000.000, and the delegate is 20 blocks before the hardfork. So he waits for the regenesis block, computes the snapshot, and then runs core 3.0. with the snapshot? How long does the computation of the snapshot take? What if he's the next one to forge after the regenesis block?
Jan: Good question! So the timestamp of the snapshot block is actually 2 hours after the last Lisk Core 2.x block (height 20,000,000 in your example). This means that there are no blocks for 7200 seconds (2 hours). The idea is that this allows for sufficient time to compute the snapshot block and start the Lisk Core 3.0 node. Assume that snapshot block has timestamp T. Then from time T onwards delegates get assigned their 10 second forging slots. If you miss, then the delegate in the next slot forges afterwards. So it is not different compared to now if someone misses their slot.
Thanks to everyone in the community for their participation in the Research related live AMAs on Lisk.chat. We would like to invite all community members to go to the Lisk Research forum, where we are always happy to hear your feedback and to participate in discussions on Research topics.