How to write Tendermint Applications using Python 🌠

source tendermint.com

Byzantine-fault tolerant state machine replication. Or blockchain, for short.

This is the first things you see when you open the Tendermint website. Odds are you already know what a blockchain is, It’s the technology which powers Bitcoin, Ethereum and other decentralized coins you heard of. The basic premise is that people can make applications in a decentralized way, and you can write in any programming language you prefer. Maybe you are confused so let’s back up a bit.

Byzantine-fault tolerant state replication machine — a really large word which makes no sense, unless you are good in decentralized systems. Let’s break up this word.

Byzantine-fault tolerant

It’s better explained using the Byzantine Generals Problem.

Need to remove traitor generals.

This situation can be expressed abstractly in terms of a group of generals of the Byzantine army camped with their troops around an enemy city. Communicating only by messenger, the generals must agree upon a common battle plan. However, one or more of them may be traitors who will try to confuse the others. The problem is to find an algorithm to ensure that the loyal generals will reach agreement. It is shown that, using only oral messages, this problem is solvable if and only if more than two-thirds of the generals are loyal; so a single traitor can confound two loyal generals. With unforgeable written messages, the problem is solvable for any number of generals and possible traitors.

A system is said to be BFT if it can continue to operate even if certain participants drop out or are adversarial. With respect to consensus algorithms, Byzantine fault tolerance normally means the algorithm is guaranteed to converge, or capable of reaching consensus, even if there are adversarial nodes or if nodes drop from the network, etc. In this sense, BFT is a property of a consensus algorithm rather than an algorithm itself.
The idea is that for a system of parties who are decentralized all over the world, for them to have a common agreement or consensus among each other. And for this, there is a need for 2/3rd’s of the people, to be honest.

Hard parts over, now let’s look at the second part.

State machine replication

In computer science, a state machine refers to something that will read a series of inputs and, based on those inputs, will transition to a new state.

With Tendermint’s state machine, we begin with a “genesis state.” This is analogous to a blank slate before any transactions have happened on the network. When transactions are executed, this genesis state transitions into some final state. At any point in time, this final state represents the current state of Tendermint.

The programmer defines the initial state that should tolerate failures and the set of allowable transitions on this state, and the system ensures that every replica starts in the same state, and every replica makes the same state transitions in the same order. This is state machine replication.

Or we can just call it a blockchain. Now let’s get into the details of things.

Writing Applications using Tendermint

Tendermint can replicate deterministic state machines written in any programming language.

TSMP is called ABCI now

Which is pretty good, considering if you wish to make an application for Ethereum you end up learning solidity. Bare with me, it’s a bit tricky.

Start off by installing Tendermint

link

Let’s first run a pre-built application and then understand how to make it our-self. We will use an application called kvstore. It’s a simple application which will take a “key” and a “value” and stores it. So that later you can get the value when you search for the “key”. A very simple database application.

Running pre-built kvstore

The Tendermint team has already build kvstore example into the application.
Better you type these parts as we go, that’s why I didn’t type the output.

# Create the genesis block, for the node.
tendermint init

# Now that the gensis block is inilaised we can start 
# the application on the system 
tendermint node --proxy_app=kvstore

# This will start the kvstore application in your
# computer in port 26657.
# Lets test the kv store application by sending a request to the application
curl -s 'localhost:26657/broadcast_tx_commit?tx="gautham=awesome"'

# You will get a JSON file if its successful

# Now you can check if it actually worked
curl -s 'localhost:26657/abci_query?data="gautham"'

# Now this will give you an output 'awesome' or YXdlc29tZQ== which 
# is awesome in base 64.

Let’s breakdown the process we started an application using the command ‘tendermint node’. The tendermint application runs on port 26657 on your local machine. Go to that port to see the application running smoothly. You have so many methods which you can use.

KVStore using abci-cli

Why don’t we do the same task in a different way, where you don’t have you type curl command always. Tendermint has this tool called abci-console which allows you to send requests easily to the Tendermint application.

ABCI is short for application blockchain interface, which connects the application with the blockchain.

The abci-console allows people to send Transactions easily. Now let us see the methods we used in abci-console to send transactions to Tendermint.

# Lets do in a different way 
tendermint init --home ".node"

# Start a tendermint node 
tendermint node --home ".node"

# Open a new terminal and start kvstore from abci-console
abci-console kvstore

# Open new terminal and start abci-console
abci-cli console
I[2019-01-31|22:16:34.170] Starting socketClient                        module=abci-client impl=socketClient
> deliver_tx "gautham=awesome"
-> code: OK
> commit
-> code: OK
-> data.hex: 0x0400000000000000
> query "gautham"
-> code: OK
-> log: exists
-> height: 0
-> key: gautham
-> key.hex: 6761757468616D
-> value: awesome
-> value.hex: 617765736F6D65

We used deliver_tx to send a transaction, this is the part where the state is updated in our machine. we committed that transaction to store the transaction, this is where the state increases the height of the chain stores the block. After which we query it from the Tendermint node, which is basically looking up the state for the value for the given key.

Let us summarize.

You deliver a transaction, then you commit the transaction and then you can query the transaction.

Now, let us look at how to make the KV Store application as you want.
If you notice I used the home flag, It’s to make sure the genesis token is in the folder named node.

Let’s code our application

I will be writing an application in Python, which is a very simple language I think you can follow it even if you don’t understand it. The full code is here if you wish to see it.

I will be using the py-abci module, which is a wrapper for abci in python. I am not going to reinvent the wheel.

We will start off by creating a state of the Tendermint node. I will add a DB which is MemoryDB, a size to define the size of the state, a height to define the height of the block and an apphash for the application.

class State(object):
    def __init__(self, db, size, height, apphash):
        self.db = db
        self.size = size
        self.height = height
        self.apphash = apphash

Deliver_tx

If you remember the deliver_tx in the above example we send the key and value pair to the Tendermint node. And this key and value are stored in the DB and state is updated.

  def deliver_tx(self, tx):
      """Validate the transaction before mutating the state.
      Args:
          raw_tx: a raw string (in bytes) transaction.
      """
      parts = tx.split(b'=')
      if len(parts) == 2:
          key, value = parts[0], parts[1]
      else:
          key, value = tx
      self.state.db.set(prefix_key(key), value)
      self.state.size += 1
      return ResponseDeliverTx(code=CodeTypeOk)

The data input is in bytes. So we split the string using ‘=’ and save the key and value pair. The size variable we had set will be increased.

Now that a transaction is delivered we need to save the state, for that we use commit.

Commit

During commit, we need to return an apphash, representing the change of state in the chain. I am creating a big-endian notation of the size as the apphash. The height of the chain is increased and we are saving the state.

  def commit(self):
      byte_length = max(ceil(self.state.size.bit_length() / 8), 1)
      app_hash = self.state.size.to_bytes(byte_length, byteorder='big')
      self.state.app_hash = app_hash
      self.state.height += 1
      self.state.save()
      return ResponseCommit(data=app_hash)

Now that we have updated the state, we need to ensure that the application works. So let us make a query.

Query

The query seems straight forward. You check if the value is in the DB and return it.


  def query(self, req):
          value = self.state.db.get(prefix_key(req.data))
          return ResponseQuery(code=CodeTypeOk, value=value)

That’s it. HURRRRRRAAAAAAAAAAAAYYYYYYYY.

We took care of delivering the transaction, then we made the commit and query.

One thing we can ensure is that if the transaction is bad, we can neglect it from entering the mempool of the blockchain. We can use check_tx method for that, but that’s unnecessary for this use case.

Now do check out the whole code or write the rest yourself, in your favourite programming language.

There is another example in Go if you wish to take a look.

Notes

After you start the tendermint node then run your python code to run the application. Then you should be able to send requests just like the kvstore example above.

The tendermint node runs in port 26657 and abci in 26658.

Conclusion

…Phew! You made it to the end. I hope?

There’s a lot to digest in this post, I know. If it takes you multiple reads to fully understand what’s going on, that’s totally fine. I personally read the various parts of the codebase many times and coded the application before grokking what was going on.

Nonetheless, I hope you found this overview helpful. If you find any errors or mistakes, I’d love for you to write a private note or post it directly in the comments. I look at all of ’em, I promise.

And remember, I’m human (yep, it’s true) and I make mistakes. I took the time to write this post for the benefit of the community, for free. So please be constructive in your feedback, without unnecessary bashing. 🙏

Resources

[1]https://www.reddit.com/r/CryptoTechnology/comments/8782mb/what_is_the_difference_between_normal_byzantine/

[2]https://web.archive.org/web/20170205142845/http://lamport.azurewebsites.net/pubs/byz.pdf

[3]http://hackingdistributed.com/2013/12/26/introducing-replicant/

[4] https://medium.com/@preethikasireddy/how-does-ethereum-work-anyway-22d1df506369

# How to write Tendermint Applications using Python 🌠

# Byzantine-fault tolerant

# State machine replication

# Writing Applications using Tendermint

# Start off by installing Tendermint

# Running pre-built kvstore

# KVStore using abci-cli

# Let’s code our application

# Deliver_tx

# Commit

# Query

# Notes

# Conclusion

# Resources