A Shallow Dive Into Bitcoin’s Blockchain

From natural objects to coins, to paper money backed by gold, to just paper money, to digital money. Whatever the format, money serves as a mean of exchange, a method of payment, a standard of value, a store of wealth.

Today most of the transactions are becoming just digital. For example, this is what happens when you buy stuff from a merchant using your card:

1. You are being authorized by entering your PIN to the merchant’s machine

2. Transaction and card data are sent to the merchant’s bank

3. Merchant’s bank (Acquirer in the figure below) forwards the details to your card’s organization (Visa, Mastercard, etc.)

4. Then the card organization contacts your bank (issuing bank in the figure below) to make sure the card is valid and has money in it

5. Once the merchant receives the OK message back, they can give you the goods/services

Acquirer is Merchant’s Bank. Issuing Bank is the Customer’s bank. Source

6. At the end of the day merchants’ bank sends all the valid transaction data to the card organization (Visa for example) for settlement

7. The card organization forwards all merchant’s transactions to the customers’ banks (issuing bank)

8. Customers’ banks will need to transfer the funds to the card organization

9. Card organization pays the merchants’ bank less an interchange fee

10. Merchant’s bank deposits the funds to the merchant’s account, less a fee

Acquirer is Merchant’s Bank. Issuing Bank is the Customer’s bank. Source

Card organizations act as the middleman for financial institutions. They are facilitating transactions among people’s bank accounts. It’s like they are on top of banks giving you secure access to your funds with just a card.

This article, however, analyses how we can securely transact amongst ourselves.

Let’s try to build a system without having to depend on any organization.

The first approach would be having a public place where we save all transactions happening among us. A database ledger, for example, on a public server somewhere. So, whenever a transaction is happening we would save it on that database. A digital wallet would automatically send the details of a transaction to that server.

But you can quickly see problems arising with this approach. Who will be in charge of such a database server? The one with access to the server will have the power to add, edit or delete a transaction however he pleases.

To remove that bit of trust we can have every participant of the network keep a copy of that database ledger on their machine.

But here is another question. How can participants keep this ledger in sync? How can you get everyone to agree on the same ledger?

To address this challenge, central authorities are substituted using a consensus algorithm called Proof of Work (PoW).

Sidenote:
The term for PoW is “mining”. And the purpose of mining is not to find new coins as the term mislead, but rather to keep the blockchain secure.

The basic idea is to trust the ledger with the most computational work done on it.

Remember from a participant’s point of view he can trust no one!

On the bitcoin network, miners put together transactions in a block and broadcast them to the rest of the participants. Not all participants need to create blocks (be a miner) though. Participants can also just listen and verify the validity of new transactions; the transactions that they are interested in.

A wallet app on a mobile phone, for example, can also participate but not necessarily support the functionalities that a full mining participant runs (this includes verify and broadcast all newly created transactions and blocks, create new blocks, etc). This would require lots of computational power and Gigabytes of space.

Alice, running a full mining node, just found a new block and broadcasted it to the rest of the network. Bob, being a participant in the network, listening for new blocks of transactions from the miners, receives the newly created block. Source

Let’s say you are a participant on this network. How can you trust that miners do their job right? In other words, make sure that the blocks of transactions that you receive from them are valid, and at the same time, everyone else records the same blocks.

The way you trust the miners is by validating that they put some computational effort (PoW) into creating those blocks. This mining process will be discussed later in more detail.

Bad actors can put some effort as well and send you blocks right?

Correct! But there here is the caveat:

Because it takes time and effort to create the PoW for those blocks, bad actors can’t just trick the whole network easily.

Assume that you receive a block with a bad miner’s transaction in it. After a little while, you hear another block from a good miner but with a conflicting transaction. From your point of view, you don’t really know which of the two blocks to trust.

The key point is to keep both blocks and create two separate blockchains (a fork event) and keep listening for new blocks; along with their PoW.

For more details see this animation’s caption:

In this example, Alice is trying to cheat the network by trying to convince it that her version of the chain is the valid one (and thus the transactions in it). Bob, who is a participant in the network, is trying to understand which blockchain is the correct one. Source

A fork is not necessarily a bad thing. It happens every day in the network. It occurs naturally as a result of transmission delays in the global network. i.e. two different miners finding and transmitting a block at the same time (each miner will include different transactions or transactions in a different order when creating a block and it can happen for 2 different miners finding a nonce for their blocks at the same time). But as long as all nodes wait and select the longest chain, the network eventually converges to a consistent state (consensus through the network).

Visualization of a blockchain fork event: two blocks propagate at the same time. Some nodes receive block “triangle” first and some receive block “upside-down triangle” first. Source

Therefore, if bad miners’ blocks do not eventually make it and convince the rest of the network, all the effort is wasted, among the resources they used to make it. And that’s one of the factors that disincentivize bad actors from cheating.

What kind of computational work do miners need to do in order to prove that they put some effort into the creation of a block?

To understand this let’s explain cryptographic hash functions first.

The first thing to note is that everything in the digital world boils down to bits (0s and 1s).

A cryptographic hash function is a mathematical function that takes any size of bits (this is the input message — a text, an image, a number) and outputs a fixed length of 256-bits. This output is called the hash value, digest or simply the hash of the message.

Cryptographic hash functions have these properties:

1. It is infeasible to modify a message without changing the hash value. If you slightly change the input message, even one bit, the resulting hash value changes completely!
2. It is infeasible to find two different messages with the same hash value. The same input message (set of bits) will always give the same hash value (256-bits) and two different messages (sets of bits) will give different hash value (256-bits).
3. It is infeasible to generate a message from its hash value. Having the 256-bit hash value you can go backwards and find the message used as an input.

Hashing comparison. Note that for readability hexadecimal notation is used for the resulting hash. One hex value represents 4 bits. 64 hex values times 4 bits = 256 bits. Source

Bitcoin uses the SHA256 hash function.

How can such a function prove that the participants did spend a lot of computational power? In other words, how do we know that a miner actually did the work required for a specific block of transactions?

The rule is that miners will have to find a number, called nonce, that when added to the block of transactions and hashed together will result to a 256-bit hash (called the block hash) that starts with a specific amount of 0s. This amount of leading 0s is defined by the difficulty level.

 

Remember in the digital world everything is represented in bits:

Each block in the blockchain contains various information about the block (transactions, creation time, etc.). This information is represented as a set of bits as well. Those bits are unique for each block because it has a unique creation time and set of transactions.

What makes it hard work for the miners is to find an extra number of bits (the nonce number) that when added to the existing block’s bits, and hashed together using the SHA256 hash function, will result to a hash value with a specific set of 0s (defined by the difficulty level). The only way to produce a hash value matching a specific target is to try again and again, randomly modifying the input (by trying a different nonce number) until the desired hash value appears by chance.

Take time to read that again. That’s the cornerstone of Proof of Work.

Hashing numbers to find the nonce that satisfies the difficulty level. Source

This number (nonce) acts as proof that a certain amount of work was done to produce a resulting hash that matches a specific target. Thus the term Proof of Work.

The nice thing about using this method is that every participant can easily verify that they did a large amount of work, without having to do the same effort. Having 1. the block’s data, 2. the nonce and 3. the resulting hash, anyone can quickly validate its correctness using the same function.

A block that satisfies the difficulty level is considered valid.

So, if a bad actor was about to change a transaction (essentially some bits in the message), the block hash would be different and most probably not starting with the required amount of 0s and as a result, invalidating the block.

The block hash acts as a digital fingerprint, or a block identifier lets say. It identifies a block uniquely and can be independently derived by any node by simply hashing the block’s data (nonce, transactions, difficulty level, timestamp, etc.).

One thing I haven’t mentioned is that a bad miner, could not just change any transaction. There as an extra bit of security which I will discuss on Part 2 — transactions. But for now, take this for granted.

Having this extra bit of security, a bad miner could only change one of his own transactions.

So in order to cheat the network, he will have to go to the specific block in the chain (the block where his transaction is saved) change the transaction in his favor and re-do the work of finding a block hash that satisfies the block’s difficulty level. Propagating such a block to the rest of the network would be considered be totally valid.

Unless we add another bit of security.

All the blocks are connected with each other! By hashing the previous block’s hash and not just the transactions, nonce, and timestamp, we get this tight relationship between all blocks.

Blocks’ contents. All transactions (Tx0, Tx1, Tx2, Tx3) can be represented with just one hash number, called the Merkle Root (Tx_Root in the diagram). Source

So, if you were about to change any block, or try to swap the order of two blocks, it would change the block after it. And since a single change on a block changes its hash, it will change the next block’s hash, and so on, invalidating all blocks after it. That would require redoing all the work, finding a new special number for each of these blocks that makes their hashes start with a specific number of 0s. This requires lots of work!

Side note:
At the time of writing the hash rate of the network is around 65 quintillion Hashes per second. A bad actor will need to have at least half of it to trick the network (known as majority or 51% attack). Since the attacker can generate blocks faster than the rest of the network, he can simply persevere with his private fork until it becomes longer than the branch built by the honest network. However, the current network’s hash rate makes it financially impossible (the machines and energy needed cost millions of dollars). And that’s just to change one of his transactions. So, it better be worth it!

Difficulty level

Earlier I said that the PoW is to find a special number (nonce) so that the hash of the block starts with a specific number of 0s.

The more leading 0s the harder for the miners to find a nonce that satisfies that hash. That’s because the range of acceptable resulting hashes becomes smaller.

But why making it hard for the miners in the first place?

The target is to keep the difficulty of mining one block to 10 minutes for the entire network.

This is a design compromise between fast confirmation times (settlement of transactions) and the probability of a fork. A faster block time would make transactions clear faster but lead to more frequent blockchain forks, whereas a slower block time would decrease the number of forks but make settlement slower.

The difficulty parameter is adjusted dynamically, every 2016 blocks, to keep the block interval at the 10-mins target. So, the formula is that for every 2016 blocks the miners' average block creation time is calculated and if that time is more than 10 minutes the difficulty parameter is going to be adjusted. If for example, it took the network on average 12.5 minutes to build a block (20% more than the 10-mins target), the difficulty will be lowered by 20%, for the next 2016 blocks. This revision of the block creation time follows the improvements on the miners’ hashing power (essentially the hardware).

Side note:

Today lots of miners are coordinating together in a mining pool competing for the creation of a block. They split the work of searching for a solution, earning a share in the rewards. This is centralizing the whole idea of mining but it also has some advantages. But, in any case, as current financial institutions are closely watched for inside trading, big mining pools are also being watched for misbehaviors. Any misbehavior would make them lose credibility or even worse, lose users' faith in Bitcoin, dumping the whole project into the abyss. This would put them out of business.

Today, mining pools act as a bank, keeping people’s funds secure. But, at the end of the day, most of them are centralized corporations! 😉Peer-to-peer pools also exist. But they are a minority.

Funds are kept in digital wallets. Or I should rather say tracked by digital wallets. There is a common misconception that your coins are stored in your wallet. Coins (or should rather say numbers) are actually stored into the blockchain in form of transactions; transactions that are created by digital wallets.

Digital wallets are nothing but a piece of code that tracks and calculates the owner’s balance by scanning transactions within the blocks of the blockchain.

Transactions are structures within the block structure. The whole chain of blocks is stored into a bitcoin participant’s hard disk.

One of the main wallets’ functionalities is to sign transactions on behalf of its owner.

They are a bit like real-world signatures, but much stronger. Digital signatures can:

1. Prove that the owner of an account is the only one who authorized the spending of its funds
2. Proof of authorization is undeniable (non-repudiation)
3. Prove that the transaction has not and cannot be modified by anyone after it has been signed

Private/public keys and bitcoin addresses

To achieve this, each account owner in the network has to create a public/private key pair. They are considered a “pair” because the public key is derived from the private key. Your private key is nothing more but a number! A 256-bit number between 0–2²⁵⁶(remember in the digital world everything is represented in bits). That’s a huge range of numbers, which makes it statistically infeasible for 2 people getting the same keys (there are 10⁷⁷ keys available).

These keys are based on mathematical functions that have a special property: it is easy to calculate them, but hard to calculate their inverse.

A public key is a mathematical result of its associated private key. And even if you know the mathematical function that created it, you can’t infer its private key.

Your public key is derived from your private key using elliptic curve multiplication. Your address is derived from your public key. Both of those actions are irreversible. Source

It’s good to mention here that your Bitcoin address is derived from your public key through the use of a one-way cryptographic hash. Your address is the way to transact with the rest of the bitcoin owners. It just represents the destination where bitcoins need to be sent.

 

The way you create your private key though is super important. You would never choose the number 1 as your private key. That’s too dangerous! Anyone, using the same mathematical functions, can infer a bitcoin address from a private key. And if that bitcoin address owns coins, they can easily be stolen.

In fact, if you run a script that tries every number (private key), counting from the number 1 to 100,000, you will find (in some seconds) dozens of usable bitcoin addresses! In order to find if an address is usable (an address owning some coins in the bitcoin network), one has to iterate through the entire blockchain and if a reference to that account is found, Boom! One can steal all the coins from it using that weakly generated number (private key).

In fact, the bitcoin address derived from the private key number 1 is usable: 1EHNa6Q4Jz2uvNExL497mE43ikXhwF6kZm.

If you click the link you will see that whatever comes into this address is withdrawn within a few seconds! As soon as money comes in, someone can quickly create a transaction (using his own address as the destination), sign it with the private key number 1, and propagate it to the rest of the network!

Great! Why don’t just everyone do the same then?

The reason is that checking all possible private key numbers is a very difficult task. For example, it would require about 3×10⁵¹ years for fifty supercomputers (that check a quintillion keys per second!) to exhaust the whole 2²⁵⁶ keyspace.

Poor private key generation algorithms

Some wallets used to create private keys (essentially numbers) using passwords provided by humans. Hackers, however, can easily check weak or commonly used passwords pretty quickly (this range is much smaller).

Examples of accounts with stolen private keys (derived from weak passwords):

asd — 1G4Mt5JLtrdj4hM6MkyaQpHmZzVoojLFX3
cat — 162TRPRZvdgLVNksMoMyGJsYBfYtB4Q8tM
hello — 1HoSFymoqteYrmmr7s3jDDqmggoxacbk37
password — 16ga2uqnF1NqpAuQeeg7sTCAdtDUwDyJav
test — 1HKqKTMpBTZZ8H5zcqYEWYBaaWELrDEXeE
fuckyou — 1LdgTMX2MEqdfT3VcDpX4GyD1mqCP8LkYe
1–12AKRNHpFhDSBDD9rSn74VAzZSL3774PxQ
icecream — 1CwjHYsPUc4Du8dx7AkdBJj4ebWC8bxkF3
alfanumerico — 19JsLFDRxuTsAjapE79FgoVNdNdB2hNU5M
[empty string] — 1HZwkjkeaoZfTSaJxDw6aKkxp45agDiEzN

 

Digital signatures

The process of signing transactions involves a mathematical function that depends both on the message (the transaction details), and your private key.

sign(message, private key) = signature

The result is a signature that can be verified using your public key and the message (the transaction details).

verify(message, signature, public key) = [true or false]

Verifying digital signatures. Source

All 3 together, private, public keys and signatures are mathematically correlated.

The verification process determines beyond doubt that the transaction could have only come from someone with the private key that corresponds to their public key. Thus all participants of the network receiving a transaction are sure that it could have only been approved by the private key owner!

The owner can also be sure that nobody can alter the signed parts of his transaction. That’s because a simple change would invalidate his transaction and thus dropped by all network participants.

 

The Importance of Randomness in Signatures

Focusing on weak random number generators used for transactions, hackers can still gain control of an address’s coins.

The mathematical basis used to create the signature of a message uses a random number (see formula below). If the same random number is used to sign two different messages (transactions), then someone, using the two resulted signatures can extract the private key number.

The mathematical formula generating a signature is calculated this way:

S = k-1 (Hash(m) + dA * R) mod p
where:
k is the random number
R is the x coordinate derived form the random number k
dA is the signing private key
m is the transaction data
p is the prime order of the elliptic curve

People actually had funds stolen because of the inadvertent reuse of this random number.

To avoid this vulnerability, wallets have to follow the industry-standard algorithm for deterministic initialization of this random number as defined in RFC 6979.

 

Now that you’ve got all the necessary details in mind let’s go through the journey of a transaction:

1. Your wallet is an application that serves as a user interface. It shows your balance and asks your permission when sending money. In order to do that it tracks your incoming and outgoing transactions, sums up your spendable money and creates and signs transactions on your behalf.
2. You start by specifying the amount and destination address of a bitcoin owner. Based on this info, your wallet constructs a transaction (as per the Bitcoin protocol) and signs it with your private key.
3. Your wallet starts broadcasting your new transaction (containing transaction details, the signature and the public key) to the Bitcoin Network through its immediate peers, who hand it to their peers, etc. In a few seconds, the entire network has verified and passed on your transaction to every other node on the Bitcoin network.

Transaction propagation

4. Every participant on the network that receives your transaction checks its validity. They check whether the signature is okay, if there are any errors, whether you are trying to perform a double-spend, etc. If your transaction fails any of the criteria, it is ignored by the participant entirely. Otherwise, the participant keeps your transaction in a temporary memory called mempool.

5. Transactions with a fee less than minrelaytxfee variable (0.00001 bitcoin) are treated as free and are only relayed if there is space in the mempool; otherwise, they are dropped. If the fee attached is smaller than the average competing transactions’ fee (normally calculated by your wallet but you can also specify the fees) miners/mining pools will give your transaction lower priority when creating a block.

6. Eventually, your transaction reaches the mining pools and the wallet of the recipient of your transaction. The latter will see the new transaction in their wallets and store a copy of it indefinitely, but it will appear as zero confirmations. Mining pools construct a candidate block by aggregating transactions off the mempool. Depending on your fee, they will eventually include yours in a future block.

7. The pool splits the work of searching for the nonce that satisfies the block’s difficulty level to its pool miners. The miners don’t know anything about your transaction. Their job is to crunch numbers, not to check for block validity, as that’s a task for the pool.

8. Eventually, your transaction is included in a block that gets solved. It gets broadcasted through the Bitcoin network and everyone stores it to their local blockchain(s) (if there are transaction conflicts they will fork their existing blockchain and keep both chains of blocks). Now your transaction has one confirmation (one valid block accepted by the network).

9. The block creation process continues, and as more and more blocks build on top of the block where your transaction is included, it gains more confirmations. When it reaches 6 or more confirmations, it is considered to be fully confirmed. The more confirmations elapse, the harder it becomes to invalidate a transaction with an attack.

 

Transactions are much more complicated structures. They look very different behind the scenes. They have inputs and outputs which are being accompanied by scripts that lock and unlock the values of bitcoin.

But that’s for another article; a deeper dive into transactions. :)