In May 2015, I wrote a blog post about the potential of the blockchain. I was optimistic about its potential.1 Yet, recent Bitcoin developments suggest that I should re-visit my opinion about Bitcoin and the blockchain in light of new information.

The new information included, firstly, the very public resignation of Mike Hearn, a former Bitcoin core developer, from the project, complete with a post on Medium titled, “The resolution of the Bitcoin experiment”. It also included a discussion with a learned friend about the physical cost of securing consensus and preventing tampering on the Bitcoin blockchain.

A Blockchain Refresher

First, I should note, for the purpose of precision, that “blockchain” is not synonymous with Bitcoin. We can think of the blockchain as an idea for a permissionless, tamper-proof distributed database, and the Bitcoin blockchain as a specific implementation of that idea.2 There are cryptocurrencies that run on other blockchains, for example Litecoin, Dogecoin, Bytecoin, and Primecoin. There are also payment protocols that use blockchain technology, such as Ripple and Stellar. Finally, there are distributed application platforms such as Ethereum that rely on blockchain technology to secure its distributed applications platform.

Second, blockchains do not all rely on the same methods to secure consensus and prevent tampering: the Bitcoin blockchain relies on a method called proof of work, which I have described previously, and which is described more fulsomely by Satoshi Nakamoto in his 2009 Bitcoin whitepaper. Another method that has become the subject of some discussion is a proof of stake method, which in essence is based on the premise that validating a transaction requires the validator to prove that it owns a certain amount of economic value, and thus any party that wishes to tamper with the blockchain would have to amass a certain percentage of the total economic value that utilizes that blockchain.3 There are also other methods of securing consensus and preventing tampering with transactions, such as the federated Byzantine agreement method, which is used by the Stellar financial platform.4

Third, notwithstanding what I have said about the existence of other blockchains, it nevertheless remains true that the Bitcoin blockchain has the largest (by an order of magnitude) amount of computing power operating to secure it. The reward for processing transactions and securing the Bitcoin blockchain—by generating new blocks of valid transactions—is also the most economically valuable (by an order of magnitude).5 As such, what happens to the Bitcoin blockchain matters, at least in the short term.6

So, without further ado, I shall turn my attention now to some of the recent developments in the Bitcoin blockchain that trouble me.

The problems with the Bitcoin blockchain

There are three main problems with the Bitcoin blockchain, as I see it:

  1. Computational problem
  2. Energy problem
  3. Network capacity problem

Of these three problems, the first two are not specific to the Bitcoin blockchain; they apply to any blockchain that use proof of work to secure consensus and prevent tampering. The third, which was the subject of Mike Hearn’s Medium post, is specific to the way Bitcoin implements certain technical elements. I will first address the linked problems of computation and energy use, since they are relevant to a general class of blockchains. The network capacity problem I will reserve for last, since that is specific to Bitcoin.7

Computational problem

One of the problems with using proof of work to secure a blockchain is that it requires someone, whom I will henceforth call “miners” in keeping with the common parlance, to solve a computationally difficult puzzle using a defined algorithm in order to add new transactions to the blockchain. The Bitcoin blockchain uses an algorithm called SHA-256. Other blockchains using proof of work systems have chosen other algorithms, for example scrypt, SHA-3 BLAKE, or Quark.

The security of a proof of work system is based on the computational power necessary to generate a false block of transactions. To keep things practical and grounded, I will use the Bitcoin blockchain and network as an example of how a proof of work system works, but all proof of work systems generally function in this way.

  1. A transaction between two parties is broadcast to the Bitcoin network.
  2. The newly broadcasted transaction is received by a node on the network and goes through a first check to confirm that the spender has enough bitcoins to fulfill the transaction.
  3. If the transaction passes this first check, it is then put in a queue of pending transactions, otherwise it is rejected as an invalid transaction.
  4. A miner will then package a number of these pending transactions from the queue into a block, and begin “mining”, i.e. trying to find a solution that matches the defined criteria for a valid block, thus expending computational power. (Note that each miner in the Bitcoin network is working concurrently on its own block of pending transactions)8
  5. Once a miner has found a solution that matches the defined criteria for a valid block, it broadcasts its solution and the block from which it was derived to the entire network, and claims the reward of 25BTC for successfully mining a block.
  6. The other miners and nodes in the network verify that the block is valid, and if it is, they add the newly broadcasted valid block to their copy of the blockchain. This is the new starting point for subsequent blocks to be constructed, the “latest block”.
  7. The unsucessful miners stop trying to verify the block they were working on at the time the successful miner broadcasted its solution. They package a new block of transactions, and begin extending the blockchain from the new “latest block”.

It is important to note that the mining process is a “race to the finish line”, where there is a reward (25BTC as at Feb 20, 2016) only for the first miner to mine a valid block and broadcast that block to the Bitcoin network. The other miners have just expended computational power trying to solve the puzzle, but have received no compensation for that expenditure. Is this a waste of computational power?

Not exactly. The computational power in the network is a crucial part of the security of the a blockchain using proof of work. Let us think about it carefully. How would you conduct a fraud on a network that uses a blockchain based on a proof of work system? In the case of Bitcoin (which I use to provide a practical example), the main way in which this can happen is by “double spending” bitcoins, i.e. by spending the bitcoins in a transaction with one party, and then spending the same bitcoins in a transaction with another party.9 To do so, one would have to have one transaction included in a valid block, and then replace that transaction and the corresponding block with another block in which the transaction never happened.

So, to sum up, we can intuit that in a blockchain using proof of work:

  1. The probability of a miner successfully finding a valid block is a function of: a) the fraction of the total computational power held by that miner, and b) the difficulty of the puzzle.
  2. The security the blockchain is a function of: a) the total computational power held by honest participants in the network, and b) the number of independent participants in the network.

These two intuitions lead to the following conclusions: (assuming some level of rationality)

  1. A rational miner will seek to increase its fraction of the total computational power in order to increase its probability of successfully finding a valid block and earning the attendant reward. It will keep doing this until it is no longer economical to add more computational power. The problem, of course, is that every miner will be doing this, and thus the total computational power in the network increases, and thus unless the rate of change in computational power of a miner is greater than the rate of change in computational power of the network, that miner will at best retain its starting fraction of the total computational power of the network, or even see its share decline. As such, one can visualize the miners as hamsters on running wheels, running frantically just to stay in one place.
  2. The ability of an attacker to have a reasonable probability of compromising the blockchain and overwriting transactions encoded in the blockchain is dependent on the attacker possessing more computational power than the majority of the active miners on the network. Thus, the total computational power active on the network is of great importance: a blockchain secured by too little computational power is not tamper-proof.11
  3. If the number of independent participants in the network falls, the risk of oligopolistic collusion increases.
  4. If the number of independent participants in the network falls, the risk of coercion by governmental or economic entities increases.

The Bitcoin blockhain is illustrative of these conclusions.

First, there is an arms race among Bitcoin miners to acquire more efficient and powerful mining hardware. We can see this in the historical record: the initial miners on the Bitcoin blockchain used CPUs, the kind that you can find in your typical laptop or desktop computer. These eventually gave way to miners using GPUs, which offered significant improvements in the “hash rate”12 when compared to miners using CPUs. Eventually, even GPUs were insufficient, and miners based on FPGAs entered the market, providing an order of magnitude more computational power. Finally, Bitcoin miners switched to ASICs to eke out the maximum computational power for a given amount of power consumed.13 This arms race has resulted in a roughly exponential increase in the total computing power on the bitcoin network, as show in the graph below.

The Bitcoin arms race increases the capital expenditure (on up-to-date mining hardware and cooling systems) for a profitable Bitcoin miner. This inevitably throws up barriers to entry, and has a tendency to result in a more oligopolistic market with a higher degree of centralization as miners that cannot keep up with the arms race drop out of the network. Indeed, as at February 20, 2016, the largest five miners controlled 81% of the total computing power in the network.14 Note that the majority of the largest five miners are based in China, which also increases the risk of government coercion, since it is easier to identify a small number of entities that are geographically concentrated. (Note that this would apply no matter where the majority of the computing power on the Bitcoin blockchain is located; one cannot imagine that the governments of Europe or the U.S. would be any less likely to impose control if they felt their sovereign interests were best served by doing so.)

Second, while all the miners on the network are calculating hashes at a furious rate, only one miner will successfully solve a block and claim the reward. All the other miners are simply providing security for the network, and are not compensated for that service. This has led to acusations that the Bitcoin network is “wasting” computing power (and electricity, as we will discuss below).

I will note (because someone will undoubtedly point out this fact) that while SHA-256 based proof of work systems are amenable to highly specialized mining hardware using ASICs, not all algorithms are so amenable. Some alternatives to Bitcoin have built blockchains that use other algorithms in an effort to reduce the likelihood that an arms race for computational power will centralize computing power in a small number of entities and raise substantial barriers to entry. (That being said, scrypt, which was meant to be less amenable to mining by way of ASICs, has largely failed at that, given that there are a number of ASIC-based scrypt miners for sale today.15)

All in, therefore, there are problems with any blockchain using a proof of work system. They tend to result in increasing barriers to entry, as the capital expenditure necessary to participate meaningfully in earning rewards for verifying and securing transactions increases. They also tend to result in centralization of power in the blockchain, which makes coercion by governments or economic actors easier. Finally, logically, centralization of power in the hands of a limited number of participants increases the likelihood that such participants can collude with one another to earn economic rents.

Energy problem

By definition, a blockchain based on proof of work requires work to be done in order to secure consensus and protect the integrity of the transaction ledger. This computational work consumes energy in the form of electricity. Indeed, one can argue that the consumption of energy is the fundamental way in which such a blockchain is secured: the deterrent to an attacker launching an attack on the blockchain is the cost of such an attack, in terms of electricity and hardware. The latter is an initial hurdle that must be surmounted: an attacker must first acquire the hardware to compete against the honest participants in the network, amounting to a minimum of 51% of the total computing power in the network. The former is a second hurdle: having acquired the hardware, the attacker must then pay the cost of operating the hardware to make its attack. By the time an attacker has acquired the hardware and has the wherewithal to operate it, it would be more cost effective for the attacker to be an honest participant in the network.

Now, there are some people who claim that this energy usage is “wasteful”. It may be so. But I would view this as a more nuanced question. The cost of securing a proof of work blockchain is the computational work that goes into verifying transactions and adding them to the blockchain. This cost must be compared against the total cost of setting up an alternative security system. For example, what is the total cost of securing a transaction on a payment platform like VISA? One would need to compute VISA’s operating costs, and then add to that the cost of the regulatory infrastructure that allows users to have confidence that VISA is a trusted intermediary that can be relied upon to process payments. Until one does such a comparison, it is not possible to say whether the cost of running the blockchain is higher or lower than a platform like VISA.

Yet, even if the energy usage is not “wasteful”, it does create potential problems for blockchains based on proof of work. It is energy intensive. For the miners, this translates directly into operating costs that must be paid regardless of the revenue they earn from the provision of their services. The miners are not the only ones affected by this energy usage: depending on the age and quality of the electrical grid, this could result in problems for the utility companies that maintain the grid, as well as other users of the grid.

To illustrate this, we’ll look at the Bitcoin blockchain again. Christopher Malmo has written that Bitcoin is unsustainable due to its energy costs, which he claimed in June 2015 could be in the range of 250MW to 500MW. Now, this article is not without its critics, and the figures given by Malmo may be inflated. (I express no opinion one way or the other, as I lack the data to perform a meaningful estimate and have no desire to drill into the energy efficiency figures for all the mining hardware available on the market and estimate their market share in order to come up with my own—likely also inaccurate—estimate.) It is, however, not controversial that the Bitcoin network consumes a substantial amount of electricity. There is a reason why many large miners are located near abundant and relatively cheap sources of electricity, e.g. in western Sichuan province, China (hydroelectric dams), Iceland (abundant geothermal power plants), and the Republic of Georgia (hydroelectric dams).

Moreover, to the extent that miners are not using zero-carbon emission energy sources, the Bitcoin miners are contributing to the increase in carbon emissions in the world. Indeed, one should note that excepting those miners fortunate enough to be located near hydroelectric or geothermal power plants, the rest are likely to be drawing their energy from coal-fired or gas-fired power plants, which are among the cheapest sources of reliable baseload power known to exist. (Some miners, of course, will be getting their electricity from nuclear power plants, which are a source of zero-carbon baseload power.) For anyone concerned with the environment, this is a matter that ought to be of some concern.17

Aside: Blockchains and the Internet of Things

As a brief aside, some very intelligent people—including scientists at IBM have suggested that the various forms of blockchain technology may be the key to powering the Internet of Things. This may be the case, but where is the computational power and its associated energy consumption going to come from? That is a question that I have not yet seen fully addressed, and it is a very relevant question. We are not, after all, talking about machines that have a lot of computational power or (in some cases) energy to spare. It could be that, firstly, we are relying on the fact that there will eventually be millions or even billions of tiny connected machines, each contributing a tiny bit of processing power to securing their blockchain. Secondly, it may be that the computation necessary to secure an Internet of Things blockchain might be external to the “things” connected to it, i.e. computation might still occur on dedicated miners (which would need to be compensated). Thirdly, it may be that some alternative consensus protocol that is not as energy intensive as a proof of work protocol may be used by an Internet of Things blockchain: proof of stake, federated Byzantine agreement, or something else that has yet to be invented. What is clear, though, is that the question of there being “no free lunch” when it comes to proof of work based blockchains will likely be an issue that needs to be tackled.

So far, I would say the most likely outcome is that the Internet of Things would have to rely on an externally secured blockchain, for example, the Bitcoin blockchain, because securing and verifying the transactions on the blockchain would otherwise consume computing power and energy that these devices may not have. For example, while a washing machine or refrigerator connected to the grid might be able to draw power from the grid to “mine” blocks, I doubt that most users will want to pay for that cost, or for the cost of embedded ASICs within their appliances to do the computations. Moreover, that does not help any number of power constrained devices, such as wearables and distributed sensors.

Of course, it may well be that OEMs could agree on some alternative consensus and security protocol to power a dedicated Internet of Things blockchain, one that does not require the kind of computational power necessary for a proof of work based blockchain, but that seems slightly optimistic to me.

Network capacity problem

And now we come to a problem specific to the Bitcoin blockchain, and one that has garnered a fair bit of attention since mid-2015. If we are interested in blockchains per se, and not any specific example of a blockchain, why is this important? It is important because Bitcoin is the largest and most public blockchain presently operating. It is being used by some developers and startups as a platform or layer for their own products, such as a diamond certification ledger run by Everledger or a smart contracts platform developed by Counterparty. As such, what happens on the Bitcoin blockchain matters to users who are using it to transact, and to developers using it as the basis for their own products.

Moreover, should the most public blockchain and cryptocurrency fail, it will have inevitable ripple effects on all other blockchain experiments out there. People will pull back from this technology, at least for a while after a public failure.

Mike Hearn does a fairly good job of explaining the network capacity problem, but for the sake of completeness I will summarize the key takeaways here.

The Background: There is a 1MB limit on the size of a block of transactions on the Bitcoin blockchain; this limit is encoded into the core Bitcoin software. This means that only a maximum of 1MB of transactions (each transaction being on average 600 bytes circa October 201518) can be included in any given block; the Bitcoin network will reject as invalid any block of transactions that is larger that 1MB. This limitation was inserted into the Bitcoin software during its infancy, when it might have made sense.

The Problem: As of January 2016, the average block size is of new blocks is bumping up against that 1MB hard limit. This means that the Bitcoin network has reached close to the maximum number of transactions it can process at any given time (because only as many transactions as can fit into that 1MB block can be added to the blockchain at any point in time). If there are transactions that cannot be included in the current block, they remain in a queue of pending transactions, and may be included in the next block. Let us call the transactions presently in the queue “Batch 1”. The problem is that there are new transactions entering the pending transaction queue while Batch 1 is waiting for its turn to be processed. So the pending transaction queue is growing longer behind Batch 1. This leads to a growing backlog of pending transactions.19

The consequences: There are two key consequences from this problem. First, from the perspective of the end user, it is no longer possible to determine exactly how long it will take for any given transaction to execute and be added to the blockchain. Per Mike Hearn and others, waiting periods for transactions to be added to the blockchain can now range from 10 minutes to upwards of 10 hours, with the wait time for any given transaction selected at random. This, for any business that transacts in bitcoins, is unacceptable. You cannot run a business with that kind of variability in the time it takes for your transactions to clear. Second, in order to expedite a transaction, an end user can pay a transaction fee to miners to incentivize them to expedite the processing of his or her transaction. Transaction fees,20 though, can be difficult to implement when one of the early selling points that the media touted about Bitcoin was low or zero fees. Moreover, it raises the question of the amount of transaction fees end users will stomach before deciding that Bitcoin is not competitive with other payment mechanisms.

A solution: So, if there is a capacity problem created by the limit on the size of a block on the Bitcoin blockchain, why not just increase the size of a block? The answer is, some people tried to do exactly that in the second half of 2015. It didn’t work, for various reasons. This drama is far too long and convoluted to lay out here. It suffices to say that Bitcoin XT was a hard fork of the core Bitcoin software that increased the maximum block size to 2MB, and unfortunately, it did not gain sufficient traction to become the default implementation of Bitcoin.

Where we are now: At present, there is an ongoing debate about what to do. There are at least two competing approaches: Bitcoin Core which has chosen to retain the 1MB block size limit, and the Bitcoin Classic which has chosen to increase the block size to 2MB. This will likely turn into what Coinbase CEO Brian Armstrong describes as an “election” in which miners will decide if they want to stay with Bitcoin Core or Bitcoin Classic.

Aside: The Great Firewall of China

One interesting observation that has emerged from reading the literature is that the Great Firewall of China does have a substantial impact on the Bitcoin blockchain. I suppose it stands to reason, given the fact that so many of the largest miners are based in China. One of the reasons why Bitcoin has been unable to agree upon a larger block size has been the reluctance of the Chinese miners to agree to such an increase. Due to bandwidth limitations (exacerbated by the Great Firewall of China), information about newly discovered blocks is often bottlenecked at the Great Firewall of China. (This bottleneck cuts both ways: if a block is discovered outside of China, the Great Firewall slows down the speed at which that information propagates into China and to the Chinese miners, while if a block is discovered in China, the Great Firewall slows down the speed at which that information propagates to miners outside of China, with real economic consequences in terms of wasted computations and energy.21) As such, the Chinese miners are concerned that if the block size increases substantially, incoming blocks will take longer to traverse the Great Firewall, and put them at a disadvantage in mining new blocks. (The flip side is also true, as miners outside the Great Firewall would also be disadvantaged if a block is found by Chinese miners inside the Great Firewall, so in my view the effect tends to cancel out in the long-run.)

Aside: A storage and bandwidth problem

One related issue which ties into the problem of centralization and network bandwidth is the growing size of the Bitcoin blockchain. As at February 15, 2016, the entire Bitcoin blockchain is 55GB. Looking at charts of its growth, it appears to be growing more or less exponentially. Downloading all 55GB of the blockchain over peer-to-peer file transfers takes upwards of 48 hours, and while storing 55GB is trivial in an era where desktop hard drives reach 6TB, remember that the blockchain is growing at an accelerating rate. That 55GB could soon become 110GB, then 220GB, et infinitum.

Moreover, synchronizing the blockchain across the network can result in monthly uploads of 200GB or more (likely more, since that figure dates from 2014). Imagine explaining that kind of bandwidth usage to the average residential internet service provider, for example, ComCast.22

This brings us to another problem with Bitcoin and its blockchain. The resources demanded to store and update a full local copy of the blockchain is increasing exponentially. The number of users that can afford to store and update a full local copy of the blockchain are therefore decreasing, with implications for the security of Bitcoin.23 Many average users are likely relying on “intermediaries” such as Coinbase, Circle, and Strongcoin that provide bitcoin wallets and transaction processing services. These intermediaries are the entities that handle transaction verification and relay an average user’s transactions to the rest of the network. This, at least from where I’m standing, means that an average user must, ultimately, trust his or her intermediary to be an honest participant because that intermediary is hosting his Bitcoin wallet, and is verifying his transactions and relaying the results back to him or her. So much for a trustless, decentralized payment protocol where individual participants deal directly with one another and do not need to trust each other in order to transact.

Some concluding thoughts

This is a lot to take in. I suspect that most non-Bitcoin enthusiasts will have tuned out by now. Let me first sum up the three problems:

  • Blockchains based on proof of work depend, by definition, on computational power to attain consensus and secure transactions.
  • Blockchains based on proof of work, as a corollary to the use of computational power, consume substantial amounts of energy.
  • The Bitcoin blockchain is suffering from a network capacity problem that requires significant changes to address, changes that have been controversial and have (as of the date of this post) not been resolved.

What does this all mean for people looking at Bitcoin, the blockchain, and other similar technologies?

Bitcoin

First, recognize that the failure of Bitcoin is a real risk that must be factored into your decision whether to hold bitcoins, invest in Bitcoin-related startups, or develop technology that depends for its operation on the continued operation of the Bitcoin blockchain. Think of Bitcoin for what it is: an experiment.

Second, recognize that the existence of very substantial economic resources committed to mining bitcoins and thereby maintaining the blockchain imposes some measure of rationality on the participants. At some point these individuals will, in their own self-interest, have to act to find a solution to the Bitcoin network capacity problem.

Third, recognize that there are external factors that distort the economic and technical incentives of participants in the Bitcoin network, most notably the existence of the Great Firewall of China. This will inevitably distort their incentives when it comes to approving technological solutions that might be adversely affected by the bandwidth bottlenecks at the Great Firewall of China.

Fourth, recognize that culture will also play a part in how participants perceive the arguments for and against increasing the Bitcoin block size above 1MB. Chinese miners may not perceive the arguments in the way that participants brought up in an Anglo-American cultural millieu might. See, for example, Laura Shin’s article in Forbes on this topic.

Blockchain

First, remember to separate blockchain from Bitcoin. What happens on the Bitcoin blockchain is important because it is the largest and most public example of the technology. It is not the only example of the technology, and there are experiments out there that are taking different approaches to solving the same problems that the Bitcoin blockchain solves.

Second, recognize that any proof of work based blockchain requires expenditures of computational power and energy that:

  1. Creates an incentive for an arms race to increase one’s share of the total computing power on the network while minimizing the energy consumption per unit of computing power
  2. Creates a tendency towards centralization as increasing capital expenditure and operating costs weed out the less efficient participants in the network
  3. Creates a tendency towards fragility in the network, as the survivors become too big too fail, or become susceptible to coercion or collusion.

Third, be curious about alternatives to the traditional proof of work based blockchain. Some of them may work better in specific use cases, and it is always better, at the infancy of a technology, to be open to the possibility that the first few implementations of the technology may be flawed, and may be supplanted by better implementations at some future date.

Fourth, recognize that very intelligent individuals have chosen to invest their time and money in this nascent space. Many of the startups and ideas they come up with will fail. Some will not. Selecting which ones, well… that is a question for a different kind of entry.