Towards consensus against consensus.

Consensus in a distributed system is an incredible tool. It solves coordination problems by enforcing a state across the system as a whole. In Bitcoin, this solves the double-spend problem by creating a shared ledger. In Ethereum consensus algorithms create a completely transparent virtual machine, and ensures the integrity of any computation made by that machine. Global agreement without trust allows users to interact with confidence on these shared systems.

To create consensus, Proof of Work miners convert electricity to timestamped blocks and broadcast them to the network. The expected value of adding x additional hashes per second is similar to x/(1+x). As a result, miners race to the bottom, reinvesting mining rewards in more hashpower until profit trends to zero. This pursuit of hashrate by independent parties causes resource investment in the consensus system to trend upwards over time. Consensus creates coordination from dis-coordination.

This is incredibly valuable to systems that many people rely on. Creating a permanent, immutable ledger shared across thousands of independent nodes lets us pay each other directly, and execute code on a distributed virtual machine with near-perfect reliability. It lets machines to directly interact with money, and soon smart contracts and distributed marketplaces will allow machines to directly buy and sell storage, computing, and physical goods without human involvement. None of this would be possible without consensus. Consensus enables cooperation without trust.

Inefficient, expensive, and slow

Unfortunately, known consensus algorithms have a number of drawbacks compared to traditional systems.

To participate in the consensus, each node must perform a certain amount of work to validate the block. First they must retrieve the chain from the network. Then each node must validate the history of the chain. After this initial syncing, nodes must validate all new blocks as they come in. Ethereum nodes validate signatures, hashes, and every change of state of the virtual machine. This is a non-trivial amount of calculation repeated across thousands of nodes, again using electricity, storage space, and network resources. As such, consensus always uses more resources than a consensus-less system to achieve the same effect.

Both creating consensus and participating in consensus are expensive. And each node must expend resources to maintain the shared state. As such, consensus has large costs that tend to increase over time. At best the resources spent scale proportional to the number of nodes in the network. Proof of work is a mechanism for finding the market price of security, and like any price-discovery mechanism it's inefficient by definition. Consensus is more expensive than a consensus-less system.

In addition, consensus is slow. Distributed consensus algorithms have to propagate all information to all nodes, and have no guarantees that there is latency or bandwidth to do so quickly. Then all information must be independently validated by all nodes. This limits the overall speed at which information can propagate through the system. In Bitcoin it's 1 MB per 10 minutes: unfathomably slow compared to consensus-less networks. Consensus is slower than a consensus-less system.

The technical requirements of public consensus systems create a poor user experience. Public consensus systems require management of private keys. Users must be responsible for their own identities and security. Nobody has ever made a key-management application with a positive user experience. Personal responsibility is terrible UX. Keybase and Lastpass come close, but only for people who are already technical.

Bitcoin is a prime example of this. It consumes massive amounts of electricity, and stores the same 90GB of data redundantly on over 5000 nodes, while providing less than 0.01% of the capacity of other payment services. Bitcoin payments are far less convenient than credit card payments, Venmo, PayPal, and bank transfers. Other payment services offer valuable features like fraud protection, rewards programs, and interest. Why do we even use bitcoin?

Consensus is useful

Coordination without trust is extremely valuable. The advantages eked out by consensus over traditional models in certain systems could be immense. Money, supply chains, power grids, governance, social services. Consensus systems could expose or prevent misappropriation of funds, automate distribution of physical goods, and enable machine-to-machine economies.

When we're honest with ourselves, we have to admit: the problem set consensus solves is very narrow. Consensus is only useful when the following conditions are met: 1) the service is valuable to many people, 2) trust can't be safely delegated, and 3) downtime is not permissible. Most systems don't need consensus. Even systems we rely on to live.

We don't need consensus to buy coffee; I can safely trust the barista with my $5 latte purchase. Consensus doesn't help us stream video, or text friends. We don't need consensus to play video games or card games. Most of the software you interact with on a daily basis has no use for consensus. But when consensus is useful, it's incredible. Bitcoin, Ethereum, and others will find these market, and rework important parts of the world. And that's incredibly exciting.

Consensus Against Consensus

Because consensus is extremely expensive, slow, and inefficient, and the problem set is narrow, we should be strongly biased against consensus. Every project that tries to use consensus where not strictly necessary will fail because it can't compete with a traditional model implementing the same system.

We're all irrationally excited about consensus. We're throwing it against the wall of tech problems, just to see where it sticks. Consensus-powered currencies for blogging, file storage, collaboration, protein folding. Hundreds of teams working on hundreds of blockchains, and most of them will end up being wasted effort. We can predict at least some of these failures, and prevent them.

Software developers and systems engineers should reach consensus against consensus. When we start a new project, we should ask ourselves "does this really need consensus? Isn't there a better way to do this?" And when we come across a new consensus-based project, we should demand evidence that it requires consensus.