Distributed Ledgers: what is so interesting about them?

27 September 2018

Some may have noticed I have gone part-time at University College London, and plan to spend the next two years engineering and launching distributed systems as part of the Chainspace project, Vega Protocol and Kryptik. A number of people have asked me “Why would an expert on Privacy Enhancing Technologies develop such an interest in distributed ledgers and blockchains?”. In this post, I will lay down why I think this space is interesting technologically, and also how it interacts and influences the engineering, adoption, and deployment of Privacy Technologies; and  how a number of  societal issues in Privacy may in fact be deeply linked with developments in Blockchains in unexpected ways.

Why are Privacy Enhancing Technologies so interesting in the first place?

Privacy is a multi-faceted field, and I have spent about 20 years working on, evaluating, and advising on systems that broadly protect privacy better than current engineering practices. Those who follow my work may notice a pattern: I research what I call in class “hard privacy” systems, namely those that assume there is no central party that may be trusted to manage honestly user data. Instead, I have used cryptography, and distributed trust to ensure that users can do useful work in a peer-to-peer manner — without disclosing any data to third parties — or, when there is a need for infrastructure, it is distributed and data is encrypted in such a way that multiple entities need to conspire to violate user privacy properties.

“Soft privacy” systems, such as those that help organizations manage privacy of data they hold better, guide cookie and retention policies, prevent discriminatory uses, and implement internal controls are very important and useful — but they are just not my main thing. Why is that? I am fascinated about how privacy interacts with power, be it corporate or government power, and see privacy as a means to ensure both liberty and the possibility of political dissent as well as economic equity through undermining what is now broadly called surveillance capitalism. Collection and access of vast personal datasets already creates such a huge imbalance of power and potential for abuse, that I would rather prevent it through “hard privacy” technologies, rather than manage it through alternatives.

It should already be obvious that the engineering tools we have to use to build cryptographic and distributed systems for privacy, are related to those used to build distributed ledgers: they involve top-notch networking and distributed systems security, including prevention of byzantine faults; the most cutting edge of zero-knowledge and other modern crypto that was initially developed for privacy applications; and a good dose of security economics to ensure all participants in the system have incentives to do the right thing. As such the technical tools between  my type of Privacy Engineering and Distributed Ledgers are one and the same.

So it is obvious why, purely on the basis of technical affinity and curiosity, I may be attracted to distributed ledgers. But it turns out there are also deeper reasons.

So what are distributed ledgers good for?

This is the question that many ask today, as the hype around this family of systems is somewhat tempered by the falling prices of tokens. And the reason the answer is harder than one might expect for a technology that has been in the public eye for over a year, is that many have promised that blockchains are a panacea for all social and technical evils that no technology can possibly be.

So let’s start with what current distributed ledgers are not particularly good for — research challenges that are best summarized in an article by Sarah Meiklejohn. Blockchains do not scale particularly well, and in particular open Nakamoto consensus based on PoW or even PoS does not seem to scale. A number of proposals are on the table, but right now they are all experimental. Blockchains are not particularly usable, either in terms of software but also due to their semantics: probabilistic finality, and a resulting high latency to get assurance. Blockchains also struggle to provide data privacy guarantees for transactional data in the chain — zcash is a gold standard in that space, and recent research shows most transactions are not very private despite best-of-breed techniques being employed. Particularly under the light of the last issue, what could possibly make blockchains interesting to a Privacy Engineer, besides technical curiosity?

There are two interesting aspects of blockchains in that respect: (1) they decentralize transaction systems; and (2) they provide an alternative monetization strategy for providing on-line services. Those, it turns out, could — and the path of technology and its impact on social life is always contingent — be extremely influential when it comes to privacy. I will explain those in turn.

Decentralized Transactions are hard.

As many in the redecentralize movement have pointed out the web and other internet systems, such as email, were initially decentralized. Then something happened in the years 2000-2010, and somehow large quasi-monopolistic service providers appeared to dominate most interactions (lets not be coy we are talking about Facebook, Google, Amazon, and smaller friends). What happened is no conspiracy: we have very good tools to decentralize the distribution of static, read-only, content. In 2000 research on peer-to-peer systems was all the rage, and led to bittorrent on the edgy side, to the DHTs that underpin today interesting projects such as IPFS. Cloudflare could be distributed with the right technical model using those techniques, and we could serve most static content in a decentralized manner.

What we do not know how to decentralize — in general — are “transaction systems”. Those are system, in which users from different security domains (not just one natural owner) may “write” to objects, and mutate their state. The core business of Amazon, Uber, AirBnB as well as more traditional industries such as banking, travel, HR, etc are based on keeping such records and mutating them safely. Traditional databases from oracle to mysql have been the traditional workhorse behind such systems. Cloud platforms allow databases to scale, using complex techniques such as crash-fail resilient consensus and replication — but they require all the infrastructure to be under a single authority.

This technical reality, namely that it is hard to build scalable decentralized transaction systems is one of the reasons why internet monopolies came to dominate most of our interactions.  However, this cannot explain everything (nothing can): techniques for achieving decentralized byzantine fault tolerance have been known since the 1980s-1990s, with PBFT and all that. So how comes they did not catch up? And when I say they did not catch up, I mean pretty much not at all: a couple of years ago one would struggle to find a single PBFT usable library in most languages. I would argue this is related to economics.

Lack of Economic model.

When the googles and facebooks of this earth established themselves it was not clear how they would make any money. Therefore they innovated, and established the ad-based model by which the user is not the customer, but rather their attention is a product to be targeted and sold for adverts. This led to the establishment of surveillance capitalism that now seems to run deeper, and also interacts in questionable ways with the advancements in machine learning — another fascinating field of computer science.

So for the years 2000-2010 the main monetization vision of online services launched was the collection of personal information, and the resulting generation of ad revenue. However, it turns out that personal information is more akin to tar sands than oil: an organization needs a lot of it to refine it into something useful — and that also leads to a monopolistic situation in which platforms that already have captured the ad revenue and a lot of personal data are difficult to dislodged. I think this has now sank in the psyche of engineers and entrepreneurs and I hardly hear any new businesses aiming to dislodge Google in terms of ad revenue — for which I am thankful. That market is captured and closed.

For a while an alternative model, based on selling mobile apps first, and then selling in-app purchases gave a glimpse of hope that on line services may be able to monetize, but that also did not last. The most monetized app were akin to “digital crack”, seeking to develop users with strange addictions; eventually others such as Instagram and Whatsapp got bough by large incumbents. Today most independent app developers make little money are are subject to the whims of every app store that may delist them at will — also controlled by large incumbents that take up to a 30% cut on revenues.

How could one have built decentralized alternatives to those? What is WhatsApp or Instagram were to operate their systems on the basis of decentralization — they would each require not one, by over four authorities to operate. What would be the incentives of such authorities to do a good job? And how could they cover their costs? This was a problem. The Tor network, for example, operates on the basis of volunteers running relays, research, foundation and state department funding — a model that cannot scale to run significant infrastructures.

Blockchains provide both technical and economic answers to enable decentralization.

This is where blockchains start to become interesting. Distributed ledgers provide platforms to technically make building decentralized apps minimally humane. Now, writing larger correct decentralized apps in solidity for Ethereum, for example, cannot be described as the most pleasant development experience. But take my word for it, it is much better than having to start from scratch writing your own byzantine consensus protocols, and ensuring they are correct. Thus they offer a technical alternative for writing transaction systems that are decentralized.

Secondly, blockchain systems integrate a system of incentives and monetization. Nodes operating the infrastructure are remunerated in many ways, from mining rewards to fees, using a micropayment system that is integrated into the platform. This solves the question of “who are the decentralized authorities and why would they work for me?” that blocked the deployment of such decentralized systems before.

At the same time, for all its ills, the ICO model — by which projects would issue their own tokens for use in a system — also provided incentives for founders and developers to initiate a project, fund development and often maintenance. Thus services on blockchains do not have to rely on ad revenue, but can instead rely on fees to survive and be sustainable, at least in theory. The open access model of those platforms also ensure that both infrastructure nodes and developers can invest in building apps without fear they may be arbitrarily excluded.

Those two features of blockchain platforms have profound implication for privacy, even though the platforms themselves today do not protect data privacy very well. First, they undermine the monopolistic position of large service providers — that by virtue of accumulating masses of user data, and using it as part of an advertising based economic model, fundamentally cannot be privacy friendly. Undermining those large silos of data both frees people from the whims of those platforms, the wide use of data to manipulate them, but also the secondary threats of governments (domestic and foreign) then dipping into those databases for their own purposes. Allowing users to pay for services they access also ensures that those services can survive and be sustainable, in ways other than selling out their users’ data.

So in brief, blockchains align incentives correctly: control over the service is decentralized, and usually subject to code (smart contracts) to ensure users are not subject to the arbitrary and opaque decision making of large online service monopolies; and secondly that payments are made to those that maintain infrastructure and services, to ensure they do not need to be tempted to pry as a business model. If this model is a success — subject to a number of contingencies — it may provide a good foundation for better, more open and humane, transaction systems that could actually redecentralize the internet.

Challenges ahead.

While the incentives are aligned this does not mean that current blockchains actually achieve all those great goals, and in particular that they provide strong privacy guarantees. For this reason I have spent some of my research time in the past looking at how we can use efficient zero-knowlege to protect privacy in transaction systems, as well as how to scale up cryptographic monetary systems and smart contract platforms with Chainspace. Scaling up those platforms to make them truly competitive with large on-line service providers and other ‘sharing economy’ silos is indeed the subject of my new start-up chainspace.io.

With their fall the adoption of Privacy Enhancing Technologies can finally be unblocked. And already we see some of the most advanced cryptographic techniques, including zero-knowledge and selective disclosure credentials, being fielded in the context of blockchains, where they have seen little traction elsewhere in the past 20 years. While large internet services make most of their money from mining user data for ads or optimization those technologies stand no chance to see the light of day at a large scale.

I would welcome anyone working in the fields of threshold crypto-systems, multi-party computation, and censorship circumvention — techniques that intrinsically require multiple authorities to work together, to consider how their distributed infrastructures could be both engineered and incentivized using and adapting ideas from the distributed ledger research community.

Are you off for good?

The final question people ask: am I going away from academia and UCL for good? Rest assured I am coming back. My roles in all ventures involve research; I am committed to my PhD students, joint projects and colleagues at UCL; and I feel deeply passionate about teaching new generation both security engineering and privacy technologies. UCL is my natural home in the UK, being a truly open university to the world, traditionally progressive in its outlook, and in the heart of London. My adventures in industry, however successful, will only make me a stronger scholar.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

<span>%d</span> bloggers like this: