Wednesday, September 26, 2018

The last mile problem in trust

Blockchains are supposed to solve the trust problem. But blockchains attack only the easy part of the trust problem, and avoid the hard part. The easy part is to store the transactions in a tamper-resistant database. The hard part is to attest to physical world actions and state.

The blockchain is a database technology and it does not attempt to attest to physical world actions/state. It solves the problem of tamper-proofing the state after it is added to the database. It doesn't attempt to validate/test/certify if the state is correct as it is added to the database. If humans create the state, there is inherently a trust problem: Were the lettuce bad before it was loaded to the trucks, or are the truck conditions to blame? Did the farmer or the trucker lie?

If sensors create the state, this is still a very hard problem, but not because the sensors may have been tampered with ---that is a relatively easy problem to solve in hardware. The problem is hard because of the corner-cases involved; how do you even start to pretend that the sensors have complete coverage (or good/fair sampling) and the detection/verdict is accurate? It is really a very complex and messy problem. As far as complete coverage of food supply-chains are concerned, you need DNA-sequencing and metagenomics.

This is a classic last mile problem. The last mile problems are always hardest to solve because of the massive fan-out both in terms of scale and in terms of corner cases to handle. The last mile problems haunted many domains, most notoriously the telecommunications and transportation domains.

Walmart, Lettuce, and Blockchains

A couple days ago there was a lot of hype about Walmart starting to use blockchain in its supply chain, to pinpoint where the lettuce come from in an E.Coli contamination event.

Ok, let's get to the bottom of this. "Walmart, Lettuce, Blockchain." It felt very weird to type this in Google search, but I did it anyways... for science.

See, I knew there was a lot of hype: "The giant retailer will begin requiring lettuce and spinach suppliers to contribute to a blockchain database that can rapidly pinpoint contamination."

De-hyped, this just says Walmart wants the farmers to record transactions in a database.  And actually the article makes sense if you replace blockchain with database:  "Walmart says it now has a better system for pinpointing which batches of leafy green vegetables might be contaminated. After a two-year pilot project, the retailer announced on Monday that it would be using a blockchain, the type of database technology behind Bitcoin, database to keep track of every bag of spinach and head of lettuce."

I blame IBM's over-excitement in blockchains for the hype in the article. Supply-chains is a very complex topic, and this use of a database to record information doesn't come close to scratching the surface of it. There are many automation and logistics problems that remain to be solved. And the dreaded last mile problem of course.

MAD questions

1. What is the nature of trust?

What or who do you trust?

Trusting a deterministic machine with few inputs/environmental-parameters is reasonable. Especially if you verified and validated it, and tested it extensively.

But what would make you trust humans? Humans are complex nondeterministic beings, and the input and environment surrounding humans are also very complex.

Reid Hoffman defines trust as consistency through time. But this is assuming the conditions don't change. If conditions change, that is the inputs/environmental conditions change, the other side can change its actions.

The answer to the trust puzzle has got to do with "consequences", right?

It is easier to trust in a situation where you have little to lose, but the other side has a lot at stake. And ironically, this makes the other side have problems trusting you, since you have little at stake, and she is risking a lot. For mutual trust and better collaboration, all parties should have skin in the game.

So what is at stake? This can be reputation, if reputation is a currency valued by the individual and his environment. What is at stake can be jail time, if one breaks laws and get caught. This is assuming one doesn't enjoy jail. Under certain conditions, people commit crimes to get into jail to get fed and have reliable healthcare, and even not to feel lonely.

I think trust is not complicated, rather the calculation, alignment, and managing of consequences/incentives is complicated. And this again harkens back to the last mile problem in trust.

I believe the parties involved are going to push the limits of what they can get away with as long as the deterrents do not outweigh the incentives.

I don't know if there is a technology solution here.

At a recent A16Z podcast, one speaker was rightfully complaining that we have a lot of trust issues and fight among complementary business rather than substitute/alternative business. For example even though iphone apps and iphone platform are complementary businesses, there is a lot of fight there. Or consider the Yelp versus Google fight. Or the fights Facebook, the platform, picks with the applications it enables. The speaker was implying that with the right incentivization and cuts from cryptocurrencies like ethereum gas, the parties will actually synergize and grow together rather than fight.

This sounds nice and simple, but I don't think I buy this. The fights are due to the greedy nature of humans and companies. To repeat what I said said above, I believe the parties involved are going to push the limits of what they can get away with as long as the deterrents do not outweigh the incentives. Even if cryptocurrencies and Ethereum gas is used between platforms and applications enabled, next we will see fights over how much of the payment is fair etc. I don't know if technology can fix that. Maybe this is supposed to be a dynamic equilibrium with constant push-backs and small-battles erupting from the parties involved.

2. What is the verdict?

I don't hate/despise blockchains, as I have seen some colleagues do. That is a radical and unreasonable position. There are many smart people working on this domain, they cannot be all and completely wrong.

I am still ambivalent about blockchains. I believe there is still a big contribution potential coming from blockchains and smartconracts. But the hype news make things harder to see.

Saturday, September 8, 2018

Book review. Ignorance: How it drives science

I picked this up from my local library, because the title was interesting. I wrote about this earlier.
Once you get a B.S., you think "you know everything". Once you get an M.S., you realize "you know nothing". Once you get a Ph.D., you realize that "yes, you know nothing, but that is not a problem, because nobody knows anything!"
This turned out to be a nice read. The author, Stuart Firestein, has a very interesting background. He was working at a theater, and started a biology undergraduate at 30, and got his PhD at 40.

Here are some tidbits from the book.

Leibniz. page 38
The 17th-century German philosopher and mathematician Gottfried Leibniz, one of the inventors of calculus, had a lifelong project to construct a "basic alphabet of human thoughts" that would allow one to take combinations of simple thoughts and form any complex idea, just as a limited number of words can be combined endlessly to form any sentence -- including sentences never before heard or spoken. Thus, with a few primary simple thoughts and the rules of combination one could generate computationally (although in Leibniz's day it would have been mechanically) all the possible human thoughts. It was Leibniz's idea that this procedure would allow one to determine immediately if a thought were true or valuable or interesting in much the same way these judgments can be made about a sentence of an equation -- is it properly formed, does it make sense, is it interesting? He was famously quoted as saying that any dispute could be settled by calculating-- "Let us calculate!" he was apparently known to blurt out in the middle of a bar brawl. It was this obsession that led Leibniz to develop the branch of mathematics known today as combinatorics. This in turn sprang from the original insight that all truths can be deduced from a smaller number of primary or primitive statements, which could be made no simpler, and that mathematical operations (multiplication was the one Leibniz proposed but also prime factorization) could derive all subsequents thoughts. In many ways this was the beginning of modern logic' indeed, some consider his /On the Art of Combinations/ the major step leading from Aristotle to modern logic, although Leibniz himself never made such claims.

Godel. page 41
What Godel showed, using a strange new correspondence between mathematics and logic that he invented, was that if a system were the rules of that system. This means that something that could be shown to be true using the system could not in fact be proved to be so. Since proofs are the foundation of mathematics, it is quite curious when obviously true statements cannot be proved.

Godel. page 42
Was this the end of the messianic program to establish the primacy of mathematics and of logical thinking? As it turns out, quite the contrary. Godel's small, by comparison, but revolutionary output is so asttonishing because of the technical and philosophical research opportunities it has created. Previously unconsidered ideas about reccursiveness, paradox, algorithms, and even consciousness owe their foundations to Godel's ideas about imcompleteness. What at first seems like a negative --eternal incompleteness-- turns out to be fruitful beyond imagining. Perhaps much of computer science, an area one might think was most dependent on empirical statements of unimpeachable logic, could not have progressed without the seminal ideas of Godel. Indeed, unknowability and incompleteness are the best things that ever happened to science.

Hilbert. page 48
In fact, one of the most predictable things about predictions is how often they're wrong. Nonetheless, they are a measure, even if somewhat imprecise, of our ignorance. They are a catalog of what we think the important ignorance is, and perhaps also a judgment of what we think is the most solvable ignorance.

May ignorance lead your research. page 55
Ignorance is not just an excuse for poor planning. We must think about how ignorance works, and we have to be explicit about how to make it work to our advantage. While for many experienced scientists this is intuitive, it is not so obvious to the layperson, and it often seems not so apparent to young scientists starting out their career and worrying about grant support and tenure.

Grants. page 59
How do scientists ponder these big questions about ignorance? How do they get from these and other interesting and important issues to an actual scientific research program? Well, at the most pedestrian, but nonetheless critical level, there are grant proposals. Every scientist spends a significant percentage of his or her time writing grants. Many complain about this, but I actually think it's a good idea. These documents are, after all, a detailed statement of what the scientist hopes to know, but doesn't, as well as  a rudimentary plan for finding it out.

Models. page 70
This strategy of using smaller questions to ask larger ones, is, if not particular to science, one of its foundations. In scientific parlance this is called using a "model system". As Marvin Minsky, one of the fathers of artificial intelligence, points out, "In science one can learn the most by studying the least". Think how much more we know about viruses and how they work than about elephants and how they work. The brain, for example, is a very complicated piece of biological machinery. Figuring out how it works is understandably one of humankind's great quests. But, unlike a real machine, a man-made, designed machine, we have no schematic. We have to discover, uncover, the inner workings by dissection-- we have to take it apart. Not just physically but also functionally. That's a tall order since there are some 80 billion nerve cells that make up the human brain, and they make about 100 trillion connections with each other. ... So instead of a human brain, neuroscientists study rat and mouse brains, fly brains because they  can do some very fancy genetics on them, or even the nervous system of the nematode worm, which has exactly 302 neurons.

MAD questions

1. What do you feel ignorant about in your line of work?

Two-phase commit and beyond

In this post, we model and explore the two-phase commit protocol using TLA+. The two-phase commit protocol is practical and is used in man...