Metadata

This is a VDLB 2017 paper. Last week we reviewed the F1 paper from 2012. It seems like F1 was an experiment and sort of a preview towards adding serious SQL support in Spanner. The original Spanner paper was published in 2012 had little discussion/support for SQL. It was mostly a "transactional NoSQL core". In the intervening years, though, Spanner has evolved into a relational database system, and many of the SQL features in F1 got incorporated directly in Spanner. Spanner got a strongly-typed schema system and a SQL query processor, among other features. This paper describes Spanner's evolution to a full featured SQL system. It focuses mostly on the distributed query execution (in the presence of resharding of the underlying Spanner record space), query restarts upon transient failures, range extraction (which drives query routing and index seeks), and the improved blockwise-columnar storage format. I wish there was discussion on the evolution of data manipulation/modi...

This is a VLDB 2013 paper (appeared earlier at Sigmod'12 it seems) from Google about paying tech-debt. F1 replaces the sharded MySQL hacky implementation of AdWords with a principled well-engineered infrastructure that builds a distributed SQL layer on top of Spanner. My reaction to this paper probably up until 5 years ago would be "ugh, schemas, database stuff... No distributed algorithms, pass!". But I like to think that I improve with age. After having learned the importance of databases as real-world killer applications of distributed systems, I now look at papers like this as a potential Rosetta stone between the two fields. I am looking for papers that can shave off weeks and months from my journey to learn about distributed databases. I think this paper has been very useful for understanding several issues regarding distributed databases, as it gave information about many facets of deploying a large scale distributed SQL database. But, the paper fails to become a...

This paper (from Sigmod 2023) is a followup to the deterministic database work that Daniel Abadi has been doing for more than a decade. I like this type of continuous research effort rather than people jumping from one branch to another before exploring the approach in depth. The backstory for Detock starts with the Calvin paper from 2012 . Calvin used a single logically centralized infallible coordinator (which is in fact 3 physical nodes under the raincoat using Paxos for state machine replication) to durably lock-in on the order of oplogs to be executed. The coordinator also gets rid of nondeterminism sources like random or time by filling in those values. The oplogs then get sent to the workers that execute them and materialize the values. The execution is local, where the executors simply follow the logs they receive. This deterministic database architecture makes an interesting tradeoff. Rather than primaries executing and replicating the value of the results to secondaries, her...

Sponsored industry talks Amazon: Innovation in AWS Database Services — Marc Brooker Marc, who is a VP/Distinguished Engineer at AWS, talked about systems issues in AWS database services. (If you are not following Marc's blog feed , you should take a couple seconds and fix this problem now.) He talked about the enormous scale that the AWS databases deals with by mentioning that the DynamoDB processed 89 million req/sec during Prime Day. The problem is not only to scale up, but also down. There is a spectrum of scale, with billion times difference in scale. Systems support is need to enable scale up and down. Consider Aurora Serverless v2, which launched couple years ago. We had to solve problems like how to grow and shrink in place, how much cache to allocate, and how much temporary storage to use. (CPU provisioning was easier, since it is elastic.) The logic and reasoning of the 5 minute rule by Gray/Putzolu still remain relevant, although the constants changed since 1986. We still...

I mentioned this panel in my SIGMOD/PODS day 2 writeup.  The panel consisted of (from right to left) Gustavo Alonso (ETH), Swami Sivasubramanian (AWS), Anastasia Ailamaki (EPFL), Raghu Ramakrishnan (Microsoft), and Sam Madden (MIT). Sailesh Krishnamurthy from Google was unable to attend, so Anastasia who is on sabbatical at Google, also covered for him. The panel started with 5 minutes speeches from participants. The panel was lively and playful. The panelists tried to make their speeches controversial, and have zingers for each other.   Sam talked about the demise of DBMS monolith. He said, the Oracle of 90s is dead because it was incompatible with modern data ecosystem, and the modern DBMSs are shifting towards disaggregated designs. When Sam was a PhD student under Eric Brewer, Eric said we should build disaggregated databases, and people said that won't happen. But we are there now: Disaggregation has arrived! While Sam mentioned about the benefits of disaggregation, he al...