Metadata

It is time to reflect back on the year. Here are some selected papers I covered in 2022. Production systems Amazon Aurora: Design Considerations + On Avoiding Distributed Consensus for I/Os, Commits, and Membership Changes Aurora provides a a high-throughput relational database on AWS cloud by decoupling the storage to a multi-tenant scale-out storage service. Each database instance acts as a SQL endpoint and supports query processing and transactions. Many database functions, such as redo logging, materialization of data blocks, garbage collection, and backup/restore, are offloaded to the storage nodes. Aurora performs replication among the storage nodes by pushing the redo log; this reduces networking traffic and enables fault-tolerant storage that heals without database involvement. Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service (USENIX ATC 2022) DynamoDB has massive scale; it powers Alexa, Amazon.com sites, and all Amazon fulfillment c

This paper, which appeared in OSDI 2018 , considers the question of how to design a read-optimized RDBMS database with built-in support for caching and fast lookup answers to complex queries. The resulting system, Noria is implemented in Rust, and is available as opensource at https://github.com/mit-pdos/noria . There are already two good summaries written of the paper (from Adrian Colyer and Micah Lerner ), so my job today will be simpler. Artwork created by Stable Diffusion Motivation and overview Databases (mostly relational databases) are overwhelmingly used with read-heavy workloads due to all those SQL querying they serve, but they fail to take advantage of this and improve their designs to optimize for read-heavy workloads. Some crutch solutions was added to address this: memcached, read-only replicas etc., but these have shortcomings. Many deployments employ an in-memory key-value cache (like Redis, memcached, or TAO) to speed up common-case read queries. Such a cache avoids r

Created by Stable Diffusion The 70s were a time of excess and rebellion, a carefree era where people embraced their wildest impulses and let their hair down. But as the years went by, people realized that the true freedom lies not in the absence of rules, but in the discipline to follow them. And so, they put away their acid-washed jeans and neon-colored hair, and embraced a more structured, disciplined way of life. As we hit 2010, the 70s spirit of rebellion spawned a new breed of databases known as "NoSQL". The rise of NoSQL databases was a rebellion against the strict rules and constraints of SQL databases. NoSQL databases promised to free users from the rigid, structured world of SQL, with their strict schemas and complex query languages. Instead, NoSQL databases offered a simple, flexible approach to data storage and retrieval, allowing users to store and access data in whatever way they saw fit. In contrast to SQL databases, which provide strong consistency guarantees t

This is a 2009 paper by Leslie Lamport, Dahlia Malkhi, and Lidong Zhou. This paper was very well written and it was clear, and easy to follow (at least for me after having read so many Paxos papers). I finished reading the paper in one hour -- a record time for a slow reader like me. I knew about this work, but had not read the paper carefully before. I think I was overindexing on the reconfiguration aspect of Vertical Paxos, but by reading this paper (by going to the source) I found that the primary-backup replication application of Vertical Paxos was as much the point of emphasis as the reconfiguration aspect. The paper talks about the connection between Paxos and primary-backup replication applications, and presents vertical Paxos as a way of bridging these two. The paper delves down in to the leader handover and reconfiguration, and shows a practical application of these in the context of primary-backup replication protocols. Although it is possible to let the state machine reconf