Metadata

The full title of the book is Enlightenment Now: The Case for Reason, Science, Humanism, and Progress . It came out in 2019, and Bill Gates called this book his "new favorite book of all time." For me, it is an OK book, I give it a 3.5 stars out of 5.  The book makes an important point about the importance of science and reason, and how much we accomplished thanks to them. As Yuval Harari nicely summarized, the book "extols the amazing achievements of modernity, and demonstrates that humankind has never been so peaceful, healthy, and prosperous." Ok, so it is a feel-good book, nothing wrong with that, right? Well, it is a very long-winded book, and it is hard not to get bored after some time. I really liked the first 50 pages: "Ok, this is going good, we have a good rapport as writer and reader". After page 100, I was like "OK dude I really get it, let's move on to something new now", and after page 150, I was pleading "Stop, stop, this

This paper appeared in OSDI 18, and is authored by Joshua Lockerman, Jose M. Faleiro, Juno Kim, Soham Sankaran, Daniel J. Abadi, James Aspnes, Siddhartha Sen, Mahesh Balakrishnan. The paper does not suggest many novel ideas or algorithms, but rather shows a nice amalgamation of existing ideas and algorithms for providing a more scalable version of a shared log across multiple datacenters. Having a shared log, and funneling all updates through a globally shared log  provides simplicity for building fault-tolerant consistent distributed systems. Tango paper from SOSP'13 , upon which they build here, argued for the simplicity benefits of such a system. Recently Jay Kreps wrote a nice and accessible explanation of the benefits of using globally shared logs for building distributed systems. Despite the simplicity it provides, maintaining a shared log with a system-wide total order is impractical, because it is expensive : At large scale, the single sequencer will be a bottleneck for the

This paper appeared at Eurosys 2020, and is authored by Vitor Enes, Carlos Baquero, Tuanir França Rezende, Alexey Gotsman, Matthieu  Perrin, and Pierre Sutra. The paper introduces Atlas, a consensus protocol similar to EPaxos, that builds on the Fast Paxos idea. Most people call these protocols leaderless, but I think that is a misnomer. I call these protocols as opportunistic/any-leader protocols. In these protocols, any node can propose a command by becoming an opportunistic leader and using a fast path. If there is no other simultaneously proposed command by another opportunistic leader or if this command commutes with all simultaneously proposed commands, the fast path is enough for achieving consensus. In the presence of concurrent non-commuting commands, the protocol will have to take a slow path, the purpose of which is to establish (teach to the other nodes) the dependencies learned for this command in the fast path attempt. This opportunistic/any-leader family of protocols do

I am looking for PhD students to work with me on my project "Paxos Unpacked". I am attaching a short abstract below. The work done here is very useful for distributed databases and cloud computing systems. So if you are interested in a distributed systems PhD, or you know someone interested in a distributed systems PhD, please let me know by emailing me with a short CV and qualifications.  I will have research assistantship funding available for qualifying applicants after they get accepted to the PhD program at University at Buffalo, SUNY. Paxos Unpacked Due to their excellent fault-tolerance properties, Paxos protocols have been employed by many cloud computing systems for consistent coordination of clusters of computers. This project aims to show that when properly customized, Paxos family of protocols can also deliver top-notch performance and scalability, which they currently lack. To this end, the project will design high-performance, scalable, practical, and usable Pax

Making Databases Work: the Pragmatic Wisdom of Michael Stonebraker, December 2018. This is available as a free download at https://dl.acm.org/doi/book/10.1145/3226595 . In addition to databases, the book talks about Stonebraker's approach to advising and finding research problems .   Datacenter RPCs Can Be General and Fast CockroachDB: The Resilient Geo-Distributed SQL Database (Sigmod 2020) eXtreme Modelling in Practice (VLDB 2020) Beyond Analytics: the Evolution of Stream Processing Systems (Sigmod 2020) Follow on Twitter:  Jonathan Ellis ,  Neil Gunther ,  Steven Strogatz   ICYMI: Research, writing, and career advice  and Learning about distributed systems: where to start? Podcast suggestion: Weird name aside, this is a nice podcast. This one was particularly a very good listen. I put the Stephen Cope's book on my reading queue. As for true calling, I have this weird fixation with distributed systems. The first time I took an undergraduate class that introduced distributed

Wow! I had written more than 30 advice posts over the last decade.  When I was writing them, I was hoping to first help myself. You see, I understand a concept well only after I think about it in depth, distill it to a small set of principles, and connect it with my existing knowledge-base. Writing helps me achieve this, so I write on things I want to understand better. These advice can also help you by showing you what to expect as you take on new projects. You can get a head start and avoid costly mistakes by learning from someone who had traveled down that road.  I selected 21 posts, and categorized them as Research, Writing, and Career. Enjoy! Research Learn by doing, realize we are all stupid, and just ask   Publish--not perish, work hard, and understand the philosophy of research   Endure the twisty journey to taste the joy of figuring things out Sequence it, write it down, find the right perspective, and persevere   Use writing to generate your best ideas, insight, and content  

This paper is by Pulkit A. Misra, Jeffrey S. Chase, Johannes Gehrke, Alvin R. Lebeck, and it appeared at ASPLOS'17. The paper describes *Semel*, a durable multi-version read-optimized key-value store, and *Milana*, a distributed OCC transaction system built on top of Semel. The main ideas in the paper are to exploit precision time (PTP) and efficient persistent memory based on flash NVM/SSD, and to show how they helped for building OCC based distributed transactional system. In a way, this paper revisited and revised Thor and showed the benefits achievable from using modern clocks and storage technologies inside a datacenter. Thor (Sigmod 1995), was from Barbara Liskov's group, and introduced loosely synchronized clocks for OCC, and performed validation on the storage servers. Before we summarize Semel and Milana, let's recap the main contributions of this paper: Move ordering off the critical path: Each version of a key's value is timestamped using PTP. These

This is definitely not a "learn distributed systems in 21 days" post. I recommend a principled, from the foundations-up, studying of distributed systems, which will take a good three months in the first pass, and many more months to build competence after that. If you are practical and coding oriented you may not like my advice much. You may object saying, "Shouldn't I learn distributed systems with coding and hands on? Why can I not get started by deploying a Hadoop cluster, or studying the Raft code." I think that is the wrong way to go about learning distributed systems, because seeing similar code and programming language constructs will make you think this is familiar territory, and will give you a false sense of security. But, nothing can be further from the truth. Distributed systems need radically different software than centralized systems do.  --A. Tannenbaum This quotation is literally the first sentence in my distributed systems syllabus. Inst

This paper appeared in NSDI'19, and is by Seo Jin Park and John Ousterhout. You can access the paper as well as the conference presentation video here.  Here is also a YouTube link to a presentation of the paper in our Zoom DistSys reading group. This was the 10th paper we discussed in our Zoom group. We have been meeting on Zoom every Wednesday since April 1st at 15:30 EST. Join our Slack channel to get paper information, discussion and meeting links  https://join.slack.com/t/distsysreadinggroup/shared_invite/zt-eb853801-3qEpQ5xMyXpN2YGvUo~Iyg The problem The paper considers performing consistent replication fast, in 1 round trip time (RTT) from client. Of course, if we were to do asynchronous replication, we could get 1 RTT. The client would contact the leader in the replication group, the leader will acknowledge back and start replicating to backups/followers asynchronously. The problem with this scheme is that it is not fault-tolerant. The leader may crash after acking