Posts

Showing posts from March, 2023

Getting schooled by AI, colleges must evolve

As we enter the age of AI, it becomes more important for knowledge workers to excel in their strengths, and aim big for some strikes instead of settling for a comfortable average all around. How can colleges reform to endow graduates better? Here are my opinions, for what they are worth. Human skills for the AI era In the age of AI, doing rather than knowing becomes more important. Shallow information is worthless, but mastery of principles, critical thinking, and synthesis is priceless. Colleges should teach collaboration, entrepreneurship/innovation, communication/writing, and critical thinking and problem solving skills. How can colleges reform to cultivate these skills? First, they should transition from zero-sum mentality to the win-win mentality. This is not easy, because the system has been built on making students compete against each other and stack-ranking them. I don't know what kind of structural changes and scaffolding can help for this. I have some practical advice ...

Characterizing Microservice Dependency and Performance: Alibaba Trace Analysis

Image
This paper got the best paper award at SOCC 2021 . The paper conducts a comprehensive study of large scale microservices deployed in Alibaba clusters.  They analyze the behavior of more than 20,000 microservices in a 7-day period and profile their characteristics based on the 10 billion call traces collected. They find that: microservice graphs are dynamic in runtime most graphs are scattered to grow like a tree size of call graphs follows a heavy-tail distribution Based on their findings, they offer some practical tips about improving microservice runtime performance. They also develop a stochastic model to simulate microservice call graph dependencies and show that it approximates the dataset they collected (which is available at https://github.com/alibaba/clusterdata ). What are microservices? Microservices is a software development approach that divides an application into independently deployable services, owned by small teams organized around business capabilities. Each servi...

Aria: A Fast and Practical Deterministic OLTP Database

Image
This paper is from VLDB2020. Aria is an OLTP database that does epoch-based commits similar to the Silo paper we discussed last week. Unlike Silo, which was a single-node database, Aria is a distributed and deterministic database. Aria's biggest contribution is that it improves on Calvin by being able to run transactions without prior knowledge of read and write sets. Another nice thing in Aria is its deterministic re-ordering mechanism to commit transactions in an order that reduces the number of conflicts. Evaluation results on YCSB and TPC-C show that Aria outperforms other protocols by a large margin on a single node and up to a factor of two on a cluster of eight nodes. Aria versus Calvin Recall that Calvin uses locks ( here is a summary of the Calvin paper ). The key idea in Calvin is that read/write locks for a transaction are acquired according to the ordering of input transactions and the transaction is assigned to a worker thread for execution once all needed locks are gr...

Popular posts from this blog

Hints for Distributed Systems Design

Learning about distributed systems: where to start?

Making database systems usable

Looming Liability Machines (LLMs)

Foundational distributed systems papers

Advice to the young

Linearizability: A Correctness Condition for Concurrent Objects

Understanding the Performance Implications of Storage-Disaggregated Databases

Scalable OLTP in the Cloud: What’s the BIG DEAL?

Designing Data Intensive Applications (DDIA) Book