OSTEP Chapters 4,5

I recently started reading "Operating Systems: Three Easy Pieces" (OSTEP) as part of Phil Eaton's offline reading group. We are tackling a very doable pace of 2 chapters a week.

The book is structured into three major parts: Virtualization, Concurrency, and Persistence. It is openly accessible to everyone for free, which is a tremendous contribution to computer science education by the Arpaci-Dusseau couple (Remzi and Andrea).

This is a very user-friendly book, sprinkled with a lot of jokes and asides that keep the mood light. The fourth wall is broken upfront, the authors talk directly to you, which is great. It makes it feel like we are learning together rather than being lectured at. Their approach is more than superficial; it inspires you, motivates the problems, and connects them to the big picture context. The book  builds scaffolding through "The Crux" of the problem and "Aside" panels. It actively teaches you the thought processes, not just the results. I’ve talked about the importance of this pedagogical style before: Tell me about your thought process, not just the results.

Chapter 4: The Process Abstraction

In the first part, Virtualization, we start with the most fundamental abstraction: The Process. Informally, a process is simply a running program.

To understand a process, we have to look at its machine state. This includes its address space (memory), registers (like the Program Counter and Stack Pointer), and I/O information (like open files).

One distinct memory component here is the Stack versus the Heap.

• The Stack: C programs use the stack for local variables, function parameters, and return addresses. It's called a stack because it operates on a Last-In, First-Out basis, growing and shrinking automatically as functions are called and return.
• The Heap: This is used for explicitly requested, dynamically-allocated data (via malloc in C). It’s needed for data structures like linked lists or hash tables where the size isn't known at compile time.

Reading this reminded me of 30 years ago, using Turbo Pascal to debug programs line-by-line. I remember watching the call stack in the IDE, seeing exactly how variables changed and stack frames were pushed and popped as the program executed. Turbo Pascal in the 1990s was truly a thing of beauty for learning these concepts interactively.

The book provides some excellent diagrams to visualize these concepts.

This figure shows how the OS takes a lifeless program on disk (code and static data) and hydrates it into a living process in memory. Like the trisolarans dehydrating to survive dormancy, the program exists as inert structure until the OS loads its bytes into an address space, reconstructing state and enabling execution. Only after this hydration step can the program spring into action.

To track all this, the OS needs a data structure. In the xv6 operating system (a teaching OS based on Unix), this is the "struct proc".

Figure 4.5 shows the code for the process structure in xv6. It tracks the process state (RUNNING, RUNNABLE, etc.), the process ID (PID), pointers to the parent process, and the context (registers) saved when the process is stopped. In other words, this struct captures the "inventory" the OS keeps for every running program.

Chapter 5: The Process API via Fork and Exec

In Chapter 5, we get to the UNIX process API. This is where the beauty of UNIX design really shines. The way UNIX creates processes is a pair of system calls: fork() and exec(). The fork() call is weird: it creates an almost exact copy of the calling process. Then things get really weird with exec(), as exec() takes an existing process and transforms it into a different running program by overwriting its code and static data.

This code snippet demonstrates the fork() call. It shows how the child process comes to life as if it had called fork() itself, but with a return code of 0, while the parent receives the child's PID.

This figure puts it all together. It shows a child process using execvp() to run the word count program ('wc') on a source file, effectively becoming a new program entirely.

You might ask: Why not just have a single CreateProcess() system call? Why this dance of cloning and then overwriting? The separation of fork() and exec() is essential for building the UNIX Shell. It allows the shell to run code after the fork but before the exec. This space is where the magic of redirection and pipes happens. For example, if you run 'wc p3.c > newfile.txt', the shell:

1. Calls fork() to create a child.
2. Inside the child (before exec): Closes standard output and opens 'newfile.txt'. Because UNIX starts looking for free file descriptors at zero, the new file becomes the standard output.
3. Calls 'exec()' to run 'wc'.
4. The 'wc' program writes to standard output as usual, unaware that its output is now going to a file instead of the screen.

Figure 5.4 shows the code for redirection. It shows the child closing STDOUT_FILENO and immediately calling 'open()', ensuring the output of the subsequent 'execvp' is routed to the file.

This design is unmatched. It allows us to compose programs using pipes and redirection without changing the programs themselves. It is a testament to the brilliance of Thompson and Ritchie. The book refers to this as Lampson's Law: "Get it right... Neither abstraction nor simplicity is a substitute for getting it right". The UNIX designers simply got it right, and that is why this design still holds up 55 years later as the gold standard. 

As a final thought to spark some discussion, it is worth remembering that science advances by challenging even its most sacred cows. While we laud fork() and exec() as brilliant design, a 2019 HotOS paper titled "A fork() in the road" offers a counterpoint, and argues that fork() was merely a "clever hack" for the constraints of the 1970s that has now become a liability. The authors contend that it is a terrible abstraction for modern programmers that compromises OS implementations, going so far as to suggest we should deprecate it and teach it only as a historical artifact. This kind of debate is exactly how science works; as the OSTEP authors remind us in the end note for the chapter.