Skip to main content

Chapter 4: The Abstraction A Process

This page is a generated reference surface for selective reading. It exists to keep the learner apps guide-first while still preserving source access.

Learning objectives​

  • Explain the main ideas and vocabulary in The Abstraction A Process.
  • Work through the source examples for The Abstraction A Process without depending on raw chunk order.
  • Use The Abstraction A Process as selective reference when learner modules point back to Ostep.

Prerequisites​

  • Earlier prerequisite concepts leading into Chapter 4: The Abstraction A Process.

Module targets​

  • module-01-processes-scheduling
  • module-02-memory-management-virtual-memory
  • module-04-file-systems-io

AI companion modes​

  • Explain simply
  • Socratic tutor
  • Quiz me
  • Challenge my understanding
  • Diagnose my confusion
  • Generate extra practice
  • Revision mode
  • Connect forward / backward

Source-of-truth note​

This unit is anchored to Ostep and the source chapter "Chapter 4: The Abstraction A Process". Use external resources only to clarify, extend, or modernize details without replacing the chapter's conceptual spine.

External enrichment​

No chapter-specific enrichment resources are curated yet. Add them in the unit manifest when a source clearly improves learning.

Source provenance​

  • Primary source: Ostep
  • Source chapter 04: Chapter 4: The Abstraction A Process
  • Raw source file: 019-4-1-the-abstraction-a-process.md
  • Raw source file: 020-4-3-process-creation-a-little-more-detail.md
  • Raw source file: 021-4-4-process-states.md
  • Raw source file: 022-4-5-data-structures.md

Merged source​

The Abstraction A Process​

4.1 The Abstraction: A Process

4 The Abstraction: The Process​

In this chapter, we discuss one of the most fundamental abstractions that the OS provides to users: theprocess. The definition of a process, informally, is quite simple: it is arunning program[V+65,BH70]. The program itself is a lifeless thing: it just sits there on the disk, a bunch of instructions (and maybe some static data), waiting to spring into action. It is the operating system that takes these bytes and gets them running, transforming the program into something useful.

It turns out that one often wants to run more than one program at once; for example, consider your desktop or laptop where you might like to run a web browser, mail program, a game, a music player, and so forth.

In fact, a typical system may be seemingly running tens or even hundreds of processes at the same time. Doing so makes the system easy to use, as one never need be concerned with whether a CPU is available; one simply runs programs. Hence our challenge:

THECRUX OF THEPROBLEM:​

HOWTOPROVIDETHEILLUSIONOFMANYCPUS?

Although there are only a few physical CPUs available, how can the

OS provide the illusion of a nearly-endless supply of said CPUs?

The OS creates this illusion byvirtualizingthe CPU. By running one process, then stopping it and running another, and so forth, the OS can promote the illusion that many virtual CPUs exist when in fact there is only one physical CPU (or a few). This basic technique, known astime sharingof the CPU, allows users to run as many concurrent processes as they would like; the potential cost is performance, as each will run more slowly if the CPU(s) must be shared.

To implement virtualization of the CPU, and to implement it well, the OS will need both some low-level machinery as well as some highlevel intelligence. We call the low-level machinerymechanisms; mechanisms are low-level methods or protocols that implement a needed piece 1

TIP: USETIMESHARING(ANDSPACESHARING)

Time sharingis one of the most basic techniques used by an OS to share a resource. By allowing the resource to be used for a little while by one entity, and then a little while by another, and so forth, the resource in question (e.g., the CPU, or a network link) can be shared by many. The natural counterpart of time sharing isspace sharing, where a resource is divided (in space) among those who wish to use it. For example, disk space is naturally a space-shared resource, as once a block is assigned to a file, it is not likely to be assigned to another file until the user deletes it.

of functionality. For example, we'll learn later how to implement acontext switch, which gives the OS the ability to stop running one program and start running another on a given CPU; thistime-sharingmechanism is employed by all modern OSes.

On top of these mechanisms resides some of the intelligence in the

OS, in the form of policies. Policies are algorithms for making some kind of decision within the OS. For example, given a number of possible programs to run on a CPU, which program should the OS run? A scheduling policyin the OS will make this decision, likely using historical information (e.g., which program has run more over the last minute?), workload knowledge (e.g., what types of programs are run), and performance metrics (e.g., is the system optimizing for interactive performance, or throughput?) to make its decision.

The abstraction provided by the OS of a running program is something we will call aprocess. As we said above, a process is simply a running program; at any instant in time, we can summarize a process by taking an inventory of the different pieces of the system it accesses or affects during the course of its execution.

To understand what constitutes a process, we thus have to understand itsmachine state: what a program can read or update when it is running.

At any given time, what parts of the machine are important to the execution of this program?

One obvious component of machine state that comprises a process is its memory. Instructions lie in memory; the data that the running program reads and writes sits in memory as well. Thus the memory that the process can address (called itsaddress space) is part of the process.

Also part of the process's machine state areregisters; many instructions explicitly read or update registers and thus clearly they are important to the execution of the process.

Note that there are some particularly special registers that form part​

of this machine state. For example, theprogram counter(PC) (sometimes called theinstruction pointerorIP) tells us which instruction of the pro-

TIP: SEPARATEPOLICYANDMECHANISM

In many operating systems, a common design paradigm is to separate high-level policies from their low-level mechanisms [L+75]. You can think of the mechanism as providing the answer to ahowquestion about a system; for example,howdoes an operating system perform a context switch? The policy provides the answer to awhichquestion; for example, whichprocess should the operating system run right now? Separating the two allows one easily to change policies without having to rethink the mechanism and is thus a form ofmodularity, a general software design principle.

gram is currently being executed; similarly astack pointerand associated frame pointerare used to manage the stack for function parameters, local variables, and return addresses.

Finally, programs often access persistent storage devices too. SuchI/O informationmight include a list of the files the process currently has open.

4.2 Process API​

Though we defer discussion of a real process API until a subsequent chapter, here we first give some idea of what must be included in any interface of an operating system. These APIs, in some form, are available on any modern operating system.

  • Create: An operating system must include some method to cre- ate new processes. When you type a command into the shell, or double-click on an application icon, the OS is invoked to create a new process to run the program you have indicated.
  • Destroy:As there is an interface for process creation, systems also provide an interface to destroy processes forcefully. Of course, many processes will run and just exit by themselves when complete; when they don't, however, the user may wish to kill them, and thus an interface to halt a runaway process is quite useful.
  • Wait: Sometimes it is useful to wait for a process to stop running; thus some kind of waiting interface is often provided.
  • Miscellaneous Control:Other than killing or waiting for a process, there are sometimes other controls that are possible. For example, most operating systems provide some kind of method to suspend a process (stop it from running for a while) and then resume it (continue it running).
  • Status: There are usually interfaces to get some status information about a process as well, such as how long it has run for, or what state it is in. code static data heap stack

Process code static data

Program Loading:

Takes on-disk program and reads it into the address space of process

Disk

Figure 4.1:Loading: From Program To Process

Process Creation A Little More Detail​

4.3 Process Creation: A Little More Detail

One mystery that we should unmask a bit is how programs are transformed into processes. Specifically, how does the OS get a program up and running? How does process creation actually work?

The first thing that the OS must do to run a program is toloadits code and any static data (e.g., initialized variables) into memory, into the address space of the process. Programs initially reside ondisk(or, in some

modern systems,flash-based SSDs) in some kind ofexecutable format;

thus, the process of loading a program and static data into memory requires the OS to read those bytes from disk and place them in memory somewhere (as shown in Figure 4.1).

In early (or simple) operating systems, the loading process is doneeagerly, i.e., all at once before running the program; modern OSes perform the processlazily, i.e., by loading pieces of code or data only as they are needed during program execution. To truly understand how lazy loading of pieces of code and data works, you'll have to understand more about the machinery ofpagingandswapping, topics we'll cover in the future when we discuss the virtualization of memory. For now, just remember that before running anything, the OS clearly must do some work to get the important program bits from disk into memory.

Once the code and static data are loaded into memory, there are a few other things the OS needs to do before running the process. Some memory must be allocated for the program'srun-time stack(or just stack).

As you should likely already know, C programs use the stack for local variables, function parameters, and return addresses; the OS allocates this memory and gives it to the process. The OS will also likely initialize the stack with arguments; specifically, it will fill in the parameters to themain()function, i.e.,argcand theargvarray.

The OS may also allocate some memory for the program'sheap. In C programs, the heap is used for explicitly requested dynamically-allocated data; programs request such space by callingmalloc()and free it explicitly by callingfree(). The heap is needed for data structures such as linked lists, hash tables, trees, and other interesting data structures. The heap will be small at first; as the program runs, and requests more memory via themalloc()library API, the OS may get involved and allocate more memory to the process to help satisfy such calls.

The OS will also do some other initialization tasks, particularly as related to input/output (I/O). For example, in UNIX systems, each process by default has three openfile descriptors, for standard input, output, and error; these descriptors let programs easily read input from the terminal as well as print output to the screen. We'll learn more about I/O, file descriptors, and the like in the third part of the book onpersistence.

By loading the code and static data into memory, by creating and initializing a stack, and by doing other work as related to I/O setup, the OS has now (finally) set the stage for program execution. It thus has one last task: to start the program running at the entry point, namelymain(). By jumping to themain()routine (through a specialized mechanism that we will discuss next chapter), the OS transfers control of the CPU to the newly-created process, and thus the program begins its execution.

Process States​

4.4 Process States

Now that we have some idea of what a process is (though we will continue to refine this notion), and (roughly) how it is created, let us talk about the differentstatesa process can be in at a given time. The notion that a process can be in one of these states arose in early computer systems

[DV66,V+65]. In a simplified view, a process can be in one of three states:

  • Running: In the running state, a process is running on a processor. This means it is executing instructions.

  • Ready: In the ready state, a process is ready to run but for some reason the OS has chosen not to run it at this given moment.

  • Blocked: In the blocked state, a process has performed some kind of operation that makes it not ready to run until some other event takes place. A common example: when a process initiates an I/O request to a disk, it becomes blocked and thus some other process can use the processor. Descheduled

Running Ready

Scheduled

I/O: initiate I/O: done

Blocked

Figure 4.2:Process: State Transitions

If we were to map these states to a graph, we would arrive at the diagram in Figure 4.2. As you can see in the diagram, a process can be moved between the ready and running states at the discretion of the OS.

Being moved from ready to running means the process has beenscheduled; being moved from running to ready means the process has been descheduled. Once a process has become blocked (e.g., by initiating an

I/O operation), the OS will keep it as such until some event occurs (e.g.,

I/O completion); at that point, the process moves to the ready state again (and potentially immediately to running again, if the OS so decides).

Let's look at an example of how two processes might transition through some of these states. First, imagine two processes running, each of which only use the CPU (they do no I/O). In this case, a trace of the state of each process might look like this (Figure 4.3).

Time Process0 Process1 Notes 1 Running Ready 2 Running Ready 3 Running Ready 4 Running Ready Process0now done 5 - Running 6 - Running 7 - Running 8 - Running Process1now done

Figure 4.3:Tracing Process State: CPU Only

In this next example, the first process issues an I/O after running for some time. At that point, the process is blocked, giving the other process a chance to run. Figure 4.4 shows a trace of this scenario.

More specifically, Process0initiates an I/O and becomes blocked waiting for it to complete; processes become blocked, for example, when read-

Time Process0 Process1 Notes 3 Running Ready Process0initiates I/O 4 Blocked Running Process0is blocked, 5 Blocked Running so Process1runs 6 Blocked Running 7 Ready Running I/O done 8 Ready Running Process1now done 9 Running - 10 Running - Process0now done

Figure 4.4:Tracing Process State: CPU and I/O ing from a disk or waiting for a packet from a network. The OS recognizes Process0 is not using the CPU and starts running Process1. While

Process1 is running, the I/O completes, moving Process0 back to ready.

Finally, Process1finishes, and Process0runs and then is done.

Note that there are many decisions the OS must make, even in this​

simple example. First, the system had to decide to run Process1 while

Process0 issued an I/O; doing so improves resource utilization by keeping the CPU busy. Second, the system decided not to switch back to

Process0 when its I/O completed; it is not clear if this is a good decision or not. What do you think? These types of decisions are made by the

OSscheduler, a topic we will discuss a few chapters in the future.

Data Structures​

4.5 Data Structures

The OS is a program, and like any program, it has some key data structures that track various relevant pieces of information. To track the state of each process, for example, the OS likely will keep some kind ofprocess list for all processes that are ready, as well as some additional information to track which process is currently running. The OS must also track, in some way, blocked processes; when an I/O event completes, the OS should make sure to wake the correct process and ready it to run again.

Figure 4.5 shows what type of information an OS needs to track about each process in the xv6 kernel [CK+08]. Similar process structures exist in "real" operating systems such as Linux, Mac OS X, or Windows; look them up and see how much more complex they are.

From the figure, you can see a couple of important pieces of information the OS tracks about a process. Theregister contextwill hold, for a stopped process, the contents of its registers. When a process is stopped, its registers will be saved to this memory location; by restoring these registers (i.e., placing their values back into the actual physical registers), the

OS can resume running the process. We'll learn more about this technique known as acontext switchin future chapters.

// the registers xv6 will save and restore
// to stop and subsequently restart a process
struct context {
int eip; int esp; int ebx; int ecx; int edx; int esi; int edi; int ebp;
};
// the different states a process can be in
enum proc_state { UNUSED, EMBRYO, SLEEPING,
RUNNABLE, RUNNING, ZOMBIE };
// the information xv6 tracks about each process
// including its register context and state
struct proc {
char *mem; // Start of process memory

uint sz; // Size of process memory

char *kstack; // Bottom of kernel stack
// for this process

enum proc_state state; // Process state int pid; // Process ID

struct proc *parent; // Parent process
void *chan; // If non-zero, sleeping on chan
int killed; // If non-zero, have been killed
struct file *ofile[NOFILE]; // Open files
struct inode *cwd; // Current directory
struct context context; // Switch here to run process
struct trapframe *tf; // Trap frame for the
// current interrupt
};

Figure 4.5:The xv6 Proc Structure

You can also see from the figure that there are some other states a process can be in, beyond running, ready, and blocked. Sometimes a system will have aninitial state that the process is in when it is being created.

Also, a process could be placed in afinal state where it has exited but has not yet been cleaned up (in UNIX-based systems, this is called the zombiestate1). This final state can be useful as it allows other processes (usually theparentthat created the process) to examine the return code of the process and see if the just-finished process executed successfully (usually, programs return zero in UNIX-based systems when they have accomplished a task successfully, and non-zero otherwise). When finished, the parent will make one final call (e.g., wait()) to wait for the completion of the child, and to also indicate to the OS that it can clean up any relevant data structures that referred to the now-extinct process.

1Yes, the zombie state. Just like real zombies, these zombies are relatively easy to kill.

However, different techniques are usually recommended.

ASIDE: DATASTRUCTURE-- THEPROCESSLIST​

Operating systems are replete with various important data structures that we will discuss in these notes. Theprocess listis the first such structure. It is one of the simpler ones, but certainly any OS that has the ability to run multiple programs at once will have something akin to this structure in order to keep track of all the running programs in the system.

Sometimes people refer to the individual structure that stores information about a process as aProcess Control Block(PCB), a fancy way of talking about a C structure that contains information about each process.

4.6 Summary​

We have introduced the most basic abstraction of the OS: the process.

It is quite simply viewed as a running program. With this conceptual view in mind, we will now move on to the nitty-gritty: the low-level mechanisms needed to implement processes, and the higher-level policies required to schedule them in an intelligent way. By combining mechanisms and policies, we will build up our understanding of how an operating system virtualizes the CPU.

Per Brinch Hansen

Communications of the ACM, Volume 13, Number 4, April 1970

This paper introduces one of the firstmicrokernelsin operating systems history, called Nucleus. The idea of smaller, more minimal systems is a theme that rears its head repeatedly in OS history; it all began with Brinch Hansen's work described herein.

[CK+08] "The xv6 Operating System"

Russ Cox, Frans Kaashoek, Robert Morris, Nickolai Zeldovich

From: http://pdos.csail.mit.edu/6.828/2008/index.html

The coolest real and little OS in the world. Download and play with it to learn more about the details of how operating systems actually work.

[DV66] "Programming Semantics for Multiprogrammed Computations"

Jack B. Dennis and Earl C. Van Horn

Communications of the ACM, Volume 9, Number 3, March 1966

This paper defined many of the early terms and concepts around building multiprogrammed systems.

[L+75] "Policy/mechanism separation in Hydra"

R. Levin, E. Cohen, W. Corwin, F. Pollack, W. Wulf

SOSP 1975

An early paper about how to structure operating systems in a research OS known as Hydra. While

Hydra never became a mainstream OS, some of its ideas influenced OS designers.

[V+65] "Structure of the Multics Supervisor"

V.A. Vyssotsky, F. J. Corbato, R. M. Graham

Fall Joint Computer Conference, 1965

An early paper on Multics, which described many of the basic ideas and terms that we find in modern systems. Some of the vision behind computing as a utility are finally being realized in modern cloud systems.