Skip to main content

Build Your Own Operating System

"An operating system is just a program that runs other programs." — every OS textbook, eventually

Building an OS — even a small one — is the most demanding project in this curriculum, and one of the most rewarding. By the end you have a kernel that boots on bare metal (or in QEMU), manages its own memory, schedules processes, handles interrupts, and reads/writes to a disk. Plan for 8–16 weeks if you're serious.

1. Overview & motivation​

A minimal OS kernel needs:

  1. Bootloader — code that runs from BIOS/UEFI and loads the kernel.
  2. Initialization — set up CPU mode, GDT, IDT, page tables.
  3. Interrupt handling — keyboard, timer, page faults, syscalls.
  4. Memory management — physical frame allocator, virtual memory, kernel heap.
  5. Scheduler — switching between tasks.
  6. Filesystem — at least reading from disk.
  7. User-space — running unprivileged code with system calls.

What you can only learn by building one:

  • Why rings, modes, and page tables are the hardware mechanisms behind every security boundary you take for granted.
  • Why interrupts are how an OS reasserts control over a CPU it gave to a user process.
  • Why fork() is literally "copy the page tables, share until written" — and why that matters.
  • Why system calls are slower than function calls but cheaper than IPC.
  • Why running your own code on bare metal feels different from running it inside another OS.

2. Where this fits in the degree​

  • Phase: Systems
  • Semester: 5 (OS and Networking)
  • Modules deepened: Module 1 (processes & scheduling), Module 2 (memory management & virtual memory), Module 3 (concurrency — interrupts are the original concurrency primitive), Module 4 (file systems & I/O).

This project is the spine of Semester 5. Don't take it as an elective; treat it as the semester project's core, with the required Sem 5 project becoming the operational layer on top.

Cross-phase relevance:

  • Concretizes everything in Operating Systems: Three Easy Pieces.
  • Direct connection to the Memory Allocator tutorial — your kernel needs one.

3. Prerequisites​

  • Complete the Memory Allocator tutorial first. Bringing your own allocator code into the kernel is the cleanest path.
  • C or Rust. Strong C/Rust skills are mandatory; this is not a learn-the-language project.
  • Assembly: at least reading x86-64 or ARM. You will write small amounts.
  • Comfort with objdump, gdb, and Makefiles.
  • Familiarity with QEMU.

4. Theory & research​

Required reading​

  • Remzi & Andrea Arpaci-Dusseau, Operating Systems: Three Easy Pieces — pages.cs.wisc.edu/~remzi/OSTEP. Free. The single best modern OS textbook. ⭐ read in parallel.
  • Phil Opp, "Writing an OS in Rust" — os.phil-opp.com. The canonical modern OS-building tutorial. 17+ blog-post chapters. Rust + bare-metal x86_64. ⭐ recommended primary path.
  • Tanenbaum & Bos, Modern Operating Systems — alternative textbook. Standard.
  • Andrew Tanenbaum, Operating Systems: Design and Implementation — the original MINIX book, the reason Linux exists.
  • Silberschatz, Galvin, Gagne, Operating System Concepts — the "dinosaur book". Widely used in undergraduate courses.

Architecture-specific references​

  • Intel® 64 and IA-32 Architectures Software Developer's Manual, Volume 3 — protected mode, paging, interrupts. Free PDF. You will need this.
  • AMD64 Architecture Programmer's Manual, Volume 2 — same content, often more readable.
  • OSDev wiki — wiki.osdev.org. Community resource. Goes very deep.

For RISC-V (alternative architecture)​

5. Curated tutorial list (from BYO-X)​

  • Assembly: Writing a Tiny x86 Bootloader
  • Assembly: Baking Pi – Operating Systems Development — Cambridge tutorial for Raspberry Pi (ARM)
  • C: Building a software and hardware stack for a simple computer from scratch [video]
  • C: Operating Systems: From 0 to 1 — github.com/tuhdo/os01
  • C: The little book about OS development — littleosbook.github.io — clear, concise
  • C: Roll your own toy UNIX-clone OS — JamesM's kernel development tutorials
  • C: Kernel 101 – Let's write a Kernel, Kernel 201 – Let's write a Kernel with keyboard and screen support
  • C: Build a minimal multi-tasking kernel for ARM from scratch — arjunsreedharan/elementary-os
  • C: How to create an OS from scratch — cfenollosa/os-tutorial
  • C: Malloc tutorial
  • C: Hack the virtual memory
  • C: Learning operating system development using Linux kernel and Raspberry Pi — s-matyukevich/raspberry-pi-os ⭐ excellent for ARM
  • C: Operating systems development for Dummies
  • C++: Write your own Operating System [video] — Algorithman, full series
  • C++: Writing a Bootloader
  • Rust: Writing an OS in Rust — Phil Opp, os.phil-opp.com ⭐ recommended primary
  • Rust: Add RISC-V Rust Operating System Tutorial — stephenmarz.com
  • (any): Linux from scratch — linuxfromscratch.org — a different project: assembling a working Linux distro from sources, not writing a kernel

There are two clearly distinct paths.

Rust, bare-metal x86_64. 17 blog posts. Each is ~30 minutes of reading + 1–2 hours of work. Covers:

  1. A freestanding Rust binary.
  2. A minimal kernel that prints to VGA text mode.
  3. Testing.
  4. CPU exceptions.
  5. Double faults.
  6. Hardware interrupts.
  7. Introduction to paging.
  8. Paging implementation.
  9. Heap allocation.
  10. Allocator designs.
  11. Async/await (a unique chapter for a tutorial OS). 12+. Edition 2 in progress.

Pros: Excellent writing, modern, Rust's safety helps you skip whole bug classes, very motivated community.
Cons: Rust toolchain quirks. Doesn't get to user-space.

Path B: MIT 6.S081 + xv6​

Read xv6 source. Implement the labs (file system, virtual memory, multithreading, locks). Plan: 6.S081 publishes everything online. The labs are graded against tests.

Pros: Gets you to user-space, fork, exec, shell — a complete miniature Unix. The lab framework is more guided than a free-form tutorial.
Cons: RISC-V (less familiar than x86). C. The "tutorial" is reading xv6's code, which is intentionally minimal.

Path C: Raspberry Pi OS (ARM)​

Sergey Matyukevich's tutorial. ARM64. Real hardware. Very thorough.

Pros: Runs on real hardware (Pi 3 / 4). ARM. Modern.
Cons: Longest of the three.

For this degree, Path A (Phil Opp) is the default. Path B is the right alternative if you prefer C and a more textbook-shaped experience.

7. Implementation milestones (Phil Opp path)​

Milestone 1: Freestanding binary + bootable kernel​

A Rust binary that doesn't depend on std. Use bootimage to create a bootable disk image. Prints "Hello World!" to VGA text mode.

#![no_std]
#![no_main]

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! { loop {} }

#[no_mangle]
pub extern "C" fn _start() -> ! {
    let vga_buffer = 0xb8000 as *mut u8;
    for (i, &byte) in HELLO.iter().enumerate() {
        unsafe {
            *vga_buffer.offset(i as isize * 2) = byte;
            *vga_buffer.offset(i as isize * 2 + 1) = 0xb;
        }
    }
    loop {}
}

Evidence: cargo run boots QEMU and you see "Hello World!".

Milestone 2: VGA text mode driver​

Wrap the VGA buffer in a Writer that implements core::fmt::Write so println! works.

Evidence: println!("Hello, {}", "Rust kernel") works inside your kernel.

Milestone 3: CPU exceptions (IDT)​

Set up an Interrupt Descriptor Table. Handle the divide-by-zero exception. Handle breakpoints.

Evidence: Triggering int 3 from the kernel produces a controlled handler call, not a triple fault.

Milestone 4: Double faults​

Trickier than it sounds. A double fault occurs when the CPU fails to handle an exception. You need a separate stack for the double-fault handler.

Evidence: A deliberately-bad stack pointer causes a double fault, which your handler catches.

Milestone 5: Hardware interrupts (timer, keyboard)​

Configure the 8259 PIC (or APIC). Handle timer ticks and keyboard input.

Evidence: Kernel prints a . every timer tick. Pressing keys prints scancodes; with a layout table, prints characters.

Milestone 6: Paging​

Read the existing page tables set up by the bootloader. Then build your own. Map a frame, unmap a frame, walk the page table.

This is the conceptual high point. Paging is the mechanism behind virtual memory.

Evidence: Allocate a frame, map it to a virtual address, write to it, unmap it.

Milestone 7: Heap allocation​

Implement a kernel heap allocator. Three designs in Phil Opp: bump, linked-list, fixed-size block. Each is instructive.

This is where your memory allocator tutorial directly applies.

Evidence: Box::new(...) works in kernel code. Vec works. Allocating a million small objects does not OOM.

Milestone 8: Tasks and async (Phil Opp edition 1) OR user-space, fork, exec (xv6 path)​

Phil Opp ends with cooperative async tasks running in kernel mode.

For a more traditional OS, you would add: user-mode, system calls, fork, exec, simple filesystem.

8. Tests & evidence​

TestHow
BootsQEMU loads the image and runs the entry point
VGA outputVisible text on screen
InterruptsTimer ticks measurable; keyboard input handled
PagingAllocate/map/write/unmap works; double-mapping triggers expected behavior
HeapAllocator passes the same tests as your memory-allocator project
ResilienceTriggering a CPU exception (divide-by-zero, page fault) is handled gracefully
Boot on real hardware (optional)Write image to USB; boot a real machine

Phil Opp's chapters include cargo test infrastructure that runs your kernel in QEMU and asserts behaviors.

9. Common pitfalls​

  • Linker scripts. You will fight one. Read Phil Opp's chapter slowly.
  • #![no_std] blindness. Most Rust crates assume std. Many useful ones don't work. Find no_std alternatives or write minimal versions.
  • Unsafe everywhere. OS code is unsafe by necessity. Comment every unsafe block with its invariant.
  • Triple faults are silent. QEMU resets. With -no-reboot flag, you'll see at least some failure trace.
  • GDB on a kernel is awkward. -s -S flags + gdb + target remote :1234 is the dance. Practice it on Milestone 1.
  • Race conditions before you have locking. Interrupts can fire any time. Disable interrupts during critical sections (and document this).
  • Confusing virtual and physical addresses. Once paging is on, the same number can be either depending on context. Use newtypes.
  • Forgetting to flush the TLB. After unmapping a page, the CPU may still have a cached translation. invlpg instruction.

10. Extensions​

  • User-space — drop a process to ring 3. Implement syscall/sysret. Run a hello-world user program.
  • fork() — copy page tables, share with COW.
  • Simple file system — read FAT12 from disk. (Floppy disk image is fine for tutorial.)
  • VFS layer — virtual filesystem abstraction over multiple filesystems.
  • Multi-core — boot the other CPU cores. Per-CPU data, cross-CPU IPIs.
  • Networking — implement a minimal TCP/IP stack inside the kernel. See Network Stack tutorial.
  • GUI — frame-buffer-based, simple windowing.

A complete tutorial OS could keep growing for years. Linus's first commit was 10,000 lines; Linux is now 30 million.

11. Module integration​

ModuleWhat the OS deepens
Sem 4 Module 3 — Computer organizationYou'll set up the GDT/IDT, manage rings, write a small amount of assembly.
Sem 5 Module 1 — Processes & schedulingYour scheduler is real, not a simulator.
Sem 5 Module 2 — Virtual memoryYou set up the page tables. Page faults call your code.
Sem 5 Module 3 — ConcurrencyInterrupts are the original concurrency.
Sem 5 Module 4 — File systems & I/OIf you implement a filesystem, this is direct.
Memory Allocator tutorialKernel heap is the direct use case.
Compiler tutorialOnce you have an OS, you can run programs you compiled.

12. Portfolio framing​

What to publish:

  • Source: clearly organized as src/{boot, mem, interrupts, drivers, scheduler}.rs.
  • A Makefile or cargo/bootimage setup that builds and boots the image.
  • A README with:
    • A video of QEMU booting your kernel (the single strongest demo).
    • List of features implemented.
    • List of features not implemented (honesty about scope).
    • Acknowledgement of Phil Opp or your reference tutorial.
  • A docs/architecture.md with diagrams of memory layout, interrupt flow, scheduling.

What to keep private:

  • None of it — this is portfolio-grade by design.

Reviewer entry points:

  • src/main.rs — the _start and main loop.
  • src/memory.rs — paging & frame allocator.
  • src/interrupts.rs — IDT setup.
  • README must include the QEMU video, the feature list, and the scope-honesty section.

This is a top-tier portfolio project. "I wrote a kernel that boots on bare metal and manages memory" is one of the most distinctive sentences a software engineer can put on a résumé.

Source​

This tutorial draws from the BYO-X catalog "Operating System" section. OSTEP is the canonical OS textbook; Phil Opp's blog is the canonical modern OS-building tutorial.