Build Your Own X: Foundations
This section turns the Build Your Own X idea into elective deep-dives that run alongside (not instead of) the required Foundations work. Each project is a self-contained tutorial: research links, the curated external tutorial list from the BYO-X repo (preserved with language tags), recommended primary path, implementation milestones with code sketches, tests, pitfalls, and how the project deepens specific modules.
Use these as portfolio-grade electives. The required semester project (project.md) still owns the rubric and the checkpoint.
The improvement target for Foundations projects is scientific evidence. A completed neural-network, LLM, or renderer project must show the math, the implementation, the experiment, and the failure analysis. A notebook that only "runs" is not enough.
The BYO-X repository is a curated catalog of external tutorials. Read them when needed, then write your own code, your own explanation, and your own tests. Cite sources in notes/research.md. Do not paste external tutorials into your portfolio.
Projects in this phase
All three are substantial math-and-algorithms-heavy projects with canonical free tutorials. They share infrastructure (vectors, matrices, probability) and complement each other.
| Project | Primary module fit | Difficulty | Languages typically used |
|---|---|---|---|
| Neural Network from Scratch | Sem 1 · Linear algebra / Sem 2 · DP | Medium | Python |
| Build Your Own LLM (GPT from scratch) | Sem 1 · Linear algebra + probability / Sem 2 · DP | Medium–Hard | Python (PyTorch) |
| 3D Renderer (Ray Tracer) | Sem 1 · Linear algebra + statistics / Sem 2 · Advanced structures | Medium | C++, Python |
Cross-phase projects (covered in later phases) you may want to preview:
- Systems phase: Memory Allocator, Interpreter, Compiler, Regex Engine, Operating System
- Architecture phase: Database (KV), Search Engine, Blockchain, Consensus / Raft
- Production phase: Front-end Framework, Physics Engine
Foundations module → project map
This table is the roadmap. Each row points to one elective project that deepens that module's required content.
Semester 00: Orientation, Semester 01: Math Foundations
These semesters are about disciplined practice on foundational material. The required semester projects own the rubric.
If you want to preview a BYO project as motivation, Neural Network from Scratch and 3D Renderer are the strongest ties to Sem 1 linear algebra and statistics. Read their overviews to see how the math is applied. Defer full implementation until you have Sem 1 Module 4 (linear algebra) and ideally Sem 2 Module 4 (DP — needed for backprop).
Semester 01: Math Foundations
| Module | Recommended BYO project | Why it fits |
|---|---|---|
| Module 4 — Linear algebra | Neural Network, LLM, 3D Renderer | All three are intensive matrix-math projects with different application domains |
| Module 5 — Statistics & probability | 3D Renderer (Monte Carlo), LLM (cross-entropy, sampling) | Probability becomes operational, not abstract |
Semester 02: Algorithms
| Module | Recommended BYO project | Why it fits |
|---|---|---|
| Module 1 — Algorithm analysis & design | None as a standalone project; revisit at Module 5 | Use this module for the algorithms library project |
| Module 2 — Sorting, searching, structures | None directly — preview the Systems-phase Interpreter for state-machine work | Algorithmic work in Sem 2 is best done through the required project.md |
| Module 3 — Graph algorithms | None — depth-first / breadth-first are practiced in the Algorithms project | Save graph-heavy BYO work for Search Engine and Web Browser Engine |
| Module 4 — Dynamic programming | Neural Network (backprop) and LLM (backprop through transformer) | Backprop is literally DP over a computational graph |
| Module 5 — Advanced structures | 3D Renderer (BVH in Book 2) | Spatial trees, used in a real-world domain |
Semester 03: Software Design
The required project in Semester 3 is a refactor diary plus design pattern application. BYO-X projects do not fit naturally here — they are about new mechanisms, not about cleaning existing code. Defer all systems-flavored BYO work to Semester 4.
Build rules (apply to every project)
- Start with a walking skeleton. The smallest version that runs end-to-end is more useful than a half-done large version.
- Write the README before the code feels finished. Setup, run, test, limitations.
- Keep
notes/research.mdwith every external link you read and what you took from it. - Keep
notes/decisions.mdwith at least three tradeoffs you made. - Finish with evidence: tests, traces, screenshots, benchmarks, or proof sketches.
- One failure case minimum per project. A working demo is not the same as understanding.
- Portfolio safety: remove secrets, anonymise data, summarise private coursework before publishing.
Project quality rubric
| Dimension | Minimum acceptable evidence | Strong evidence |
|---|---|---|
| Math contract | formulas and tensor/vector shapes listed | shape checks, derivations, and gradient checks |
| Baseline | one naive implementation | naive vs optimized comparison with explanation |
| Experiment | one reproducible run | fixed seed, config file, metrics, and plotted learning curve |
| Correctness | output looks plausible | unit tests plus numerical checks against a trusted library |
| Failure analysis | one known limitation | documented overfit/underfit, instability, or sampling artifact |
| Portfolio | README and result image/table | experiment report, ablations, and reproducible command |
Use local Build Your Own sources as selective reinforcement by source ID, not as generated reference pages. The current local IDs used in this phase are build-your-own/neural-networks-kilho-shin, build-your-own/building-llms-from-scratch, build-your-own/llms-2024, and build-your-own/large-language-model-raschka.
How to use each project tutorial
Every tutorial in this section follows the same shape:
- Overview & motivation — what it is, what you can only learn by building it
- Where this fits — phase, semester, modules deepened
- Prerequisites — assumed knowledge
- Theory & research — papers, books, courses (with full citations)
- Curated tutorial list — the BYO-X entries preserved, language-tagged
- Recommended primary path — which external tutorial to follow first, and why
- Implementation milestones — 4–6 progressive stages with code sketches
- Tests & evidence — what proves each stage works
- Common pitfalls — known failure modes and how to recognise them
- Extensions — directions beyond the tutorial
- Module integration — how this strengthens specific course modules
- Portfolio framing — what to publish, what to keep private
Source
- Catalog:
codecrafters-io/build-your-own-x - Browsable mirror:
build-your-own-x.vercel.app - Books, paid tutorials, and 3rd-party guides for every project: see the Build Your Own X — Books & Paid Resources guide in the portal.