Build Your Own 3D Renderer (Ray Tracer)

"Ray tracing produces beautiful images by simulating the physics of light. Rasterization produces beautiful images by being absurdly fast. You should understand both."

Building a ray tracer is the most rewarding way to put Sem 1 linear algebra and statistics to work. By the end you have photorealistic-ish images of spheres, planes, and triangles, generated by code you wrote, all explained by Sem 1 vector math and Monte Carlo integration.

This is the substantial math-heavy companion to the Neural Network tutorial — same Foundations content (linear algebra + probability), different application domain (3D graphics instead of ML).

1. Overview & motivation

A ray tracer is conceptually simple:

for each pixel:
    cast ray from camera through pixel into scene
    find nearest object hit by ray
    compute color based on material + lighting
    write pixel

That's it. Each next step (shadows, reflections, refractions, antialiasing, soft lighting) makes the image more realistic without changing the overall structure.

What you can only learn by building one:

Why vectors and dot products are the workhorses of 3D graphics.
Why floating-point precision matters in computer graphics — and how to recognize when it's biting you.
Why Monte Carlo ray tracing (path tracing) is the unifying framework behind every modern offline renderer (Pixar's RenderMan, Disney's Hyperion, V-Ray).
Why rasterization and ray tracing are dual approaches — same goal, opposite traversal order.

2. Where this fits in the degree

Phase: Foundations
Semester: 1 (Math Foundations) and 2 (Algorithms)
Modules deepened:
- Sem 1 Module 4 (linear algebra) — intensively. Every ray cast is a sequence of vector operations.
- Sem 1 Module 5 (statistics / probability) — Monte Carlo integration is the heart of path tracing.
- Sem 2 Module 5 (advanced structures) — BVH acceleration in Book 2.

Cross-phase relevance:

Pairs with the Physics Engine tutorial — shared linear-algebra infrastructure.
Performance work in Books 2–3 looks ahead to Sem 8 Module 4 (Production phase performance discipline).

3. Prerequisites

Strong linear algebra: vectors, dot/cross products, normalization. Sem 1 Module 4 is sufficient.
Comfort with C++, Rust, or Python (NumPy).
Patience for runtime — ray tracing is slow until optimized.

You do not need a graphics course. Shirley's books teach everything you need from first principles.

4. Theory & research

Required reading

Peter Shirley, Ray Tracing in One Weekend (free book series) — raytracing.github.io. Three books: Weekend, Next Week, The Rest of Your Life. Each ~50 pages. C++ codebase. ⭐ recommended primary.
Scratchapixel (scratchapixel.com) — free CG textbook. Excellent first-principles derivations.

Strongly recommended

Matt Pharr, Wenzel Jakob, Greg Humphreys, Physically Based Rendering: From Theory To Implementation (PBRT) — the canonical book on physically-based rendering. Free online (3rd edition). 1,200 pages. Industrial. Read after Shirley.
Tom Hammersley's "Smallpt" — Global Illumination in 99 lines — kevinbeason.com/smallpt/. The famous one-screen path tracer. Read after Shirley.
Veach, "Robust Monte Carlo Methods for Light Transport Simulation" (PhD, 1997) — foundational thesis. Free PDF. Long but rewarding.

For rasterization (the other approach)

Fabian "ryg" Giesen, "A trip through the Graphics Pipeline 2011" — fgiesen.wordpress.com — 13-part series. The gold standard of modern GPU pipeline explanation.

5. Curated tutorial list (from BYO-X)

C++: Introduction to Ray Tracing: a Simple Method for Creating 3D Images
C++: How OpenGL works: software rendering in 500 lines of code — github.com/ssloy/tinyrenderer ⭐ excellent for rasterization path
C++: Raycasting engine of Wolfenstein 3D — historical, 2.5D
C++: Physically Based Rendering: From Theory To Implementation — PBRT book — the comprehensive reference
C++: Ray Tracing in One Weekend — Peter Shirley ⭐ recommended primary
C++: Rasterization: a Practical Implementation — Scratchapixel
C# / TypeScript / JavaScript: Learning how to write a 3D soft engine from scratch in C#, TypeScript or JavaScript
Java / JavaScript: Build your own 3D renderer
Java: How to create your own simple 3D render engine in pure Java
JavaScript / Pseudocode: Computer Graphics from scratch — Gabriel Gambetta ⭐ accessible alternative
Python: A 3D Modeller

6. Recommended primary path

Peter Shirley, Ray Tracing in One Weekend (Book 1).

Twelve sections, ~50 pages. Builds a path tracer in C++ producing a PPM image.

Coverage:

Output an image.
The vec3 class.
Rays, simple camera, background.
Adding a sphere.
Surface normals and multiple objects.
Antialiasing.
Diffuse materials.
Metal.
Dielectrics.
Positionable camera.
Defocus blur.
Where next?

After Book 1 (1 weekend): Book 2 (Next Week) adds motion blur, BVH, textures, perlin noise, quads, lights. Book 3 (The Rest of Your Life) adds Monte Carlo, importance sampling.

For rasterization: ssloy/tinyrenderer is the equivalent canonical tutorial — 5 lessons that build a software OpenGL.

For this degree: Ray Tracing in One Weekend → Next Week → Rest of Your Life.

7. Implementation milestones

Milestone 1: Vec3 class and PPM output

class vec3 {
public:
    double e[3];
    vec3() : e{0,0,0} {}
    vec3(double x, double y, double z) : e{x,y,z} {}
    double x() const { return e[0]; }
    double y() const { return e[1]; }
    double z() const { return e[2]; }
    vec3 operator+(const vec3& v) const { return {e[0]+v.e[0], e[1]+v.e[1], e[2]+v.e[2]}; }
    // ... etc
};

inline double dot(const vec3& u, const vec3& v) {
    return u.e[0]*v.e[0] + u.e[1]*v.e[1] + u.e[2]*v.e[2];
}
inline vec3 cross(const vec3& u, const vec3& v) { ... }
inline vec3 unit_vector(const vec3& v) { return v / v.length(); }

Output a PPM:

std::cout << "P3\n" << W << " " << H << "\n255\n";
for (int j = H-1; j >= 0; --j) {
    for (int i = 0; i < W; ++i) {
        auto r = double(i)/(W-1), g = double(j)/(H-1), b = 0.0;
        write_color(std::cout, color(r, g, b));
    }
}

Evidence: A gradient image.

Milestone 2: Rays and a sphere

A ray is P(t) = origin + t * direction. Test ray-sphere intersection by solving the quadratic equation.

double hit_sphere(const point3& center, double radius, const ray& r) {
    vec3 oc = r.origin() - center;
    auto a = dot(r.direction(), r.direction());
    auto b = 2.0 * dot(oc, r.direction());
    auto c = dot(oc, oc) - radius*radius;
    auto disc = b*b - 4*a*c;
    if (disc < 0) return -1.0;
    return (-b - sqrt(disc)) / (2.0*a);
}

color ray_color(const ray& r) {
    auto t = hit_sphere(point3(0,0,-1), 0.5, r);
    if (t > 0.0) {
        vec3 N = unit_vector(r.at(t) - vec3(0,0,-1));
        return 0.5 * color(N.x()+1, N.y()+1, N.z()+1);
    }
    vec3 unit_direction = unit_vector(r.direction());
    t = 0.5 * (unit_direction.y() + 1.0);
    return (1.0-t)*color(1,1,1) + t*color(0.5, 0.7, 1.0);
}

Evidence: A red sphere on a blue gradient sky. The sphere is colored by its surface normal.

Milestone 3: Hittable abstraction

Refactor: an abstract hittable interface; a hittable_list containing many objects; ray_color finds the nearest intersection across all.

Evidence: Multiple spheres, one large for the ground.

Milestone 4: Antialiasing

Sample N rays per pixel with random subpixel offsets. Average.

Evidence: Smooth edges; no more pixelated jaggies.

Milestone 5: Diffuse materials

A diffuse surface scatters in a random direction near the normal. Cast a secondary ray; recurse.

color ray_color(const ray& r, const hittable& world, int depth) {
    if (depth <= 0) return color(0,0,0);
    hit_record rec;
    if (world.hit(r, 0.001, infinity, rec)) {
        point3 target = rec.p + rec.normal + random_unit_vector();
        return 0.5 * ray_color(ray(rec.p, target - rec.p), world, depth-1);
    }
    // background gradient
}

Evidence: Spheres look like dull plastic, with realistic-feeling shadow falloff.

Milestone 6: Materials — metal, dielectric

Metal: reflection with possible fuzz.

vec3 reflected = reflect(unit_vector(r_in.direction()), rec.normal);
scattered = ray(rec.p, reflected + fuzz*random_in_unit_sphere());

Dielectric (glass): refraction using Snell's law, with Schlick's approximation for the reflection coefficient.

Evidence: A scene with metal, glass, and diffuse spheres, all looking distinct.

Milestone 7: Camera

A camera with position, look-at, up vector, vertical field of view, aspect ratio. Computes ray directions per pixel.

Bonus: defocus blur (depth of field) — sample rays from a small aperture instead of a single point.

Milestone 8 (Book 2): BVH (Bounding Volume Hierarchy)

Acceleration structure. Group nearby objects into bounding boxes; recurse. Ray-AABB test is cheap; an early miss skips entire subtrees.

A scene with 500 spheres goes from impossibly slow to fast.

Milestone 9 (Book 2): Textures

Replace solid colors with image lookups. Earth-texture sphere, etc.

Milestone 10 (Book 2): Lights and emissive materials

Materials that emit light. Cornell box scene.

Milestone 11 (Book 3): Importance sampling

Sample directions weighted by light contribution. Faster convergence, less noise.

Milestone 12 (optional): Rasterization comparison

Build the equivalent rasterizer (ssloy's tinyrenderer). Render the same triangle mesh with both techniques. Compare image quality, performance, and code complexity.

8. Tests & evidence

Test	How
Vec3 operations	Hand-verify dot, cross, norm
Sphere hit	A ray through center hits both surfaces; an off-axis ray hits one surface; a non-intersecting ray hits nothing
Reflection	Reflected ray's angle equals incidence angle
Refraction	Test against Snell's law for known indices
Performance	Render a 1080p image with 100 spp; record time
Quality	A Cornell box scene; compare against PBRT or Blender Cycles

The strongest evidence: a portfolio image — a high-quality render of an interesting scene. The image is the evidence.

9. Common pitfalls

Off-by-one on epsilon. Self-intersection: a scattered ray hits the surface it was scattered from. Use t_min = 0.001.
Diverging recursion. No depth limit → stack overflow. Cap depth.
Inconsistent vector normalization. Some operations require unit vectors; some don't. Be explicit.
Coordinate system confusion. Right-handed vs left-handed, Y-up vs Z-up. Pick one and document.
PPM endianness. Use P3 (ASCII) for debugging; P6 (binary) for production.
Gamma correction. Output in sRGB, not linear. Otherwise images look murky. sqrt(channel) is a cheap approximation.
Cosmic-grade noise. Path tracing converges slowly. 100+ samples per pixel for clean images.

10. Extensions

Path tracing. Already implicit in the diffuse milestone. Improve with importance sampling, next-event estimation.
Triangle meshes. Load OBJ files; render polygonal models.
Volumetric rendering. Fog, smoke, clouds.
Subsurface scattering. Skin, marble.
Photon mapping. Two-pass: emit photons from lights; gather at shading time.
Bidirectional path tracing. Trace from camera and from lights; connect.
GPU implementation. CUDA or compute shaders. Major speedup.
OptiX / Vulkan ray tracing. Use hardware ray-tracing units.

The path from "Shirley Book 1" to "Pixar's RenderMan" is multi-year. Choose your level.

11. Module integration

Module	What the renderer deepens
Sem 1 Module 4 — Linear algebra	Maximum exercise. Vector operations on every ray.
Sem 1 Module 5 — Statistics / probability	Monte Carlo integration is the heart of path tracing.
Sem 2 Module 5 — Advanced structures	BVH (Book 2) is a textbook spatial-tree structure.
Sem 8 Module 4 — Performance	Frame budget is real. Hot-loop optimization matters when you scale up.
Neural Network tutorial	Shared math (matrices, vectors). NN and ray tracer are the two big Sem-1-math-heavy projects.
Physics Engine tutorial	Shared math infrastructure.
Game Engine tutorial	2D engine focuses on rasterization; renderer here focuses on physically-based imaging. Contrast illuminates both.
Memory Allocator tutorial	Many objects per scene → arena allocators or specialized pools.

12. Portfolio framing

What to publish:

Source organized as vec/, geometry/ (sphere, plane, triangle, mesh), material/, camera/, acceleration/ (BVH).
A gallery of rendered images: Cornell box, glass sphere on diffuse plane, complex scene with multiple materials.
A README with:
- The hero image at the top.
- Performance numbers (image resolution, samples per pixel, render time).
- Algorithm references (which sections of Shirley / PBRT).
- Known limitations.

Reviewer entry points:

material/dielectric.cpp — glass material (most subtle).
geometry/sphere.cpp — ray-sphere intersection.
camera/camera.cpp — ray generation.
gallery/*.png — the portfolio images.

A working ray tracer is a striking portfolio piece because the output is literally art. A great Cornell-box render is something everyone with eyes can appreciate. Specific recommendation: render an animated frame sequence (camera orbit) and post the resulting video.

Source

This tutorial draws from the BYO-X catalog "3D Renderer" section. Peter Shirley's Ray Tracing in One Weekend series is the canonical primary path; PBRT is the definitive reference.

1. Overview & motivation​

2. Where this fits in the degree​

3. Prerequisites​

4. Theory & research​

Required reading​

Strongly recommended​

For rasterization (the other approach)​

5. Curated tutorial list (from BYO-X)​

6. Recommended primary path​

7. Implementation milestones​

Milestone 1: Vec3 class and PPM output​

Milestone 2: Rays and a sphere​

Milestone 3: Hittable abstraction​

Milestone 4: Antialiasing​

Milestone 5: Diffuse materials​

Milestone 6: Materials — metal, dielectric​

Milestone 7: Camera​

Milestone 8 (Book 2): BVH (Bounding Volume Hierarchy)​

Milestone 9 (Book 2): Textures​

Milestone 10 (Book 2): Lights and emissive materials​

Milestone 11 (Book 3): Importance sampling​

Milestone 12 (optional): Rasterization comparison​

8. Tests & evidence​

9. Common pitfalls​

10. Extensions​

11. Module integration​

12. Portfolio framing​

Source​