Module 4: Linear Algebra for CS

Primary text: Linear Algebra and Its Applications
Selective support: local semester chunks from Mathematics for Computer Science only where recurrence thinking sharpens spectral intuition

This guide is the primary teacher. You do not need to read the source books front-to-back to complete this module. You do need to become operationally strong at elimination, subspaces, rank, basis, orthogonality, least squares, determinants, eigen-analysis, positive definiteness, SVD, and the numerical judgment required to use linear algebra responsibly in CS work.

---

Scope of This Module

This module is not "matrix arithmetic week." It is where you learn to treat linear structure as a computational object.

What it covers in depth:

• linear systems, elimination, pivots, free variables, and matrix factorization
• matrices as representations of linear transformations rather than symbol tables
• subspaces, column space, nullspace, rank, basis, and dimension
• orthogonality, projections, least squares, and QR as the language of approximation
• determinants as structural signals of scaling, orientation, and singularity
• eigenvalues, eigenvectors, diagonalization, and repeated action
• positive definite matrices, quadratic forms, and optimization geometry
• singular value decomposition, low-rank structure, and compression
• norms, conditioning, iterative methods, and method choice for real computation

What it deliberately does not try to finish here:

• full abstract linear algebra beyond the finite-dimensional computational core
• Jordan form, full spectral theory, and advanced operator theory
• a complete numerical linear algebra course
• PDE, finite element, or linear programming depth beyond brief pointers

This is an in-depth foundation module. If you can multiply matrices but cannot explain what information a matrix preserves, destroys, approximates, or amplifies, you are not done.

---

Before You Start

Answer these closed-book before starting the main path:

1. Why do row operations preserve a solution set even though they change the written equations?
2. What is the difference between "a set of vectors" and "the span of that set"?
3. Why does a matrix with more columns than pivots force some nonzero vector into the nullspace?
4. Why does orthogonality make approximation cleaner than arbitrary coordinates do?
5. Why might an invertible matrix still be numerically dangerous?

Diagnostic Interpretation

4-5 solid answers

• You are ready for the full path.

2-3 solid answers

• Continue, but expect extra time in the subspace and orthogonality clusters.

0-1 solid answers

• Revisit elimination basics and coordinate geometry before continuing. Module 4 assumes you can already read systems and vectors without panic.

---

What This Module Is For

Linear algebra is the mathematics of structure-preserving change. In CS, that matters whenever you need to answer questions like:

• what outputs can this system produce and what information does it throw away?
• how do I compress, rotate, project, or approximate high-dimensional data?
• what directions dominate repeated updates or iterative algorithms?
• when does a model fit exactly, approximately, or unstably?
• what basis, factorization, or decomposition reveals the real structure fastest?

This module builds the linear algebra needed for:

• graphics transformations and camera geometry
• regression, embeddings, and dimensionality reduction
• ranking, graph methods, and iterative updates
• optimization and quadratic objectives
• signal processing, approximation, and compression
• numerical reasoning about large matrix systems

You are learning to reason about linear structure as both mathematics and computation.

---

Concept Map

---

How To Use This Module

Work in order. The later clusters assume the earlier structural habits are stable.

Cluster 1: Linear Systems and Matrix Mechanics

Order	Concept	Type	Focus
1	Elimination Turns Geometry into an Algorithm	PRIMARY	Equivalence of systems, pivots, consistency, and free variables
2	Matrix Multiplication and Factorization Encode Composition	SUPPORTING	Composition, order, and reusable factorization structure
3	Inverses and Transposes Are Structural Tools	SUPPORTING	Undoing actions, transpose logic, and structural tests

Cluster mastery check: Can you look at a reduced matrix and say what the system can solve, how many freedoms remain, and why?

Cluster 2: Vector Spaces, Rank, and Basis

Order	Concept	Type	Focus
4	Subspaces Organize Allowed Moves and Solution Sets	PRIMARY	Closure, homogeneous systems, and linear structure
5	Column Space, Nullspace, and Rank Explain What a Matrix Can Produce	PRIMARY	Reachable outputs, lost inputs, and rank-nullity
6	Independence, Basis, and Dimension Choose Coordinates Without Waste	PRIMARY	Redundancy removal and coordinate design

Cluster mastery check: Can you describe a matrix in terms of its spaces, not only its entries?

Cluster 3: Orthogonality, Projections, and Approximation

Order	Concept	Type	Focus
7	Linear Transformations Are the Real Object and Matrices Are Representations	PRIMARY	Structure-preserving maps and basis-dependent matrices
8	Orthogonality Creates Clean Separation	SUPPORTING	Dot products, orthogonal complements, and clean decomposition
9	Projections, Least Squares, and QR Turn Inconsistency into Best Approximation	PRIMARY	Approximation geometry, regression, and orthonormal factorization

Cluster mastery check: Can you explain why least squares is a projection problem rather than a disguised exact solve?

Cluster 4: Spectral Thinking and Repeated Action

Order	Concept	Type	Focus
10	Determinants Measure Scaling, Orientation, and Singularity	SUPPORTING	Volume scaling, orientation, and invertibility signals
11	Eigenvalues and Eigenvectors Reveal Invariant Action	PRIMARY	Invariant directions and dominant modes
12	Diagonalization and Similarity Make Repeated Action Computable	PRIMARY	Basis change, powers, and recurrence behavior

Cluster mastery check: Can you explain long-run behavior of repeated matrix action without multiplying blindly?

Cluster 5: Positive Definiteness, SVD, and Numerical Judgment

Order	Concept	Type	Focus
13	Positive Definite Matrices Turn Quadratic Forms into Geometry	PRIMARY	Energy, curvature, and optimization structure
14	Singular Value Decomposition Is the Universal Factorization	PRIMARY	Rank, compression, and dominant directions
15	Norms, Conditioning, Iterative Methods, and Method Choice	SUPPORTING	Stability, sensitivity, and computational judgment

Cluster mastery check: Can you tell the difference between an algebraically solvable problem and a numerically trustworthy one?

Then work these practice pages:

Order	Practice path	Focus
1	Elimination and Subspace Diagnostics Lab	Row reduction, subspace tests, rank, and nullspace reasoning
2	Basis, Rank, and Transformation Workshop	Basis design, transformation interpretation, and coordinate control
3	Orthogonality, Least Squares, and Spectral Clinic	Projection, approximation, eigen-analysis, and decomposition comparison
4	Code Katas	Algorithmic implementation and numerical habit formation

Use Module Quiz after the concept and practice path. Use Reference and Selective Reading, Book Exercise Lanes, and Learning Resources only for targeted reinforcement.

---

Learning Objectives

By the end of this module you should be able to:

1. Solve and classify linear systems using elimination, pivots, and factorization language.
2. Interpret matrices through column space, nullspace, rank, and information loss.
3. Determine independence, basis, and dimension for subspaces and model representations.
4. Represent linear transformations with matrices and explain how basis choice changes representation.
5. Use orthogonality, projection, least squares, and QR to solve approximation problems.
6. Interpret determinants as structural signals of scaling, orientation, and singularity.
7. Compute and interpret eigenvalues and eigenvectors for repeated action and stability questions.
8. Use diagonalization when possible and explain clearly when it fails.
9. Recognize positive definite structure and connect quadratic forms to optimization geometry.
10. Interpret SVD, conditioning, and iterative methods as tools for compression, stability, and computational method choice.

---

Outputs

• one elimination notebook with at least 12 fully worked systems and structural interpretation
• one subspace and basis sheet containing at least 8 rank/nullspace/basis problems
• one projection and least-squares sheet with at least 5 approximation problems
• one spectral notebook containing at least 6 eigenvalue or diagonalization problems
• one decomposition memo comparing LU, QR, eigendecomposition, and SVD by use case
• one code notebook implementing at least three of the code katas
• one mistake log naming at least 12 errors such as lost pivot, used reduced columns instead of original, confused exact solve with projection, assumed diagonalizable, or ignored conditioning
• one short memo connecting Module 4 tools to graphics, regression, ranking, or optimization

---

Completion Standard

You have completed Module 4 when you can:

• inspect a matrix and say what its pivots, rank, and nullspace imply before doing long arithmetic
• explain a transformation in terms of spaces and directions, not only entries
• use basis language to remove redundancy and choose coordinates deliberately
• justify least squares as projection and say when QR or SVD is the better computational path
• explain what eigenvectors and singular vectors are telling you operationally
• distinguish exact invertibility from numerical safety
• choose among elimination, LU, QR, eigendecomposition, SVD, or iterative methods with a reason
• connect the module to at least one concrete CS workflow without handwaving

---

Reading Policy

• Concept pages are the main path.
• Local book chunks are selective reinforcement, not a second syllabus.
• Read only if stuck means do the concept page, self-check, and drill first.
• Optional deep dive means more nuance or extra exercise lanes, not required progress.
• Because this is an in-depth module, written interpretation is required alongside calculations.

---

Suggested Weekly Flow

Day	Work
1	Concepts 1-3 and three full elimination/invertibility problems
2	Concepts 4-6 and one rank/nullspace/basis sheet
3	Concepts 7-9 and one projection or regression setup
4	Concepts 10-12 and one repeated-action or recurrence exercise
5	Concepts 13-15 and one method-choice memo on SVD versus eigen versus QR
6	Practice pages 1-2 and targeted book reinforcement
7	Practice pages 3-4, quiz, and mistake-log cleanup

---

Reference

If you need exact links into the local chunked books, use Reference and Selective Reading.

---

Build Your Own X — elective

Linear algebra becomes concrete when you use it. Three substantial projects in the Foundations BYO catalog exercise it intensively: the Neural Network (matmul, gradients), the LLM (attention is softmax(QK^T/√d)V), and the 3D Renderer (vectors, dot/cross products on every ray). See Build Your Own X overview.

Rich Learning Pages

Worked Examples | Guided Labs | Case Studies | Mistake Clinic | Reading Guide | Capstone Thread