Skip to main content

Subspaces Organize Allowed Moves and Solution Sets

What This Concept Is​

A vector space is a collection of objects that can be added and scaled without leaving the collection. A subspace of $\mathbb{R}^n$ is a subset that still obeys those same closure rules and still contains the zero vector. Formally, $W \subseteq \mathbb{R}^n$ is a subspace iff $0 \in W$, $u + v \in W$ whenever $u, v \in W$, and $cu \in W$ whenever $u \in W$ and $c \in \mathbb{R}$.

This is not an abstract definition that matters only on a final exam. It is the operational signature of linear structure. Any time a set of candidate solutions is closed under addition and scaling, you can reason about it using the full machinery of linear algebra. Any time closure fails, you have to step outside linear tools (into affine, convex, or manifold methods).

A subspace is also the minimal data required to describe a linear model of a phenomenon. "Features live in $\mathbb{R}^n$ but genuine variation lies in a $k$-dimensional subspace" is the one-sentence summary of every dimensionality-reduction technique in Cluster 5 and of PCA in Phase 7. Once you commit to "the signal lives in this subspace," the geometry of projection, approximation, and error decomposition takes over.

For this module, the important shift is conceptual: solution sets of homogeneous systems $Ax = 0$ form subspaces; spans of vectors form subspaces; column spaces and nullspaces are subspaces; orthogonal complements are subspaces. Subspaces are where linear structure lives. The non-subspace analogs -- solution sets of $Ax = b$ when $b \ne 0$, affine hulls, convex hulls -- are where linear structure starts but then gets shifted off the origin.

Formally: $W$ is a subspace of $\mathbb{R}^n$ iff for all $u, v \in W$ and all $\alpha, \beta \in \mathbb{R}$, $\alpha u + \beta v \in W$. This single "closed under linear combinations" condition is equivalent to the three bullet-point conditions above and is often the easiest single test to apply.

Distinguish subspaces from their neighbors. A linear subspace must contain $0$ and be closed under addition and scaling. An affine subspace is a translated subspace, $v_0 + W$, which need not contain $0$. A convex cone is closed under addition and non-negative scaling (used in optimization). A convex set is closed only under convex combinations. A submanifold is a locally linear thing but not globally. All of these show up later; right now the one that matters is the strict linear kind. Note that every linear subspace is also affine, convex, and a submanifold; the inclusions are strict in the other direction.

A subspace in $\mathbb{R}^n$ is always one of these: ${0}$, a line through the origin, a plane through the origin, a hyperplane through the origin, or all of $\mathbb{R}^n$. Every subspace has a dimension (Cluster 2 concept 6) equal to the minimum number of vectors needed to span it.

Two subspace operations matter throughout this module. The intersection $U \cap V = {x : x \in U \text{ and } x \in V}$ is always a subspace (closure and zero-containment inherit from both sides). The sum $U + V = {u + v : u \in U, v \in V}$ is also always a subspace, and it is the smallest subspace containing both. When $U \cap V = {0}$, the sum is called direct and written $U \oplus V$; every vector in $U \oplus V$ has a unique decomposition $u + v$. Direct sums are the algebraic content of the phrase "independent subspaces"; orthogonal complements (Cluster 3) are the canonical example.

Why It Matters Here​

Without subspace thinking, later concepts look unrelated. Basis seems like a bookkeeping trick. Nullspace seems like a special-case solve. Orthogonal complements seem mysterious. Eigenvectors look isolated. Subspaces unify all of these. They tell you what directions are available, what directions are forbidden, and what directions survive a transformation.

Operationally, subspace thinking lets you answer "what is this solution set?" in a single word (a line, a plane, the full space) rather than a paragraph. It also gives you the right mental model for infinite solution sets: they are not "lots of answers" but a single geometric object -- a subspace or an affine shift of one.

The distinction between "linear structure holds" and "linear structure is broken" is also the engineering boundary between when you can apply linear-algebra tools (elimination, projection, SVD, regression) and when you must switch to more expensive methods (convex programming, iterative simulation, manifold learning). Subspaces are the preflight check: if closure holds, the cheap toolkit works.

Concrete Examples​

Example 1 -- a plane through the origin. Consider

$$S = {(x, y, z) \in \mathbb{R}^3 : x + y + z = 0}.$$

Check closure. The zero vector $(0, 0, 0)$ satisfies the equation, so $0 \in S$. If $u = (u_1, u_2, u_3)$ and $v = (v_1, v_2, v_3)$ are in $S$, then $u + v$ satisfies $(u_1+v_1) + (u_2+v_2) + (u_3+v_3) = 0 + 0 = 0$, so $u + v \in S$. Scaling similarly. Therefore $S$ is a subspace; geometrically, it is a 2-dimensional plane through the origin with normal $(1, 1, 1)$.

Example 2 -- a shifted plane is not a subspace. Consider

$$T = {(x, y, z) \in \mathbb{R}^3 : x + y + z = 1}.$$

$0 \notin T$ (the left side is $0$, not $1$). Also, if $u, v \in T$, then $u + v$ has coordinate sum $2 \ne 1$, so closure under addition fails too. $T$ is an affine subspace, not a linear one. It equals $(1, 0, 0) + S$ where $S$ is the plane from Example 1. This distinction is the difference between the solution sets of $Ax = 0$ (always a subspace) and $Ax = b$ (an affine shift when nonempty).

Example 3 -- the span of vectors is a subspace. Let

$$v_1 = \begin{pmatrix} 1 \ 0 \ 2 \end{pmatrix}, \quad v_2 = \begin{pmatrix} 0 \ 1 \ -1 \end{pmatrix}.$$

Their span is ${c_1 v_1 + c_2 v_2 : c_1, c_2 \in \mathbb{R}}$. Closure under addition and scaling is automatic from the definition, and $0$ is the span with $c_1 = c_2 = 0$. This span is a 2-plane through the origin -- a subspace that happens to equal the nullspace of the matrix $A = \begin{pmatrix} 2 & -1 & -1 \end{pmatrix}$ (verify $A v_1 = 0$, $A v_2 = 0$).

Example 4 -- sum and intersection. In $\mathbb{R}^3$, let $U$ be the $xy$-plane and $V$ be the $yz$-plane. Then

$$U \cap V = {(0, y, 0) : y \in \mathbb{R}} = \text{the } y\text{-axis},$$

a 1-dimensional subspace. And $U + V = \mathbb{R}^3$, because any vector $(a, b, c)$ decomposes as $(a, b - t, 0) + (0, t, c)$ for any choice of $t$. The sum is not direct: the decomposition is not unique. The dimension identity $\dim(U + V) = \dim U + \dim V - \dim(U \cap V)$ gives $3 = 2 + 2 - 1$. Direct sums are the case when $\dim(U \cap V) = 0$; then decompositions are unique and the identity collapses to $\dim(U \oplus V) = \dim U + \dim V$.

Common Confusion / Misconceptions​

"Any line or plane is a subspace." False. A line or plane must pass through the origin to be a subspace. In $\mathbb{R}^2$, the line $y = 2x$ is a subspace; the line $y = 2x + 1$ is not.

"I only need to check one closure property." Three conditions: $0 \in W$, closed under addition, closed under scalar multiplication. Scalar multiplication by $0$ gives the $0 \in W$ condition as a consequence of the other two for nonempty $W$, so practitioners often check only addition and scalar multiplication -- but include the "nonempty" check or the explicit $0 \in W$ test.

"Union of two subspaces is a subspace." Generally false. The union of the $x$-axis and the $y$-axis in $\mathbb{R}^2$ contains $(1, 0)$ and $(0, 1)$ but not their sum $(1, 1)$. The correct closed operation on subspaces is the sum $U + V = {u + v : u \in U, v \in V}$, which is always a subspace.

"Subspaces are an abstract mathematician's toy." Every nullspace, column space, row space, left nullspace, eigenspace, and orthogonal complement in this module is a subspace. The concept is the cheapest vocabulary available for talking about solution sets.

"Checking $0 \in W$ plus closure under addition is enough." You also need closure under scalar multiplication. In $\mathbb{R}$ (one dimension), addition closure alone can give you the integers -- closed under $+$, contain $0$, but not under multiplication by $\tfrac{1}{2}$ so not a subspace of $\mathbb{R}$. Always verify all three conditions (with the standard "$0 \in W$" replaced by "$W$ nonempty" if you prefer).

How To Use It​

When someone hands you a candidate subspace, test it mechanically:

  1. Does it contain $0$?
  2. If $u$ and $v$ are in it, is $u + v$ also in it?
  3. If $u$ is in it and $c$ is any scalar, is $cu$ also in it?
  4. If any test fails, produce a specific counterexample; that is the cleanest refutation.
  5. If all pass, name the subspace by a generating set (a span) or by the homogeneous equations that cut it out.

Two strong signals that a set is a subspace: it is defined by homogeneous linear equations (no constant term), or it is defined as a span of vectors. Both are automatic subspaces. Two strong signals it is not: it involves an inequality, or it involves an equation with a nonzero constant. A third non-subspace signal: nonlinear combinations (products of coordinates, roots, trig), which break homogeneity of scaling.

Use subspace language whenever you describe all solutions to $Ax = 0$, all linear combinations of given vectors, all vectors orthogonal to a given set, or all vectors fixed by an operator.

A practical habit: when working with a constrained optimization problem, separate linear equality constraints (whose feasible set is an affine subspace) from inequalities and nonlinearities (convex or nonconvex). The linear-equality part can be eliminated by parameterizing the feasible subspace; the remaining problem is reduced in dimension by exactly $\dim$(constraint row space). This is the fastest "free win" in numerical optimization, and it is unlocked purely by subspace thinking.

Transfer / Where This Shows Up Later​

  • S2 (algorithms): The set of valid states reachable from a linear dynamic program forms a span; cycle spaces of graphs form subspaces of the edge space $\mathbb{Z}_2^E$.
  • S4 (computer organization): Memory address spaces are not subspaces (no closure under addition of arbitrary pointers), but cache-aligned strides form sublattices, a discrete analog.
  • S5 (queueing & Markov chains): The set of stationary distributions of a Markov chain is an affine subspace of the probability simplex.
  • S7 (architecture): The set of allowed module-interaction patterns is frequently described as a subspace of a coupling matrix space; architectural rules often reduce to "this configuration must lie in the kernel of forbidden patterns."
  • S8 (ranking): Embedding subspaces in collaborative filtering are literally low-dimensional linear subspaces of the user-item rating space.
  • Phase 7 (ML): Weight spaces of linear models are subspaces. Low-rank factorizations restrict to subspaces. The "tangent space" of a manifold used in geometric deep learning is a linear subspace of the ambient space at each point. LoRA fine-tuning parameterizes updates $\Delta W = BA$ with low rank, which literally constrains the update to a low-dimensional subspace of weight space.
  • Across phases: Whenever the notion of "valid directions" or "allowed perturbations" arises, ask whether they form a subspace. If yes, every linear-algebra tool applies. If no, you need convex, affine, or differential-geometry tools -- a strictly bigger toolkit.

Check Yourself​

  1. Why is every nullspace a subspace? Prove it from the definition of $Ax$.
  2. Why does a translated plane usually fail to be a subspace?
  3. What is the difference between a span and an arbitrary set of vectors?
  4. Why is the intersection of two subspaces a subspace but the union generally not?
  5. What is the smallest subspace of $\mathbb{R}^n$? The largest?
  6. How do you recognize a subspace from the equations that define it?
  7. Why is the dimension identity $\dim(U + V) = \dim U + \dim V - \dim(U \cap V)$ the "inclusion-exclusion" for subspaces?
  8. What is the difference between an affine subspace $v_0 + W$ and the linear subspace $W$ it is translated from? Which one is linearly relevant?

Mini Drill or Application​

For each set, decide whether it is a subspace of $\mathbb{R}^2$ or $\mathbb{R}^3$:

  1. ${(x, y) \in \mathbb{R}^2 : y = 2x}$
  2. ${(x, y) \in \mathbb{R}^2 : y = 2x + 1}$
  3. ${(x, y, z) \in \mathbb{R}^3 : x - z = 0}$
  4. ${(x, y, z) \in \mathbb{R}^3 : x^2 + y^2 + z^2 \le 1}$
  5. ${(x, y) \in \mathbb{R}^2 : xy = 0}$

For each answer, justify using closure language with a specific counterexample when applicable, not intuition alone. Then in NumPy pick a random vector in each set and verify that $2v$ and $v + w$ land in (or leave) the set, as your argument predicts.

  1. Prove: the intersection of two subspaces $U, V \subseteq \mathbb{R}^n$ is a subspace. Then construct two subspaces in $\mathbb{R}^3$ whose union is not a subspace, and produce a specific pair of vectors whose sum witnesses the failure.
  2. For $U = \text{span}{(1, 0, 0)}$ and $V = \text{span}{(0, 1, 0), (0, 0, 1)}$ in $\mathbb{R}^3$, describe $U + V$, verify $U \cap V = {0}$, and conclude that $\mathbb{R}^3 = U \oplus V$.
  3. Show that the solution set of $Ax = b$ is an affine subspace whenever it is nonempty. Identify the subspace it is a translation of, and give the particular-plus-homogeneous formula.
  4. Describe the set ${A \in \mathbb{R}^{n \times n} : \text{tr}(A) = 0}$ as a subspace of the matrix space $\mathbb{R}^{n \times n}$. What is its dimension? What is the orthogonal complement under the Frobenius inner product?

Read This Only If Stuck​