Eigenvalues and Eigenvectors Reveal Invariant Action
What This Concept Is
Most vectors are rotated, sheared, or otherwise distorted by a matrix $A$. Eigenvectors are the exceptional directions that survive $A$ with only a rescaling: there exists a scalar $\lambda$ such that
$$A v = \lambda v, \quad v \ne 0.$$
The scalar $\lambda$ is the eigenvalue; $v$ is its eigenvector. The pair $(\lambda, v)$ is an invariant direction for the action of $A$. Geometrically, the line through $v$ is mapped to itself by $A$.
Finding eigenvalues uses the characteristic polynomial: $\lambda$ is an eigenvalue iff $(A - \lambda I)$ is singular, i.e., $\det(A - \lambda I) = 0$. This yields an $n$-th degree polynomial in $\lambda$ whose roots are the eigenvalues. For each root, the eigenspace $E_\lambda = \text{Null}(A - \lambda I)$ contains all eigenvectors for that eigenvalue together with the zero vector; its dimension is the geometric multiplicity. The algebraic multiplicity is the root's multiplicity in the characteristic polynomial; always $\text{geom mult} \le \text{alg mult}$, and equality for every eigenvalue, together with a characteristic polynomial that splits over the chosen field, characterizes diagonalizability (Cluster 4 concept 12).
Distinguish eigenvalues from related spectral quantities:
- Eigenvalue $\lambda$: can be complex, can be zero, can be negative; ties to characteristic polynomial.
- Singular value $\sigma_i$: always real and $\ge 0$; square roots of eigenvalues of $A^TA$; defined for all matrices including rectangular ones.
- Spectral radius $\rho(A)$: the largest $|\lambda|$; governs long-run growth of $|A^k|$.
Two traces-and-dets identities worth burning in:
$$\text{tr}(A) = \sum_i \lambda_i, \qquad \det(A) = \prod_i \lambda_i.$$
These are free consistency checks and the cheapest way to catch arithmetic mistakes.
This is the first serious form of spectral thinking: instead of tracking all coordinates directly, look for invariant directions and their growth, decay, or sign flip. Once you pivot to this viewpoint, repeated-action problems become bookkeeping on scalar growth rates rather than on full matrix products.
Why It Matters Here
Eigen-analysis explains: repeated matrix powers $A^k$ and their long-run behavior; stability of discrete-time dynamical systems ($|\lambda| < 1$ means decay, $|\lambda| > 1$ means growth); dominant directions in iterative processes (power iteration needs a dominant-magnitude gap and a nonzero starting component in that eigendirection); graph ranking and network flow behavior; positive-definite and SVD structure in Cluster 5.
Many CS applications care more about dominant modes than full coordinate detail. Eigenvectors provide those modes. PageRank, spectral clustering, vibration analysis, Google's search ranking, quantum state evolution, and the stability of numerical integrators all reduce to "find the right eigenvalues."
Concrete Examples
Example 1 -- diagonal matrix, eigenvectors are standard basis. For
$$A = \begin{pmatrix} 3 & 0 \ 0 & 1 \end{pmatrix},$$
the standard basis vectors are eigenvectors: $A e_1 = 3 e_1$ and $A e_2 = e_2$. Eigenvalues are $3$ and $1$. Repeated application stretches the $e_1$-direction $3^k$ times faster than the $e_2$-direction: $A^k (a e_1 + b e_2) = 3^k a,e_1 + b,e_2$. For large $k$, the $e_1$ component dominates unless the initial vector has no component there.
Example 2 -- symmetric $2\times 2$. For
$$A = \begin{pmatrix} 2 & 1 \ 1 & 2 \end{pmatrix},$$
the characteristic polynomial is $(2-\lambda)^2 - 1 = \lambda^2 - 4\lambda + 3 = (\lambda - 1)(\lambda - 3)$. Eigenvalues are $\lambda_1 = 3$ and $\lambda_2 = 1$. Solve $(A - 3I)v = 0$: $-v_1 + v_2 = 0$, so $v_1 = (1, 1)^T$. Solve $(A - I)v = 0$: $v_1 + v_2 = 0$, so $v_2 = (1, -1)^T$. Trace check: $1 + 3 = 4 = 2 + 2$. Determinant check: $1 \cdot 3 = 3 = 4 - 1$. Under repeated multiplication, the $(1,1)$ direction grows $3^k$; the $(1,-1)$ direction stays at $1^k = 1$. Normalized iterates align with the line through $(1,1)$ only when the initial vector has a nonzero component in that direction. Starting at $(1,-1)$ stays in the other eigenspace.
Example 3 -- defective matrix with repeated eigenvalue. For
$$A = \begin{pmatrix} 4 & 1 \ 0 & 4 \end{pmatrix},$$
the characteristic polynomial is $(\lambda - 4)^2$, so $\lambda = 4$ with algebraic multiplicity 2. But solving $(A - 4I)v = 0$ gives only $v = (1, 0)^T$ (and scalar multiples). Geometric multiplicity is 1, less than algebraic 2. $A$ is defective -- no basis of eigenvectors exists. This is the Jordan-block world (covered in Appendix B of Strang); such matrices cannot be diagonalized, though they can be put in Jordan form.
Common Confusion / Misconceptions
"Eigenvectors are the basis vectors after diagonalization." Not exactly. They are specific invariant directions of the original transformation. If enough independent ones exist, they form a basis and diagonalize the matrix; if not (defective case), no diagonalizing basis exists.
"Find eigenvectors by guessing from a picture." Unreliable beyond $2 \times 2$. Always solve $(A - \lambda I)v = 0$ after finding $\lambda$ from the characteristic polynomial (or via np.linalg.eig).
"Eigenvalues are always real." Only guaranteed for symmetric (or Hermitian) matrices. A planar rotation through an angle not divisible by $\pi$ has nonreal eigenvalues $e^{\pm i\theta}$ -- real eigenvectors would contradict rotation's lack of invariant real directions (except multiples of rotation axes in 3D).
"Eigenvalue zero means the matrix is broken." It means the matrix is singular (has a nontrivial nullspace). This is structurally important but not pathological -- it happens whenever the transformation collapses at least one direction to zero, e.g., a projection onto a subspace.
"Eigenvalues equal singular values." True only for symmetric positive semidefinite matrices. For a general matrix, singular values equal $\sqrt{\text{eig}(A^TA)}$, and the two sets can be completely different.
How To Use It
To find eigenvalues and eigenvectors:
- Write the characteristic polynomial $p(\lambda) = \det(A - \lambda I)$ and find its roots. Explicit characteristic-polynomial root finding is mainly a small symbolic exercise. Numerical eigensolvers use dedicated matrix algorithms; do not infer a safe numerical cutoff solely from n.
- For each eigenvalue $\lambda$, solve $(A - \lambda I)v = 0$ to obtain the eigenspace basis.
- Classify: simple (alg mult $= 1$), semisimple (alg $=$ geom), defective (alg $>$ geom).
- Interpret what each eigenvalue does:
- $|\lambda| > 1$: expansion under iteration;
- $|\lambda| < 1$: decay;
- $\lambda < 0$: sign flip;
- $\lambda$ complex: rotation with scaling;
- $\lambda = 0$: collapse along that direction.
- Always connect the result back to repeated action or stability before claiming you understand.
For numerical work, use an eigensolver whose documented matrix class matches the input. Validate the symmetry assumption before using a symmetric solver, and check normalized eigenpair residuals. Sparse methods return selected modes under additional convergence assumptions.
Transfer to the calibration project
Use this concept to justify a result or reject an assumption in the calibration project. Name the matrix shape, objective, scalar field and available evidence before interpreting the result.
Check Yourself
- Why are eigenvectors important for repeated powers of $A$?
- What does an eigenvalue of $0$ tell you?
- Why can a matrix have repeated eigenvalues but too few eigenvectors (defective)?
- Why is $\text{tr}(A) = \sum \lambda_i$ and $\det(A) = \prod \lambda_i$, and how can you use these as cheap consistency checks?
- Why are symmetric-matrix eigenvalues always real?
- How does a complex eigenvalue pair $a \pm bi$ show up geometrically in $\mathbb{R}^2$?
Mini Drill or Application
- Find eigenvalues and eigenvectors of $\begin{pmatrix} 2 & 1 \ 1 & 2 \end{pmatrix}$. Verify trace and determinant checks.
- Decide which direction dominates under repeated multiplication by this matrix. Compute $A^{10} v$ for $v = (1, 0)^T$ numerically and confirm it aligns with the dominant eigenvector.
- In NumPy:
A = np.array([[2,1],[1,2]]); w, V = np.linalg.eig(A); print(w); print(V). Verify $A V = V \text{diag}(w)$. - Find the eigenvalues of the rotation matrix $R_{\pi/3} = \begin{pmatrix} 1/2 & -\sqrt{3}/2 \ \sqrt{3}/2 & 1/2 \end{pmatrix}$. Confirm they are complex conjugates on the unit circle. What real-valued invariant subspace survives rotation?
- Construct a defective $2\times 2$ matrix of your choice. Confirm its eigenvalue decomposition fails by trying
np.linalg.eig; examine what the returned eigenvectors look like.
Read This Only If Stuck
Use the bounded reading guide and the Strang book index. The book link opens an index; find the named topic in your edition, read a small section and reconstruct one argument with the book closed.
An eigenpair residual does not prove dominance
For diag(3,1), starting power iteration at (0,1) stays there. The pair lambda=1, v=(0,1) has exactly zero residual even though the dominant eigenvalue has magnitude three. A starting component, spectral gap and the matrix class are part of a convergence statement, not optional details.
For diag(-3,1), signs may alternate while the dominant line converges. Test ||Av−lambda*v||, with a named scale, instead of requiring equal successive vectors. Repeated eigenvalues permit many valid vectors within an eigenspace. Transfer: P11 and the missed-dominant fixture in Lab 3.
After an independent attempt, use practice feedback and model contracts to check your reasoning.