Skip to main content

Orthogonality, Least Squares, and Spectral Clinic

Retrieval Prompts​

  1. State the definition of orthogonality using a dot product.
  2. State the projection idea in one sentence.
  3. State the normal equations from memory.
  4. State the eigenvalue equation.
  5. State what diagonalization buys you computationally.

Compare and Distinguish​

Separate these pairs clearly:

  • exact solve versus least-squares solve
  • orthogonal basis versus orthonormal basis
  • eigenvalue versus singular value
  • diagonalizable matrix versus arbitrary matrix with eigenvalues

Common Mistake Check​

Find the error in each statement:

  1. "Least squares finds an exact solution that the original system hid."
  2. "Orthogonal vectors are only a two-dimensional picture idea."
  3. "Every matrix with repeated eigenvalues is diagonalizable."
  4. "Singular values are just the eigenvalues written differently."

Mini Application​

Do one problem from each lane:

  1. project a vector onto a line or plane
  2. set up a least-squares fit for a small data set
  3. compute eigenvalues and eigenvectors of a 2 x 2 matrix
  4. explain when diagonalization would let you compute A^20 quickly

Then write one paragraph comparing eigen-decomposition and SVD operationally.

Evidence Check​

This page is complete only if you can explain, without notes, why:

  • projection error is orthogonal
  • least squares is approximation, not rescue
  • eigenvectors matter for repeated action
  • SVD still works when eigenvector methods are not the right tool