Back

SVD as Rotate-Scale-Rotate

Every 2x2 matrix does the same three things in sequence, no matter how tangled its entries look: it rotates, it stretches along two perpendicular axes, then it rotates again. That is the singular value decomposition $M = U\Sigma V^T$. Watch a unit circle ride through the three steps and come out an ellipse: first $V^T$ turns it, then $\Sigma$ stretches it by the singular values $\sigma_1 \ge \sigma_2$, then $U$ turns the result. The singular values are exactly the longest and shortest the matrix can stretch any unit vector, the semi-axes of the output ellipse. The diagnostic plots that stretch against the input direction, so the peak is $\sigma_1$ and the trough is $\sigma_2$.

Figure 1. Singular value decomposition of a 2x2 matrix. Top: the unit circle and a marked frame ride through V^T (rotate), Σ (scale), U (rotate) into the output ellipse, with the factorization shown alongside. Bottom: the stretch factor |M v| versus input angle, peaking at σ₁ and dipping to σ₂. Method: closed form via the eigenvalues of MᵀM.
a1.50
b-0.70
c0.40
d2.10

WHAT TO TRY

  • Set b = c = 0 and vary a, d: V^T and U do nothing, the circle just stretches along the axes and the singular values are |a| and |d|.
  • Make the rows nearly parallel (for example a=1.5, b=1.5, c=1, d=1): σ₂ shrinks toward zero, the ellipse flattens to a sliver, and the condition number blows up.
  • Set b = c (symmetric M): the two rotations match, V = U up to sign, so the input and output principal axes line up.
  • Watch the stretch curve below: its highest point is σ₁ and its lowest is σ₂, reached a quarter turn apart, at the input singular directions.