The proof needs the hypothesis (1−ε)A ⪯ B ⪯ (1+ε)A in a whitened form. This section derives that restatement.

The chain of rewrites

  1. Subtract A throughout 12:33, 13:05: −εA ⪯ B − A ⪯ εA

  2. Unfold the Loewner ordering into quadratic forms, for all x ∈ ℝⁿ 13:27: −ε·xᵀAx ⪯ xᵀ(B−A)x ⪯ ε·xᵀAx

  3. Use the spectral theorem: a real symmetric matrix factors as A = UΛUᵀ with U orthogonal and (here, PSD) non-negative eigenvalues in Λ 13:56.

    • Applying a function g to a matrix means applying g to each eigenvalue in that decomposition 14:19.

  4. Hence xᵀAx = ‖A^{1/2}x‖₂². Substitute y = A^{1/2}x, i.e. x = A^{−1/2}y 14:39, 15:33.

  5. The condition becomes, for all y 15:33, 16:02:

    −ε·I ⪯ A^{−1/2}(B − A)A^{−1/2} ⪯ ε·I

  • This whitened form is exactly what reappears inside the Newton proof 16:31.

Invertibility caveat

  • The derivation assumes A is invertible; for f_λ the Hessian is invertible, so the lecture proceeds under that assumption 16:55.
  • Aside on the general case: for the sandwich inequality to hold at all, A and B must have the same column space; one can restrict to that subspace via a matrix U with orthonormal columns forming a basis for it, and work with A·Ux instead 16:55, 17:23, 17:39.