# Singular value decomposition

The singular value decomposition (SVD) is a factorization of any real or complex matrix into the product of two orthogonal (or unitary) matrices and a diagonal matrix of nonnegative numbers, written for a real matrix as \( A = U \Sigma V^T \). The diagonal entries \( \sigma_i \) of \( \Sigma \), called the singular values, are conventionally arranged in descending order, \( \sigma_1 \ge \sigma_2 \ge \cdots \ge \sigma_r \ge 0 \) with \( r = \min(m,n) \) for an \( m \times n \) matrix; the corresponding columns of \( U \) and \( V \) are the left and right singular vectors<sup>[1](https://www.netlib.org/lapack/lug/node53.html)</sup>. The decomposition exists for every \( m \times n \) matrix, square or rectangular, symmetric or not<sup>[2](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)</sup>.

| Key fact | Detail |
|---|---|
| Definition | \( A = U \Sigma V^T \) (real) or \( A = U \Sigma V^H \) (complex), with \( U, V \) orthogonal or unitary and \( \Sigma \) diagonal with \( \sigma_1 \ge \cdots \ge \sigma_r \ge 0 \)<sup>[1](https://www.netlib.org/lapack/lug/node53.html)</sup> |
| Relation to eigenvalues | The singular values are the square roots of the eigenvalues of \( A^T A \); the columns of \( V \) are the corresponding eigenvectors<sup>[2](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)</sup> |
| Rank | The number of nonzero singular values equals the rank of \( A \); the first \( r \) columns of \( U \) form an orthonormal basis for the column space of \( A \), and the first \( r \) columns of \( V \) form an orthonormal basis for the row space of \( A \)<sup>[2](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)</sup> |
| Largest singular value | \( \sigma_1 = \max \{ \lVert A x \rVert / \lVert x \rVert : x \ne 0 \} \), the maximum stretch a unit vector can receive<sup>[2](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)</sup> |
| Optimality | By the Eckart–Young theorem, truncating the SVD after \( k \) terms gives the best rank-\( k \) approximation to \( A \) in both the 2-norm and the Frobenius norm<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup> |
| First publication | Eugenio Beltrami, 1873, for simplification of bilinear forms; Camille Jordan followed in 1874<sup>[4](https://www.math.ucdavis.edu/~saito/courses/229A/stewart-svd.pdf)</sup> |
| Modern computation | LAPACK provides the simple driver xGESVD and the divide-and-conquer driver xGESDD, which is much faster for large matrices but uses more workspace<sup>[5](https://www.netlib.org/lapack/lug/node32.html)</sup> |

## Definition and basic properties

For a real \( m \times n \) matrix \( A \), an SVD is a decomposition \( A = U \Sigma V^T \) in which \( U \) is \( m \times m \) orthogonal, \( V \) is \( n \times n \) orthogonal, and \( \Sigma \) is \( m \times n \) with zeros everywhere except the diagonal entries \( \Sigma_{ii} = \sigma_i \), which are nonnegative and nonincreasing<sup>[2](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)</sup>. For complex matrices the transpose is replaced by the conjugate transpose and the orthogonal matrices by unitary ones<sup>[1](https://www.netlib.org/lapack/lug/node53.html)</sup>. The singular values and vectors satisfy \( A v_i = \sigma_i u_i \) and \( A^T u_i = \sigma_i v_i \)<sup>[5](https://www.netlib.org/lapack/lug/node32.html)</sup>.

The decomposition is computed from the symmetric matrix \( A^T A \): its eigenvalues are \( \sigma_i^2 \), and its eigenvectors are the right singular vectors \( v_i \)<sup>[2](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)</sup>. For a symmetric matrix, \( A^T A = A^2 \), so the singular values are the absolute values of the eigenvalues<sup>[2](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)</sup>.

**Geometric meaning.** The largest singular value measures the greatest amplification the map can apply to any vector, \( \sigma_1 = \max \lVert A x \rVert / \lVert x \rVert \) over nonzero \( x \)<sup>[2](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)</sup>.

**What the decomposition reveals.** The SVD provides orthogonal bases for the domain and range spaces, the 2-norm and [Frobenius norm](https://www.edgepedia.org/frobenius-norm) of the matrix, and the numerical rank<sup>[6](https://help.perforce.com/sourcepro/previous_releases/11.1/html/lapackug/8-2.html)</sup>. The Frobenius norm satisfies \( \lVert A \rVert_F^2 = \sum_i \sigma_i^2 \), the spectral norm equals \( \sigma_1 \), and the nuclear norm equals \( \sum_i \sigma_i \)<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup>. Because the diagonal form makes it easy to see when \( A \) is near a rank-degenerate matrix, the SVD is a standard tool for detecting numerical rank<sup>[4](https://www.math.ucdavis.edu/~saito/courses/229A/stewart-svd.pdf)</sup>.

## Low-rank approximation and the Eckart–Young theorem

Writing the SVD as a sum of rank-one pieces,

\[ A = \sigma_1 u_1 v_1^T + \sigma_2 u_2 v_2^T + \cdots + \sigma_r u_r v_r^T, \]

the rank-\( k \) truncation \( A_k \) keeps only the first \( k \) terms<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup>. The Eckart–Young theorem states that for every matrix \( B \) of rank at most \( k \), \( \lVert A - B \rVert \ge \lVert A - A_k \rVert \), in both the 2-norm and the Frobenius norm, so the truncated SVD is the closest rank-\( k \) matrix to \( A \)<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup>. The proof rests on the orthogonal invariance of these norms<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup>.

This optimality property drives the main data applications. In image compression, keeping the leading terms of the SVD of an image matrix gives the best approximation of that rank<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup>. In recommender systems, a user-by-movie rating matrix is approximated by a low-rank factorization whose factors represent user preferences and movie characteristics, and predicted ratings are reconstructed from the truncated sum<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup>. In MRI reconstruction, the assumption that real images are approximately low rank lets partial measurements be completed by the best rank-\( k \) approximation<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup>.

## Relation to principal component analysis

The SVD is the mathematical basis of principal component analysis (PCA)<sup>[7](https://arxiv.org/abs/1804.05090)</sup>. For a centered data matrix \( A \), the left singular vectors are the principal directions of maximum variance, and the variance explained by component \( i \) is \( \sigma_i^2 / (n-1) \)<sup>[3](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)</sup>.

There is also a numerical reason to prefer the SVD over eigendecomposition of the covariance matrix. If \( A = U \Sigma V^T \), then \( A^T A = V \Sigma^2 V^T \), so the condition number of \( A^T A \) is the square of the condition number of \( A \)<sup>[8](https://math.mit.edu/classes/18.085/summer2016/pset2016/pset5solution.pdf)</sup>. Forming the covariance matrix therefore squares the sensitivity to error, which is why PCA implementations commonly compute the SVD of the data matrix directly<sup>[8](https://math.mit.edu/classes/18.085/summer2016/pset2016/pset5solution.pdf)</sup>.

## History

The modern historian of the subject is G. W. Stewart, emeritus professor at the University of Maryland and a leading authority on matrix computations, whose 1993 SIAM Review survey organized the early record around five mathematicians: Eugenio Beltrami (1835–1899), [Camille Jordan](https://www.edgepedia.org/camille-jordan) (1838–1921), [James Joseph Sylvester](https://www.edgepedia.org/james-joseph-sylvester) (1814–1897), [Erhard Schmidt](https://www.edgepedia.org/erhard-schmidt) (1876–1959), and Hermann Weyl (1885–1955)<sup>[4](https://www.math.ucdavis.edu/~saito/courses/229A/stewart-svd.pdf)</sup>.

**Origins, 1873–1889.** Beltrami and Jordan are the progenitors of the decomposition, Beltrami by virtue of first publication in 1873 and Jordan by the completeness of his treatment; both sought simplification of bilinear forms in the 1870s, and Jordan obtained the geometric interpretation of the largest singular value<sup>[4](https://www.math.ucdavis.edu/~saito/courses/229A/stewart-svd.pdf)</sup><sup> • </sup><sup>[9](https://math.mit.edu/~gs/linearalgebra/ila6/SVD_Slides.pdf)</sup>. Sylvester wrote two papers on the SVD in 1889, giving algorithms to diagonalize quadratic and bilinear forms by orthogonal substitutions<sup>[9](https://math.mit.edu/~gs/linearalgebra/ila6/SVD_Slides.pdf)</sup>.

**Extension and computation.** Erhard Schmidt arrived at the SVD for function spaces while investigating integral equations, seeking best rank-\( k \) approximations of the form \( u_1 v_1^t + \cdots + u_k v_k^t \)<sup>[9](https://math.mit.edu/~gs/linearalgebra/ila6/SVD_Slides.pdf)</sup>. Autonne extended the SVD to complex matrices in 1913, and Eckart and Young extended it to rectangular matrices in 1936<sup>[9](https://math.mit.edu/~gs/linearalgebra/ila6/SVD_Slides.pdf)</sup>. In the early 1960s the SVD was relatively little known and no satisfactory way of computing it was available; Gene Golub and [William Kahan](https://www.edgepedia.org/william-kahan) changed this in 1965 with a numerically stable scheme that first transforms \( A \) to bidiagonal form and then diagonalizes it<sup>[10](https://epubs.siam.org/doi/abs/10.1137/0702016)</sup><sup> • </sup><sup>[11](https://eprints.maths.manchester.ac.uk/1821/1/high07g.pdf)</sup>. Golub proposed a practical algorithm in 1970<sup>[9](https://math.mit.edu/~gs/linearalgebra/ila6/SVD_Slides.pdf)</sup>, and the Golub–Kahan paper is credited with bringing the SVD to prominence by explaining its applications and advocating the now-standard device of setting negligible computed singular values to zero<sup>[11](https://eprints.maths.manchester.ac.uk/1821/1/high07g.pdf)</sup>.

## Computation in practice

Dense SVD software follows the two-stage scheme introduced by Golub and Kahan: reduce \( A \) to bidiagonal form, then compute the SVD of the bidiagonal matrix<sup>[1](https://www.netlib.org/lapack/lug/node53.html)</sup>. LAPACK offers two drivers: xGESVD, the simple driver, and xGESDD, a divide-and-conquer driver that is much faster for large matrices but uses more workspace<sup>[5](https://www.netlib.org/lapack/lug/node32.html)</sup>. The bidiagonal QR routine xBDSQR computes all singular values to nearly full relative precision, independent of their magnitudes, barring underflow and overflow<sup>[1](https://www.netlib.org/lapack/lug/node53.html)</sup>. Later LAPACK releases added a Jacobi SVD routine that can compute tiny singular values to many more correct digits than xGESVD when the matrix has columns of widely differing norms<sup>[12](https://www.netlib.org/lapack/improvement.html)</sup>.

In scientific software, SciPy's `svd` function defaults to the divide-and-conquer driver `gesdd`, while MATLAB and Octave use `gesvd`<sup>[13](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.svd.html)</sup>; for sparse matrices, SciPy's `svds` computes only the largest or smallest \( k \) singular values and vectors<sup>[14](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.svds.html)</sup>.

**Randomized SVD.** For very large matrices, randomized algorithms have become a well-established alternative<sup>[15](https://onlinelibrary.wiley.com/doi/full/10.1002/nla.70088)</sup>. The framework popularized by Nathan Halko, Per-Gunnar Martinsson, and Joel Tropp, researchers in applied and computational mathematics, uses random sampling to identify a subspace capturing most of the action of the matrix, then compresses the matrix to that subspace before finishing deterministically<sup>[16](https://epubs.siam.org/doi/abs/10.1137/090771806?download=true&journalCode=siread)</sup>. For the model problem of finding the \( k \) dominant SVD components of a dense \( m \times n \) matrix, randomized algorithms require \( O(mn \log k) \) floating-point operations against \( O(mnk) \) for classical algorithms; for sparse inputs the flop count matches Krylov methods, but the randomized approach is more robust and easier to organize for multiprocessors<sup>[16](https://epubs.siam.org/doi/abs/10.1137/090771806?download=true&journalCode=siread)</sup>.

## References

1. [Singular Value Decomposition, LAPACK Users' Guide, netlib](https://www.netlib.org/lapack/lug/node53.html)
2. [8.3. Singular value decomposition, Linear Algebra interactive textbook, TU Delft](https://interactivetextbooks.tudelft.nl/linear-algebra/Chapter8/SingularValueDecomp.html)
3. [MIT 18.065 Lecture 7: Eckart-Young Theorem, The Closest Rank k Matrix](https://ickma2311.github.io/Math/MIT18.065/mit18065-lecture7-eckart-young.html)
4. [G. W. Stewart (1993). On the Early History of the Singular Value Decomposition. SIAM Review 35(4), 551–566](https://www.math.ucdavis.edu/~saito/courses/229A/stewart-svd.pdf)
5. [Singular Value Decomposition (SVD), LAPACK Users' Guide driver routines, netlib](https://www.netlib.org/lapack/lug/node32.html)
6. [Singular Value Decomposition, Linear Algebra Module User's Guide, Rogue Wave/Perforce](https://help.perforce.com/sourcepro/previous_releases/11.1/html/lapackug/8-2.html)
7. [Regularized Singular Value Decomposition and Application to Recommender System, arXiv](https://arxiv.org/abs/1804.05090)
8. [MIT 18.085 Summer 2016 Problem Set 5 solutions, condition number of A^T A](https://math.mit.edu/classes/18.085/summer2016/pset2016/pset5solution.pdf)
9. [A brief history of SVD, MIT linear algebra slides (Gilbert Strang)](https://math.mit.edu/~gs/linearalgebra/ila6/SVD_Slides.pdf)
10. [G. Golub and W. Kahan (1965). Calculating the Singular Values and Pseudo-Inverse of a Matrix. SIAM J. Numer. Anal. 2(2), 205–224](https://epubs.siam.org/doi/abs/10.1137/0702016)
11. [Commentary on Golub and Kahan (1965), University of Manchester eprints](https://eprints.maths.manchester.ac.uk/1821/1/high07g.pdf)
12. [Summary of improvements since LAPACK 3.0, netlib](https://www.netlib.org/lapack/improvement.html)
13. [scipy.linalg.svd, SciPy v1.18.0 Manual](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.svd.html)
14. [scipy.sparse.linalg.svds, SciPy v1.18.0 Manual](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.svds.html)
15. [D. Palitta and S. Portaro (2026). Row-Aware Randomized SVD With Applications. Numerical Linear Algebra with Applications](https://onlinelibrary.wiley.com/doi/full/10.1002/nla.70088)
16. [N. Halko, P. G. Martinsson, J. A. Tropp (2011). Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions. SIAM Review](https://epubs.siam.org/doi/abs/10.1137/090771806?download=true&journalCode=siread)

---
*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Numbers and algebra › Linear and multilinear algebra › Decompositions and canonical forms › Singular value and rank factorizations*

*Initially written Sep 17, 2026 · Reviewed: Sep 17, 2026; Oct 15, 2026 · Edited: Oct 15, 2026 · Last review: Oct 15, 2026*

*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*

License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license