{ "id": "ep1vxk62wc", "slug": "metric-learning", "title": "Metric learning", "updated": "2026-10-21", "topic_path": [ { "id": "technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/topics/technology" }, { "id": "technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing" }, { "id": "technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai" }, { "id": "technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_computation" }, { "id": "technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_methods" }, { "id": "technology.computing.ai.supervised_unsupervised", "label": "Supervised, unsupervised, and semi-supervised learning", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.supervised_unsupervised" }, { "id": "technology.computing.ai.supervised_unsupervised.supervised_concepts", "label": "Supervised learning concepts", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.supervised_unsupervised.supervised_concepts" } ], "geo": [ { "id": "geo.nongeo.t1946.technology.computing.ai.supervised_unsupervised", "label": "Non-geographic · 1946 to 2000: Supervised, unsupervised, and semi-supervised learning", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing.ai.supervised_unsupervised", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t1946", "label": "Non-geographic · 1946 to 2000", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946" }, { "id": "geo.nongeo.t1946.technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology" }, { "id": "geo.nongeo.t1946.technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing" }, { "id": "geo.nongeo.t1946.technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing.ai" }, { "id": "geo.nongeo.t1946.technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing.ai.ml_computation" }, { "id": "geo.nongeo.t1946.technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing.ai.ml_methods" }, { "id": "geo.nongeo.t1946.technology.computing.ai.supervised_unsupervised", "label": "Supervised, unsupervised, and semi-supervised learning", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing.ai.supervised_unsupervised" } ] }, { "id": "geo.nongeo.t2001.technology.computing.ai.supervised_unsupervised", "label": "Non-geographic · 2001 to 2020: Supervised, unsupervised, and semi-supervised learning", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.supervised_unsupervised", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t2001", "label": "Non-geographic · 2001 to 2020", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001" }, { "id": "geo.nongeo.t2001.technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology" }, { "id": "geo.nongeo.t2001.technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing" }, { "id": "geo.nongeo.t2001.technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_computation" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_methods" }, { "id": "geo.nongeo.t2001.technology.computing.ai.supervised_unsupervised", "label": "Supervised, unsupervised, and semi-supervised learning", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.supervised_unsupervised" } ] } ], "excerpt": "Metric learning, also called distance metric learning, is a machine learning approach that learns a task-specific distance function, typically a Mahalanobis distance, for kNN, clustering, and retrieval.", "snippet": "Metric learning, also called distance metric learning, is a machine learning approach that learns a task-specific distance function, typically a Mahalanobis distance, for kNN, clustering, and retrieval.", "node": "technology.computing.ai.supervised_unsupervised.supervised_concepts", "markdown": "# Metric learning\n\nMetric learning is a machine learning approach that learns a task-specific distance function so that similar examples are mapped close together and dissimilar examples far apart. The learned metric, typically a [Mahalanobis distance](https://www.edgepedia.org/mahalanobis-distance), is used downstream for k-nearest-neighbor classification, clustering, information retrieval, and dimensionality reduction.[1](http://contrib.scikit-learn.org/metric-learn/introduction.html) In practice the output is a matrix or a linear transformation: software implementations expose the learned metric as a matrix \\( M \\) or as a transformation \\( L \\) that embeds points before Euclidean distances are applied.[2](http://contrib.scikit-learn.org/metric-learn/generated/metric_learn.ITML.html)\n\n| Key fact | Detail |\n| --- | --- |\n| What is learned | A Mahalanobis matrix \\( M \\), equivalently a linear map \\( L \\) with \\( M = L^{\\top} L \\), used to embed data or compute pairwise distances[3](https://ar5iv.labs.arxiv.org/html/1812.05944) |\n| Distance form | \\( d_{M}(x, x') = \\sqrt{(x - x')^{\\top} M (x - x')} \\), usually optimized in squared form to avoid the square root[4](https://arxiv.org/pdf/1306.6709) |\n| Supervision format | Pairwise equivalence and inequivalence constraints, or triplets; class labels can also be used directly[5](https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf) |\n| Constraint | \\( M \\) must be symmetric positive semidefinite for the function to be a metric; \\( M = I \\) recovers the Euclidean distance[3](https://ar5iv.labs.arxiv.org/html/1812.05944) |\n| Canonical algorithms | A convex program for clustering constraints, NCA, LMNN, and ITML[3](https://ar5iv.labs.arxiv.org/html/1812.05944) |\n| Typical uses | kNN classification, clustering, retrieval and ranking, face verification, bioinformatics, recommender systems[1](http://contrib.scikit-learn.org/metric-learn/introduction.html) |\n| Practical solver note | ITML requires no eigenvalue computations or semidefinite programming[6](https://www.cs.utexas.edu/~inderjit/public_papers/itml_icml07.pdf) |\n\n## How it works\n\nThe Mahalanobis distance generalizes the [Euclidean distance](https://www.edgepedia.org/euclidean-distance) by inserting a symmetric positive semidefinite matrix \\( M \\) between the difference vector and itself: \\[ d_{M}(x, x') = \\sqrt{(x - x')^{\\top} M (x - x')} \\] Learning takes place on the squared distance, which avoids the square root.[4](https://arxiv.org/pdf/1306.6709) When \\( M \\) has full rank the function is a proper distance; when it is rank-deficient it is a pseudodistance. The Euclidean distance is the special case \\( M = I \\).[3](https://ar5iv.labs.arxiv.org/html/1812.05944)\n\nBecause \\( M \\) is positive semidefinite it can be factorized as \\( M = L^{\\top} L \\), and simple algebra shows the Mahalanobis distance equals the Euclidean distance \\( \\lVert Lx - Ly \\rVert \\) after a global linear transformation.[7](https://people.bu.edu/bkulis/pubs/ftml_metric_learning.pdf) Learning \\( M \\) and learning a linear map \\( L \\) are therefore equivalent views of the same problem; learning \\( L \\) is unconstrained, while learning \\( M \\) requires maintaining the positive semidefiniteness constraint.[3](https://ar5iv.labs.arxiv.org/html/1812.05944)\n\nWhat the learned metric buys over Euclidean distance is variance awareness: Euclidean distance ignores the scatter of data clouds, whereas a Mahalanobis distance accounts for variance and can assign a point to the more tightly scattered cloud.[8](https://ar5iv.labs.arxiv.org/html/2201.09267) Strictly speaking, a positive-semidefinite learned Mahalanobis distance is a pseudo-metric: it satisfies non-negativity, symmetry, and the triangle inequality, but not necessarily the identity of indiscernibles; when the matrix is positive definite, it is a proper metric.[1](http://contrib.scikit-learn.org/metric-learn/introduction.html)\n\n## How it is done\n\nSupervised metric learning can use class labels directly or use pairwise, triplet, or other constraints, which may themselves be derived from labels: equivalence constraints say two points should be close, inequivalence constraints say they should be far, and methods divide into global approaches that satisfy all constraints and local ones.[5](https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf) Most methods fit the general objective \\[ M^{*} = \\arg\\min_{M} \\; \\ell(M, S, D, R) + \\lambda R(M) \\] where \\( \\ell \\) penalizes violated must-link, cannot-link, or relative constraints, \\( R(M) \\) is a regularizer, and \\( \\lambda \\geq 0 \\).[9](https://researchers.lille.inria.fr/abellet/talks/metric_learning_tutorial_CIL.pdf)\n\nThe main algorithmic choices are:\n\n- **Convex program for clustering.** An early formulation minimizes \\( \\sum_{(x_i, x_j) \\in S} \\lVert x_i - x_j \\rVert^{2}_{A} \\) subject to \\( A \\succeq 0 \\) and \\( \\sum_{(x_i, x_j) \\in D} \\lVert x_i - x_j \\rVert^{2}_{A} \\geq 1 \\), with the number of free parameters equal to \\( d(d+1)/2 \\) for a full symmetric \\( d \\times d \\) matrix, or \\( d \\) under an additional diagonal-matrix restriction.[5](https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf)\n- **NCA.** Optimizes the expected leave-one-out error of a stochastic nearest neighbor classifier, using the \\( M = L^{\\top} L \\) decomposition; it is nonconvex and tailored to the 1NN classifier, and a rectangular \\( L \\) allows dimensionality reduction.[4](https://arxiv.org/pdf/1306.6709) • [9](https://researchers.lille.inria.fr/abellet/talks/metric_learning_tutorial_CIL.pdf)\n- **LMNN.** Optimizes a two-term error that penalizes large distances to same-class target neighbors and small distances to different-class impostors, solved as a convex semidefinite program; a subgradient solver handles millions to billions of constraints.[3](https://ar5iv.labs.arxiv.org/html/1812.05944) • [4](https://arxiv.org/pdf/1306.6709)\n- **ITML.** Minimizes the LogDet divergence, subject to similarity constraints \\( d_A(x_i, x_j) \\leq u \\) and dissimilarity constraints \\( d_A(x_i, x_j) \\geq l \\); each Bregman-projection update \\( M_{t+1} = M_t + \\beta M_t (x_i - x_j)(x_i - x_j)^{\\top} M_t \\) costs \\( O(d^{2}) \\) per constraint and enforces positive semidefiniteness automatically.[6](https://www.cs.utexas.edu/~inderjit/public_papers/itml_icml07.pdf) • [10](https://cvg.cit.tum.de/_media/teaching/ss2017/ml4cv/metric_learning.pdf)\n- **Deep losses.** The triplet loss pushes similar neighbors together and dissimilar points apart with a margin; in deep metric learning, performance depends jointly on the loss function, the sampling strategy, and the network structure.[8](https://ar5iv.labs.arxiv.org/html/2201.09267) • [11](https://www.mdpi.com/2073-8994/11/9/1066)\n\nThe learned metric is then plugged into kNN classification, clustering, or retrieval pipelines; the metric-learn library implements ten algorithms behind a unified scikit-learn-compatible API covering supervised, pair, triplet, and quadruplet learners.[12](https://www.jmlr.org/papers/volume21/19-678/19-678.pdf)\n\n## Origin\n\nMetric learning as a field is credited to a 2002 paper by [Eric P. Xing](https://www.edgepedia.org/eric-poe-xing) and colleagues, \"Distance Metric Learning with Application to Clustering with Side-Information\", which presented a convex optimization algorithm that learns a distance metric from similar and, optionally, dissimilar pairs of points; surveys mark this work as the point where the field emerged.[13](https://dl.acm.org/doi/10.1016/j.neunet.2018.06.003) Neighbourhood Components Analysis followed in 2004, authored by Jacob Goldberger and colleagues.[12](https://www.jmlr.org/papers/volume21/19-678/19-678.pdf) The large-margin nearest neighbor formulation appeared in the 2005 NIPS paper by [Kilian Q. Weinberger](https://www.edgepedia.org/kilian-q-weinberger), John Blitzer, and Lawrence K. Saul, then at the University of Pennsylvania.[14](https://dl.acm.org/doi/10.5555/2976248.2976433) More recently, deep metric learning has shifted attention to embedding networks trained with pairwise, triplet, and batch-level losses.[8](https://ar5iv.labs.arxiv.org/html/2201.09267)\n\n## Variants\n\nGlobal methods learn one metric satisfying all constraints, while local methods learn separate metrics, for example one per cluster of training examples, which can improve results.[5](https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf) • [14](https://dl.acm.org/doi/10.5555/2976248.2976433) Linear Mahalanobis methods split into convex and nonconvex subfamilies, and kernelization extends them to nonlinear mappings by learning a linear transformation in the feature space of \\( \\phi \\), requiring only inner products of the data.[7](https://people.bu.edu/bkulis/pubs/ftml_metric_learning.pdf) Named variants include RCA, which learns a full-rank metric from a weighted sum of in-class covariance matrices with closed-form solution \\( B = (K/N) \\hat{C}^{-1} \\), MCML, LEGO, GB-LMNN, and the information-geometry methods IGML and KIGML, which yield closed-form solutions.[5](https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf) • [9](https://researchers.lille.inria.fr/abellet/talks/metric_learning_tutorial_CIL.pdf) • [15](http://proceedings.mlr.press/v5/wang09c/wang09c.pdf) A 2024 survey organizes transfer metric learning, where a metric learned on one domain is adapted to another, into direct metric approximation, subspace approximation, distance approximation, and distribution approximation; direct approximation overfits in high dimensions because of the parameter count, motivating low-rank decompositions \\( A_m = U_m U_m^{\\top} \\) with \\( U_m \\in \\mathbb{R}^{d_m \\times r} \\) that create a shared subspace for transfer.[16](https://link.springer.com/article/10.1007/s44336-024-00003-8)\n\n## Applications\n\nDocumented application areas include computer vision tasks such as image classification, face recognition, tracking, and annotation; information retrieval and ranking; bioinformatics; music recommendation; identity verification; medicine, security, speech recognition, recommender systems, person re-identification, and kinship verification.[4](https://arxiv.org/pdf/1306.6709) • [9](https://researchers.lille.inria.fr/abellet/talks/metric_learning_tutorial_CIL.pdf) • [3](https://ar5iv.labs.arxiv.org/html/1812.05944)\n\nThe NIPS 2005 LMNN paper reports a test error rate of 1.3% on MNIST handwritten digits across seven data sets of varying size and difficulty.[14](https://dl.acm.org/doi/10.5555/2976248.2976433) In the ITML paper's benchmark against MCML, LMNN, and the 2002 convex method on UCI datasets, ITML was the only algorithm to obtain the optimal error rate within the specified 95% confidence intervals across all datasets.[6](https://www.cs.utexas.edu/~inderjit/public_papers/itml_icml07.pdf) Gains are not universal: on three of four software error-reporting datasets, learned metrics yielded only marginal improvements over the Euclidean baseline.[6](https://www.cs.utexas.edu/~inderjit/public_papers/itml_icml07.pdf)\n\n## Limitations and alternatives\n\nLinear metrics often cannot capture multimodal data or nonlinear class boundaries; the standard remedies are kernelization, nonlinear metric forms, and multiple local metrics, though kernel approaches can worsen overfitting and have scaling issues.[9](https://researchers.lille.inria.fr/abellet/talks/metric_learning_tutorial_CIL.pdf) • [11](https://www.mdpi.com/2073-8994/11/9/1066) Maintaining the PSD constraint by projected gradient requires an eigenvalue decomposition scaling as \\( O(d^{3}) \\), expensive in high dimension, and optimizing \\( M \\) under a rank constraint is NP-hard.[4](https://arxiv.org/pdf/1306.6709) LMNN is prone to overfitting in high dimension due to absent regularization and is sensitive to how well the Euclidean distance selects target neighbors; NCA has \\( O(d^{2}) \\) parameters in its distance matrix, no guarantee of converging to local maxima, and tends to overfit when training examples are insufficient.[4](https://arxiv.org/pdf/1306.6709) • [3](https://ar5iv.labs.arxiv.org/html/1812.05944) In deep metric learning, inefficient pair or triplet sampling wastes time and memory, and training-time complexity can grow as \\( n^{2} \\) for pairs, \\( n^{3} \\) for triplets, and \\( n^{4} \\) for quadruplets.[11](https://www.mdpi.com/2073-8994/11/9/1066) Metric learning assumes at least some supervision is available, distinguishing it from unsupervised dimensionality reduction such as PCA; RCA learns a metric from must-link (equivalence) constraints, while DCA can additionally exploit cannot-link constraints; both can be seen as extensions of linear discriminant analysis.[7](https://people.bu.edu/bkulis/pubs/ftml_metric_learning.pdf) • [5](https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf)\n\n## References\n\n1. [What is Metric Learning?, metric-learn 0.7.0 documentation](http://contrib.scikit-learn.org/metric-learn/introduction.html)\n2. [metric_learn.ITML, metric-learn 0.7.0 documentation](http://contrib.scikit-learn.org/metric-learn/generated/metric_learn.ITML.html)\n3. [A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms, Experimental Analysis, Prospects and Challenges](https://ar5iv.labs.arxiv.org/html/1812.05944)\n4. [A Survey on Metric Learning for Feature Vectors and Structured Data (Bellet, Habrard, Sebban)](https://arxiv.org/pdf/1306.6709)\n5. [Distance Metric Learning: A Comprehensive Survey (Liu et al., CMU)](https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf)\n6. [Information-Theoretic Metric Learning (ICML 2007)](https://www.cs.utexas.edu/~inderjit/public_papers/itml_icml07.pdf)\n7. [Metric Learning: A Survey (Kulis, Foundations and Trends in ML)](https://people.bu.edu/bkulis/pubs/ftml_metric_learning.pdf)\n8. [Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey (Ghojogh et al.)](https://ar5iv.labs.arxiv.org/html/2201.09267)\n9. [Tutorial on Metric Learning (Bellet, CIL tutorial slides)](https://researchers.lille.inria.fr/abellet/talks/metric_learning_tutorial_CIL.pdf)\n10. [Metric Learning (TUM lecture slides, Chiotellis)](https://cvg.cit.tum.de/_media/teaching/ss2017/ml4cv/metric_learning.pdf)\n11. [Deep Metric Learning: A Survey](https://www.mdpi.com/2073-8994/11/9/1066)\n12. [metric-learn: Metric Learning Algorithms in Python (JMLR 2020 software paper)](https://www.jmlr.org/papers/volume21/19-678/19-678.pdf)\n13. [Survey and experimental study on metric learning methods (Neural Networks)](https://dl.acm.org/doi/10.1016/j.neunet.2018.06.003)\n14. [Distance metric learning for large margin nearest neighbor classification (NIPS 2005 proceedings version)](https://dl.acm.org/doi/10.5555/2976248.2976433)\n15. [An information geometry approach for distance metric learning (Wang, Jin, ICML 2009)](http://proceedings.mlr.press/v5/wang09c/wang09c.pdf)\n16. [Transfer metric learning: algorithms, applications and outlooks (Vicinagearth, 2024)](https://link.springer.com/article/10.1007/s44336-024-00003-8)\n\n---\n*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data › Machine learning and neural computation › Machine learning methods › Supervised, unsupervised, and semi-supervised learning › Supervised learning concepts*\n\n*Initially written Sep 29, 2026 · Reviewed: Sep 30, 2026 · Edited: Sep 30, 2026; Oct 21, 2026 · Last review: Sep 30, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [ "https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf", "https://www.cs.utexas.edu/~inderjit/public_papers/itml_icml07.pdf" ], "url": "https://www.edgepedia.org/metric-learning", "markdown_url": "https://www.edgepedia.org/metric-learning.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Metric learning\", Edgepedia (EdgeChat), https://www.edgepedia.org/metric-learning. Edgepedia Community License 1.0.", "credit_md": "\"[Metric learning](https://www.edgepedia.org/metric-learning)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/metric-learning](https://www.edgepedia.org/metric-learning). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Metric learning\", Edgepedia (EdgeChat), https://www.edgepedia.org/metric-learning. Edgepedia Community License 1.0.", "speakable": "Metric learning, also called distance metric learning, is a machine learning approach that learns a task-specific distance function, typically a Mahalanobis distance, for kNN, clustering, and retrieval." }