{ "id": "epp817rav6", "slug": "pseudo-labeling", "title": "Pseudo-labeling", "updated": "2026-10-25", "topic_path": [ { "id": "technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/topics/technology" }, { "id": "technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing" }, { "id": "technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai" }, { "id": "technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_computation" }, { "id": "technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_methods" }, { "id": "technology.computing.ai.supervised_unsupervised", "label": "Supervised, unsupervised, and semi-supervised learning", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.supervised_unsupervised" }, { "id": "technology.computing.ai.supervised_unsupervised.semi_supervised_learning", "label": "Semi-supervised and weakly supervised learning", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.supervised_unsupervised.semi_supervised_learning" } ], "geo": [ { "id": "geo.nongeo.t2001.technology.computing.ai.supervised_unsupervised", "label": "Non-geographic · 2001 to 2020: Supervised, unsupervised, and semi-supervised learning", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.supervised_unsupervised", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t2001", "label": "Non-geographic · 2001 to 2020", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001" }, { "id": "geo.nongeo.t2001.technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology" }, { "id": "geo.nongeo.t2001.technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing" }, { "id": "geo.nongeo.t2001.technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_computation" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_methods" }, { "id": "geo.nongeo.t2001.technology.computing.ai.supervised_unsupervised", "label": "Supervised, unsupervised, and semi-supervised learning", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.supervised_unsupervised" } ] } ], "excerpt": "Pseudo-labeling is a semi-supervised learning technique in which a model treats its most confident predictions on unlabeled data as true labels, introduced for deep networks by Dong-Hyun Lee in 2013.", "snippet": "Pseudo-labeling is a semi-supervised learning technique in which a model treats its most confident predictions on unlabeled data as true labels, introduced for deep networks by Dong-Hyun Lee in 2013.", "node": "technology.computing.ai.supervised_unsupervised.semi_supervised_learning", "markdown": "# Pseudo-labeling\n\nPseudo-labeling is a semi-supervised learning technique in which a model treats its most confident predictions on unlabeled data as true labels and retrains on the enlarged set. It belongs to the older family of self-labeled techniques, which iteratively accept their own predictions as correct to grow the labeled set.[1](https://link.springer.com/article/10.1007/s10115-013-0706-y) In its deep-learning form, the class with maximum predicted probability (the argmax) becomes a hard pseudo-label, recalculated at every weights update while the network trains on labeled and unlabeled data simultaneously.[2](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.664.3543) The technique addresses the central problem of semi-supervised learning: how to use abundant unlabeled data when only a small labeled set exists. A large family of later semi-supervised learning techniques for computer vision builds on the original pseudo-labeling paper.[3](https://openreview.net/pdf?id=cjivIeoSXz)\n\n| Key fact | Value |\n|---|---|\n| Introducing deep-learning paper | Dong-Hyun Lee, \"Pseudo-Label\", ICML 2013 Workshop on Challenges in Representation Learning[2](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.664.3543) |\n| Original loss weighting | Unsupervised term weighted by \\( \\alpha(t) \\), annealed to a ceiling of \\( \\alpha_{\\mathrm{f}} = 3 \\)[2](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.664.3543) |\n| Common confidence threshold | \\( \\tau = 0.95 \\) in FixMatch; \\( \\tau = 0.7 \\) with \\( \\lambda_u = 10 \\) on ImageNet[4](https://proceedings.neurips.cc/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Paper.pdf) • [5](https://papers.nips.cc/paper_files/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Supplemental.pdf) |\n| CIFAR-10 with 40 labels | FixMatch 88.61% accuracy (4 labels per class)[4](https://proceedings.neurips.cc/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Paper.pdf) |\n| ImageNet top-1 | Noisy Student 88.4%; Meta Pseudo Labels 90.2%[6](https://openaccess.thecvf.com/content_CVPR_2020/papers/Xie_Self-Training_With_Noisy_Student_Improves_ImageNet_Classification_CVPR_2020_paper.pdf) • [7](https://openaccess.thecvf.com/content/CVPR2021/papers/Pham_Meta_Pseudo_Labels_CVPR_2021_paper.pdf) |\n| Main failure mode | Confirmation bias: initially wrong predictions are reinforced by training on them[8](https://ar5iv.labs.arxiv.org/html/1908.02983) |\n\n## How it works\n\nTraining on the model's own confident predictions helps because it pushes decision boundaries through low-density regions between classes. Lee argued the method is in effect equivalent to entropy regularization: forcing hard, high-confidence predictions on unlabeled data minimizes the conditional entropy\n\n\\[ H(Y \\mid X, Z) = -\\mathbb{E}_{XYZ}[\\log P(Y \\mid X, Z)], \\]\n\nwhich favors low-density separation between classes, a standard prior for semi-supervised learning.[2](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.664.3543) Grandvalet and Bengio's earlier MAP criterion, log-likelihood minus a Lagrange-weighted entropy penalty, treats self-training as a particular case with \\( \\lambda = 1 \\), and their analysis shows unlabeled examples help mainly when classes have small overlap.[9](https://proceedings.neurips.cc/paper/2004/file/96f2b50b5d3613adf9c27049b2a888c7-Paper.pdf) Theoretical work later showed the approach is effective in the large-sample regime: as the number of unlabeled examples grows, the model reaches the same optimal population error upper bound as supervised learning, even within one iteration, given a sufficiently accurate initial model.[10](https://ar5iv.labs.arxiv.org/html/2211.10039)\n\n## How it is done\n\nThe original recipe trains with a combined cross-entropy loss\n\n\\[ L = \\frac{1}{n}\\sum_{m} L(y_m, f_m) + \\alpha(t)\\,\\frac{1}{n'}\\sum_{i} L(\\tilde{y}_i, f_i), \\]\n\nwhere the pseudo-label weight \\( \\alpha(t) \\) is raised by deterministic annealing from a small value to \\( \\alpha_{\\mathrm{f}} = 3 \\), with \\( T_{1} = 100 \\) and \\( T_{2} = 600 \\) epochs without pre-training, or \\( T_{1} = 200 \\) and \\( T_{2} = 800 \\) with a denoising auto-encoder; a weight set too high disturbs labeled training, one too small yields no benefit.[2](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.664.3543) This deviates from the classical self-training wrapper, which fully retrains a classifier each round; Lee instead fine-tunes the running model and increases the pseudo-label weight over time because early pseudo-labels are less reliable.[11](https://link.springer.com/article/10.1007/s10994-019-05855-6)\n\nModern confidence-thresholded versions follow FixMatch: pseudo-labels are generated from weakly augmented unlabeled images and retained only when the maximum class probability exceeds a threshold τ, then enforced as cross-entropy against strongly augmented views,\n\n\\[ \\ell_u = \\frac{1}{\\mu B}\\sum_{b} \\mathbf{1}(\\max(q_b) \\geq \\tau)\\, H(\\hat{q}_b, p_m(y \\mid \\mathcal{A}(u_b))), \\]\n\nadded to the supervised loss with a fixed weight λ_u.[4](https://proceedings.neurips.cc/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Paper.pdf) FixMatch omits annealing of the unlabeled loss weight because thresholding itself provides a natural curriculum; on ImageNet it uses \\( \\lambda_u = 10 \\), \\( \\tau = 0.7 \\), and 300 epochs of unlabeled examples.[4](https://proceedings.neurips.cc/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Paper.pdf) • [5](https://papers.nips.cc/paper_files/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Supplemental.pdf)\n\n## Origin\n\nThe named deep-learning method is \"Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks\", presented at the ICML 2013 Workshop on Challenges in Representation Learning in Atlanta; the version without unsupervised pre-training earned second prize in that workshop's Black Box Learning Challenge.[2](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.664.3543) The method builds on a long self-training lineage in which a model retrains over multiple rounds on its own past predictions.[12](https://ojs.aaai.org/index.php/AAAI/article/view/16852) Closer precursors include unsupervised word sense disambiguation and co-training, in which two models trained on disjoint feature sets exchange confident predictions.[1](https://link.springer.com/article/10.1007/s10115-013-0706-y)\n\n## Variants\n\nA large family now extends the basic loop. MixMatch, by Berthelot, Carlini, Goodfellow, and colleagues (2019), is a named variant in this family;[13](https://doi.org/10.48550/arxiv.1905.02249) ReMixMatch (2019) added distribution alignment and augmentation anchoring.[14](https://doi.org/10.48550/arxiv.1911.09785) Noisy Student training, by Xie, Luong, Hovy, and Le (2019), iterates a teacher-student scheme in which a student trained on labeled and pseudo-labeled data becomes the teacher for a new, larger student trained with noise such as dropout and RandAugment; it differs from distillation by using unlabeled data and noise.[15](https://doi.org/10.48550/arxiv.1911.04252) • [3](https://openreview.net/pdf?id=cjivIeoSXz) Meta Pseudo Labels, by Pham, Dai, Xie, and colleagues (2020), adapts the teacher through the student's performance on labeled data, framed as bi-level optimization, addressing the confirmation bias of a fixed teacher.[16](https://doi.org/10.48550/arxiv.2003.10580) FixMatch, by Sohn, Berthelot, Li, and colleagues (2020), merged consistency regularization with confidence thresholding.[4](https://proceedings.neurips.cc/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Paper.pdf) FlexMatch, by Zhang, Wang, Hou, and colleagues (2021), adds Curriculum Pseudo Labeling, per-class dynamic thresholds lowered for hard classes and raised for easy ones with no extra forward passes.[17](https://doi.org/10.48550/arxiv.2110.08263) FreeMatch, by Wang, Chen, Heng, and colleagues (2022), replaces fixed thresholds with Self-Adaptive Thresholding, estimating a global threshold and class-specific thresholds as exponential moving averages of unlabeled-data confidence, plus a self-adaptive class fairness regularizer.[18](https://doi.org/10.48550/arxiv.2205.07246)\n\n## Applications\n\nReported results concentrate on image classification with tiny label budgets. FixMatch reaches 94.93% accuracy on CIFAR-10 with 250 labels and 88.61% with 40 labels, and on ImageNet with 10% of the labels a top-1 error of 28.54 ± 0.52%, 2.68% better than UDA.[4](https://proceedings.neurips.cc/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Paper.pdf) Noisy Student reaches 88.4% top-1 on ImageNet using 300M unlabeled images, of a 3.4% total gain over [EfficientNet](https://www.edgepedia.org/efficientnet) 2.9% comes from the self-training itself, and it improves ImageNet-A top-1 from 61.0% to 83.7% while cutting ImageNet-C mean corruption error from 45.7 to 28.3.[6](https://openaccess.thecvf.com/content_CVPR_2020/papers/Xie_Self-Training_With_Noisy_Student_Improves_ImageNet_Classification_CVPR_2020_paper.pdf) Meta Pseudo Labels reaches 90.2% top-1 on ImageNet, 1.6% above the previous record of 88.6%, and 96.11% on CIFAR-10 with 4,000 labels versus FixMatch's 95.74%.\n\nBeyond image classification, self-training approaches are used in NLP text classification, where one study found task-adaptive pre-training outperformed five self-training methods and warned of confirmation bias when labeled or unlabeled data is small or shifted.[19](https://aclanthology.org/2023.findings-acl.347.pdf) The USB open-source platform evaluates SSL algorithms across computer vision, NLP, and audio tasks.[20](https://research.chalmers.se/publication/544964/file/544964_Fulltext.pdf) Recent work extends pseudo-labeling to LLM reasoning, where a verifier trained on a small labeled set scores reasoning traces on unlabeled questions.[21](https://doi.org/10.3389/frai.2026.1857934)\n\n## Limitations and alternatives\n\n[Confirmation bias](https://www.edgepedia.org/confirmation-bias) is the central failure mode: incorrect early predictions are used as training targets in later epochs, increasing confidence in them and producing a model that resists correction.[8](https://ar5iv.labs.arxiv.org/html/1908.02983) Thresholds interact with miscalibration: high thresholds guard against wrong labels but imply excessive trust in confidence scores biased by the small labeled sample, and the optimal threshold differs at every iteration.[22](https://arxiv.org/pdf/2202.12040) With imbalanced or mismatched class distributions, pseudo-labeling becomes biased toward majority classes and can propagate errors catastrophically.[23](https://aclanthology.org/2025.emnlp-main.658.pdf) A controlled comparison by Oliver and colleagues found error rates typically declined as more unlabeled data was added, with degradation only when labeled and unlabeled class sets mismatched.[11](https://link.springer.com/article/10.1007/s10994-019-05855-6)\n\nRecent work addresses these weaknesses. An ICML 2025 framework learns confidence scores and thresholds from first principles with an explicit error knob, giving direct control over the pseudo-label quality-quantity trade-off,[24](https://proceedings.mlr.press/v267/vishwakarma25a.html) and imbalance-aware work formulates pseudo-labeling as optimal transport solved by Sinkhorn-Knopp, constrained by a class distribution estimated from a memory bank.[23](https://aclanthology.org/2025.emnlp-main.658.pdf) Channel-masking regularization guided by a mixture-proportion model improves pseudo-labeling under distribution shift across 36 settings of six benchmarks with no additional inference cost.[25](https://eprints.whiterose.ac.uk/id/eprint/214942/1/ECCV24_DA.pdf) In the foundation-model era, parameter-efficient fine-tuning of vision foundation models on labeled data alone often surpasses traditional SSL, and because different PEFT techniques yield complementary pseudo-labels, ensembling pseudo-labels across PEFT methods and backbones becomes a simple SSL strategy;[26](https://papers.nips.cc/paper_files/paper/2025/hash/558159930b584e7f137a1dcbd380d9fd-Abstract-Conference.html) FineSSL adapts foundation models for SSL with balanced margin softmax and decoupled label smoothing, setting a new state of the art on multiple benchmarks while cutting training cost by over six times.[27](https://proceedings.mlr.press/v235/gan24a.html)\n\nAgainst alternatives: consistency regularization, as in UDA, constrains predictions under augmentation, while FixMatch combines consistency regularization with a confidence-based thresholding mechanism that selects high-confidence pseudo-labeled examples for training;[4](https://proceedings.neurips.cc/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Paper.pdf) co-training requires multiple diverse classifiers exchanging predictions, and classical self-training wraps a fully retrained classifier rather than fine-tuning one.[11](https://link.springer.com/article/10.1007/s10994-019-05855-6)\n\n## References\n\n1. [Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study (Knowledge and Information Systems)](https://link.springer.com/article/10.1007/s10115-013-0706-y)\n2. [Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks (Dong-hyun Lee, ICML 2013 Workshop: Challenges in Representation Learning)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.664.3543)\n3. [A Review of Pseudo-Labeling for Computer Vision](https://openreview.net/pdf?id=cjivIeoSXz)\n4. [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence (Sohn et al., NeurIPS 2020)](https://proceedings.neurips.cc/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Paper.pdf)\n5. [FixMatch supplemental material](https://papers.nips.cc/paper_files/paper/2020/file/06964dce9addb1c5cb5d6e3d9838f733-Supplemental.pdf)\n6. [Self-Training With Noisy Student Improves ImageNet Classification (CVPR 2020, Xie et al.)](https://openaccess.thecvf.com/content_CVPR_2020/papers/Xie_Self-Training_With_Noisy_Student_Improves_ImageNet_Classification_CVPR_2020_paper.pdf)\n7. [Meta Pseudo Labels (CVPR 2021, Pham et al.)](https://openaccess.thecvf.com/content/CVPR2021/papers/Pham_Meta_Pseudo_Labels_CVPR_2021_paper.pdf)\n8. [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning (Arazo et al., 2019)](https://ar5iv.labs.arxiv.org/html/1908.02983)\n9. [Semi-supervised Learning by Entropy Minimization (Grandvalet & Bengio, NeurIPS 2004)](https://proceedings.neurips.cc/paper/2004/file/96f2b50b5d3613adf9c27049b2a888c7-Paper.pdf)\n10. [Why the pseudo label based semi-supervised learning algorithm is effective?](https://ar5iv.labs.arxiv.org/html/2211.10039)\n11. [A survey on semi-supervised learning (van Engelen & Hoos, Machine Learning, 2020)](https://link.springer.com/article/10.1007/s10994-019-05855-6)\n12. [Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning (AAAI 2021, Cascante-Bonilla, Tan, Qi, Ordonez)](https://ojs.aaai.org/index.php/AAAI/article/view/16852)\n13. [Berthelot, David and colleagues (2019). MixMatch: A Holistic Approach to Semi-Supervised Learning. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1905.02249)\n14. [Berthelot, David and colleagues (2019). ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1911.09785)\n15. [Xie, Qizhe and colleagues (2019). Self-training with Noisy Student improves ImageNet classification. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1911.04252)\n16. [Pham, Hieu and colleagues (2020). Meta Pseudo Labels. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.2003.10580)\n17. [Zhang, Bowen and colleagues (2021). FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.2110.08263)\n18. [Wang, Yidong and colleagues (2022). FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.2205.07246)\n19. [Rethinking Semi-supervised Learning with Language Models (Findings of ACL 2023)](https://aclanthology.org/2023.findings-acl.347.pdf)\n20. [An empirical evaluation of deep semi-supervised learning (Chalmers)](https://research.chalmers.se/publication/544964/file/544964_Fulltext.pdf)\n21. [Confidence-aware pseudo-label selection and verifier training for semi-supervised LLM reasoning with minimal labels (Frontiers in Artificial Intelligence, 2026)](https://doi.org/10.3389/frai.2026.1857934)\n22. [A survey on self-training methods (arXiv 2202.12040)](https://arxiv.org/pdf/2202.12040)\n23. [Calibrating Pseudo-Labeling with Class Distribution for Semi-supervised Text Classification (PL-POT, EMNLP 2025)](https://aclanthology.org/2025.emnlp-main.658.pdf)\n24. [Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL (ICML 2025, PMLR)](https://proceedings.mlr.press/v267/vishwakarma25a.html)\n25. [Pseudo-labelling should be aware of disguising channel activations (ECCV 2024)](https://eprints.whiterose.ac.uk/id/eprint/214942/1/ECCV24_DA.pdf)\n26. [Revisiting Semi-Supervised Learning in the Era of Foundation Models (NeurIPS 2025)](https://papers.nips.cc/paper_files/paper/2025/hash/558159930b584e7f137a1dcbd380d9fd-Abstract-Conference.html)\n27. [Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning (FineSSL, ICML 2024)](https://proceedings.mlr.press/v235/gan24a.html)\n\n---\n*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data › Machine learning and neural computation › Machine learning methods › Supervised, unsupervised, and semi-supervised learning › Semi-supervised and weakly supervised learning*\n\n*Initially written Sep 29, 2026 · Edited: Oct 15, 2026; Oct 25, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [], "url": "https://www.edgepedia.org/pseudo-labeling", "markdown_url": "https://www.edgepedia.org/pseudo-labeling.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Pseudo-labeling\", Edgepedia (EdgeChat), https://www.edgepedia.org/pseudo-labeling. Edgepedia Community License 1.0.", "credit_md": "\"[Pseudo-labeling](https://www.edgepedia.org/pseudo-labeling)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/pseudo-labeling](https://www.edgepedia.org/pseudo-labeling). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Pseudo-labeling\", Edgepedia (EdgeChat), https://www.edgepedia.org/pseudo-labeling. Edgepedia Community License 1.0.", "speakable": "Pseudo-labeling is a semi-supervised learning technique in which a model treats its most confident predictions on unlabeled data as true labels, introduced for deep networks by Dong-Hyun Lee in 2013." }