# Percy Liang
**Percy Liang** (Percy Shuo Liang) is an American computer scientist at Stanford University who works on machine learning and natural language processing, and who directs the Center for Research on Foundation Models (CRFM), the Stanford group that named and systematically studied the paradigm of large, broadly trained, adaptable AI models known as foundation models.[1](https://profiles.stanford.edu/percy-liang) His current work focuses on making foundation models, especially language models, more accessible through open-source tools and more understandable through rigorous benchmarking.[1](https://profiles.stanford.edu/percy-liang)
| Key fact | Detail |
|---|---|
| Position | Professor of Computer Science at Stanford University, with a courtesy appointment in Statistics; director of the Center for Research on Foundation Models (CRFM)[2](https://cs.stanford.edu/~pliang/) • [1](https://profiles.stanford.edu/percy-liang) |
| Education | B.S. MIT (2004), MEng MIT (2005), Ph.D. UC Berkeley (2011), advised by Michael Jordan and Dan Klein; post-doc at Google in 2012[2](https://cs.stanford.edu/~pliang/) |
| Foundation models | Initiated and conceptualized the 2021 CRFM report *On the Opportunities and Risks of Foundation Models*, written by more than 100 researchers[3](https://crfm.stanford.edu/assets/report.pdf) |
| HELM | Led Holistic Evaluation of Language Models (2022), benchmarking 30 prominent language models on 42 scenarios with 7 metrics, raising average scenario coverage from 17.9% to 96.0%[4](https://arxiv.org/abs/2211.09110) |
| Transparency | Co-created the Foundation Model Transparency Index (October 2023), which scored 10 major developers on 100 indicators; the top score was 54 out of 100 and the mean 37%[5](https://crfm.stanford.edu/fmti/October-2023/index.html) |
| Major awards | Presidential Early Career Award for Scientists and Engineers (2019), IJCAI Computers and Thought Award (2016), NSF CAREER Award (2016), Sloan Research Fellowship (2015), Microsoft Research Faculty Fellowship (2014)[6](https://reglab.stanford.edu/team-members/percy-liang/) |
| Teaching | Teaches CS336 (Language Models from Scratch), a course in which students build a language model end to end, from data collection through training and evaluation[7](https://cs336.stanford.edu/spring2025/) |
## Education and early career
Liang studied at MIT, completing a B.S. in 2004 and an MEng in 2005 with Michael Collins as advisor, and then earned his Ph.D. at UC Berkeley in 2011 under [Michael Jordan](https://www.edgepedia.org/michael-jordan) and Dan Klein.[2](https://cs.stanford.edu/~pliang/) His dissertation, *Learning Dependency-Based Compositional Semantics*, addressed a central problem in semantic parsing, the task of mapping natural language questions to logical forms that a system can execute against a database. Because annotating questions with logical forms is expensive, he developed a formalism called dependency-based compositional semantics (DCS) and a training procedure that learns the parser from question-answer pairs alone, treating the logical form as a latent variable; on two standard semantic parsing benchmarks the system outperformed existing state-of-the-art systems without using any annotated logical forms.[8](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-90.html)
After a 2012 post-doc at Google, he joined Stanford as an assistant professor in fall 2012.[2](https://cs.stanford.edu/~pliang/) • [9](https://www-dev.cs.stanford.edu/node/53) His early Stanford work spanned the theoretical and applied sides of machine learning and language, from learning theory to practical natural language understanding systems.[10](https://ai.stanford.edu/wp-content/uploads/2019/05/SAIL_Brochure_2019_LowRes.pdf)
## Semantic parsing and robustness
A recurring theme in Liang's research group has been evaluation that exposes what models cannot do. With Robin Jia, he proposed an adversarial evaluation scheme for the Stanford Question Answering Dataset (SQuAD) in 2017, adding automatically generated distractor sentences that do not change the correct answer or mislead humans. Under this adversarial setting, the accuracy of sixteen published reading comprehension models fell from an average of 75% F1 to 36%, and when the adversary could add ungrammatical word sequences, average accuracy on four models dropped further to 7%.[11](https://arxiv.org/abs/1707.07328)
His group also built the SEMPRE semantic parsing framework, used across a series of papers on question answering over Freebase and semi-structured tables, and released it as open source.[12](https://github.com/percyliang/sempre) A 2015 review with Christopher Potts, a Stanford professor of linguistics, argued that the long-standing divide between logical and statistical approaches to computational semantics was eroding as statistical models learned compositional semantic theories from corpora and databases.[13](https://www.annualreviews.org/content/journals/10.1146/annurev-linguist-030514-125312)
Focusing on robustness, Liang helped lead the WILDS effort (ICML 2021), a curated collection of 10 datasets that reflect the distribution shifts encountered in real-world settings—for example, differences between hospitals in tumor identification, between camera traps in wildlife monitoring, and between times and places in satellite imaging and poverty mapping. The benchmark's authors found that existing robustness algorithms did not consistently improve on standard empirical risk minimization, and the project maintains a public leaderboard for tracking progress.[14](https://arxiv.org/abs/2012.07421)
## Foundation models and the CRFM
In August 2021, Liang and colleagues at the newly launched Center for Research on Foundation Models, part of Stanford's Institute for Human-Centered AI (HAI), released a report by more than 100 researchers on the opportunities and risks of what they called foundation models: models such as BERT, DALL-E, and GPT-3 that are trained on broad data at scale and adapted to a wide range of downstream tasks. The report argued that their scale produces emergent capabilities and that their effectiveness across many tasks incentivizes homogenization, meaning defects in a foundation model are inherited by all adapted models downstream.[3](https://crfm.stanford.edu/assets/report.pdf) Liang initiated and conceptualized the framing and structure of the report.[3](https://crfm.stanford.edu/assets/report.pdf) In press coverage at the launch, he described the consolidation as a double-edged sword: centralization lets improvements to a few models propagate everywhere, but also makes those models single points of failure whose biases and defects are amplified in a way that everyone inherits.[15](https://www.axios.com/2021/08/18/foundation-ai-models-stanford)
## Evaluation: HELM and the Transparency Index
**HELM.** In November 2022 the CRFM released Holistic Evaluation of Language Models (HELM), a large-scale collaborative benchmark. HELM taxonomized the space of scenarios (use cases) and metrics, then measured 7 metrics, accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency, for each of 16 core scenarios when possible, and ran 7 targeted evaluations on aspects such as reasoning and disinformation. It evaluated 30 prominent language models spanning open, limited-access, and closed models on 42 scenarios. Before HELM, models on average had been evaluated on just 17.9% of the core scenarios, with some prominent models sharing no scenario in common; HELM raised this to 96.0%, and the team released all raw prompts and completions publicly.[4](https://arxiv.org/abs/2211.09110) HELM has since grown into a family of leaderboards covering safety, instruction following, vision-language models, text-to-image models, medical tasks, and multiple languages.[16](https://crfm.stanford.edu/helm/)
**Transparency Index.** In October 2023, the CRFM launched the Foundation Model Transparency Index, an assessment of 10 major foundation model developers against 100 indicators covering upstream resources (compute, data, labor), model properties, and downstream use. The top-scoring developer reached only 54 out of 100 and the mean score was 37%, although 82 of the indicators were satisfied by at least one developer, showing that better practices already existed in the industry. Developers scored relatively well on user data protection (67%) and basic model details (63%), but poorly on data, labor, and compute transparency, where all developers' scores summed to just 20%, 17%, and 17% of available points respectively.[5](https://crfm.stanford.edu/fmti/October-2023/index.html)
## Open development: Marin
Liang's more recent work pushes openness beyond releasing weights or code. He created Marin, a research program and open-source platform for developing foundation models in which every experiment, successful or failed, is preregistered and conducted in the open, so that anyone can suggest ideas, review, and run experiments through GitHub.[2](https://cs.stanford.edu/~pliang/) • [17](https://github.com/marin-community/marin) The project trained a fully open 8B foundation model with Google's JAX and TPUs and then Marin 32B, described by Google's open-source blog as an example of what an open lab can build.[18](https://opensource.googleblog.com/2025/12/training-marin-32b-what-an-open-lab-can-build-with-tpus-jax-and-a-little-persistence.html) In August 2026, Liang announced the start of open training for Marin 535B-A23B, a mixture-of-experts model with roughly 535 billion total parameters and about 23 billion active per token, with the code, data plan, scaling ladder, and live telemetry published from the outset.[17](https://github.com/marin-community/marin) • [19](https://x.noodl3.net/percyliang/status/2090918070617624811)
## Honors
Among the honors Liang has received are the Presidential Early Career Award for Scientists and Engineers (2019), the IJCAI Computers and Thought Award and an NSF CAREER Award, both from 2016, a 2015 Sloan Research Fellowship, a 2014 Microsoft Research Faculty Fellowship, and a 2022 Schmidt Sciences AI2050 Senior Fellowship; his papers have also won recognition at ACL, EMNLP, ICML, COLT, and additional venues.[6](https://reglab.stanford.edu/team-members/percy-liang/) • [2](https://cs.stanford.edu/~pliang/) The NSF's PECASE citation recognized his innovations in machine learning and semantic parsing for natural language processing, his commitment to efficient and reproducible research, and his AI education.[20](https://www.nsf.gov/honorary-awards/pecase/recipients/percy-liang) Before his research career, he placed second at the 2002 ACM ICPC World Finals and won a silver medal at the 2000 [International Olympiad in Informatics](https://www.edgepedia.org/international-olympiad-in-informatics).[2](https://cs.stanford.edu/~pliang/)
## References
1. [Percy Liang's Profile, Stanford Profiles](https://profiles.stanford.edu/percy-liang)
2. [Percy Liang, personal homepage, Stanford Computer Science](https://cs.stanford.edu/~pliang/)
3. [On the Opportunities and Risks of Foundation Models, CRFM report (2021)](https://crfm.stanford.edu/assets/report.pdf)
4. [Holistic Evaluation of Language Models, arXiv:2211.09110](https://arxiv.org/abs/2211.09110)
5. [Foundation Model Transparency Index, October 2023, Stanford CRFM](https://crfm.stanford.edu/fmti/October-2023/index.html)
6. [Percy Liang, Stanford RegLab](https://reglab.stanford.edu/team-members/percy-liang/)
7. [Stanford CS336: Language Modeling from Scratch, Spring 2025](https://cs336.stanford.edu/spring2025/)
8. [Learning Dependency-Based Compositional Semantics, UC Berkeley EECS-2011-90](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-90.html)
9. [Stanford Computer Science newsletter, 2010-11](https://www-dev.cs.stanford.edu/node/53)
10. [Stanford AI Lab brochure, 2019](https://ai.stanford.edu/wp-content/uploads/2019/05/SAIL_Brochure_2019_LowRes.pdf)
11. [Adversarial Examples for Evaluating Reading Comprehension Systems, arXiv:1707.07328](https://arxiv.org/abs/1707.07328)
12. [SEMPRE: Semantic Parser with Execution, GitHub](https://github.com/percyliang/sempre)
13. [Bringing Machine Learning and Compositional Semantics Together, Annual Review of Linguistics (2015)](https://www.annualreviews.org/content/journals/10.1146/annurev-linguist-030514-125312)
14. [WILDS: A Benchmark of in-the-Wild Distribution Shifts, arXiv:2012.07421](https://arxiv.org/abs/2012.07421)
15. [AI's shaky foundations, Axios (August 18, 2021)](https://www.axios.com/2021/08/18/foundation-ai-models-stanford)
16. [HELM leaderboards, Stanford CRFM](https://crfm.stanford.edu/helm/)
17. [Marin: open-source framework for foundation model research, GitHub](https://github.com/marin-community/marin)
18. [Training Marin 32B, Google Open Source Blog](https://opensource.googleblog.com/2025/12/training-marin-32b-what-an-open-lab-can-build-with-tpus-jax-and-a-little-persistence.html)
19. [Percy Liang announcement of Marin 535B-A23B training](https://x.noodl3.net/percyliang/status/2090918070617624811)
20. [Percy Liang, PECASE recipients, National Science Foundation](https://www.nsf.gov/honorary-awards/pecase/recipients/percy-liang)
---
*Topic: Encyclopedia › Technology and the built world › Engineers and computer scientists › Computer scientists and AI researchers › Researchers in artificial intelligence and machine learning*
*Initially written Sep 17, 2026 · Reviewed: Oct 15, 2026 · Edited: Oct 15, 2026 · Last review: Oct 15, 2026*
*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*
License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license