{ "id": "epyvfz7fs3", "slug": "empirical-likelihood", "title": "Empirical likelihood", "updated": "2026-10-15", "topic_path": [ { "id": "physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical" }, { "id": "physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math" }, { "id": "physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics" }, { "id": "physical.math.statistics.stats_statistical_inference", "label": "Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference" }, { "id": "physical.math.statistics.stats_statistical_inference.inference_theory", "label": "Foundations of statistical inference", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.inference_theory" }, { "id": "physical.math.statistics.stats_statistical_inference.inference_theory_overview", "label": "Statistical inference: overview", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.inference_theory_overview" } ], "geo": [ { "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference", "label": "Non-geographic · 1946 to 2000: Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t1946", "label": "Non-geographic · 1946 to 2000", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946" }, { "id": "geo.nongeo.t1946.physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical" }, { "id": "geo.nongeo.t1946.physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math" }, { "id": "geo.nongeo.t1946.physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics" }, { "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference", "label": "Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference" } ] } ], "excerpt": "Empirical likelihood is a nonparametric statistical method that assigns probability weights to observed data points and maximizes the likelihood under constraints, yielding tests and confidence regions without parametric assumptions.", "snippet": "Empirical likelihood is a nonparametric statistical method that assigns probability weights to observed data points and maximizes the likelihood under constraints, yielding tests and confidence regions without parametric assumptions.", "node": "physical.math.statistics.stats_statistical_inference.inference_theory_overview", "markdown": "# Empirical likelihood\n\nEmpirical likelihood is a nonparametric likelihood method in statistics that assigns probability weights to observed data points and maximizes the resulting likelihood under constraints, producing likelihood-ratio tests and confidence regions without assuming a parametric distribution for the data. Because the likelihood is built directly from the sample, the method is self-normalizing, so variance estimation is typically not needed, and the resulting confidence regions have a data-driven shape and respect the range of the observations.[1](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf)\n\n| Key fact | Detail |\n|---|---|\n| What it produces | Likelihood-ratio tests and confidence regions for means, M-estimates, smooth functionals, and parameters defined by estimating equations[2](https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf) |\n| Core optimization | Maximize \\( \\prod_{i=1}^{n} p_i \\) subject to \\( p_i > 0 \\), \\( \\sum_i p_i = 1 \\), and moment constraints \\( \\sum_i p_i g(X_i, \\theta) = 0 \\)[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf) |\n| Lagrange weights | \\( p_i = \\frac{1}{N\\{1 + \\lambda^{\\prime} g(X_i, \\theta)\\}} \\) for a multiplier \\( \\lambda \\), with \\( N \\) the total number of observations[4](https://arxiv.org/pdf/2512.08182) |\n| Calibration | \\( -2 \\log R \\) converges to a chi-squared distribution under moment conditions, a nonparametric analog of Wilks's theorem[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf) |\n| Accuracy | Bartlett correction reduces coverage error from \\( O(1/n) \\) to \\( O(1/n^2) \\)[6](https://www.artowen.su.domains/pubtalks/bayesel-annotated.pdf) |\n| Main limitation | Unadjusted regions are confined to the convex hull of the observed data, which fails in small samples[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf) |\n\n## How it works\n\nThe method treats the empirical distribution as a multinomial likelihood over the observed points. Each data point receives a probability weight \\( p_i \\), and the likelihood is \\( L(p) = \\prod_{i=1}^{n} p_i \\), maximized subject to \\( p_i > 0 \\) and \\( \\sum_i p_i = 1 \\); without constraints the optimum is \\( \\hat{p}_i = 1/n \\), the empirical distribution function.[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf) Side information enters as estimating equations \\( \\mathrm{E}\\{g(X;\\theta)\\} = 0 \\), and the likelihood is maximized over weights satisfying the sample version of these constraints.[7](https://doi.org/10.1214/aos/1176325370)\n\nThe profile empirical likelihood ratio compares the constrained and unconstrained optima: \\( R = \\sup\\{L(F) \\mid \\theta(F) = \\theta_0,\\, F \\in \\mathcal{F}\\} / \\sup\\{L(F) \\mid F \\in \\mathcal{F}\\} \\), where \\( \\mathcal{F} \\) is the set of distributions supported on the observed data.[1](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf) Owen's 1988 paper defines this ratio function as \\( R(F) = L(F)/L(F_n) \\), with \\( F_n \\) the empirical distribution function.[2](https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf)\n\nSolving the constrained problem with a [Lagrange multiplier](https://www.edgepedia.org/lagrange-multiplier) gives weights of the form \\( p_i = \\frac{1}{N\\{1 + \\lambda^{\\prime} g(X_i, \\theta)\\}} \\), with \\( N \\) the total number of observations.[4](https://arxiv.org/pdf/2512.08182) Under moment conditions, when \\( g(y,\\theta) = y - \\theta \\), the statistic \\( -2W(\\theta_0) \\) converges in distribution to \\( \\chi^2_m \\) as \\( n \\to \\infty \\), the nonparametric analog of Wilks's 1938 result for parametric likelihoods.[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)\n\nThe regions are range respecting, so interval bounds for a distribution function always lie within \\( [0,1] \\), unlike conventional normal-theory intervals[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf), and they are invariant under transformations and often outperform asymptotic-normality regions in small samples.[8](http://staff.ustc.edu.cn/~zwp/teach/nonpar/EL_for_reg.pdf) The method is Bartlett correctable, which reduces coverage errors from \\( O(1/n) \\) to \\( O(1/n^2) \\), and the likelihood-ratio tests have high power.[6](https://www.artowen.su.domains/pubtalks/bayesel-annotated.pdf)\n\n## How it is done\n\nA practitioner follows these steps:\n\n1. Choose estimating equations \\( g \\) that encode the parameter or hypothesis of interest, for example \\( g(X;\\theta) = X - \\theta \\) for a mean, or score-type equations in a regression model.\n2. For each candidate \\( \\theta \\), maximize the log empirical likelihood under the constraints. The Lagrange multiplier method reduces this maximization to a set of monotone equations for the multiplier \\( \\lambda \\)[9](https://www.ms.uky.edu/~mai/research/em.pdf), solvable by a Newton algorithm, since the inner problem is a low-dimensional convex maximization[10](https://www.cirje.e.u-tokyo.ac.jp/research/dp/2006/2006cf430.pdf); the adjusted-EL variant can be computed with a modified Newton–Raphson algorithm.[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)\n3. Calibrate the ratio statistic, either by the chi-squared limit or by bootstrap resampling; a hybrid method using the bootstrap to determine critical values of the likelihood ratio was introduced in the 1988 paper.[2](https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf)\n4. Invert the test to form a confidence region. For a regression parameter, a region of nominal level \\( 1 - \\alpha \\) is \\( I_{1-\\alpha} = \\{\\beta: r_n(\\beta) \\le \\chi^2_{p,1-\\alpha}\\} \\), where \\( r_n \\) is the profile log-likelihood ratio.[8](http://staff.ustc.edu.cn/~zwp/teach/nonpar/EL_for_reg.pdf)\n\n## Origin\n\nArt B. Owen introduced the method in \"Empirical likelihood ratio confidence intervals for a single functional\" ([Biometrika](https://www.edgepedia.org/biometrika), 1988), a paper that coined the term \"empirical likelihood\" and constructed nonparametric extensions of Wilks's theorem for the sample mean, M-estimates including quantiles, and differentiable statistical functionals.[11](https://doi.org/10.1093/biomet/75.2.237) Jin Qin and Jerry Lawless then extended the framework to general estimating equations in \"Empirical Likelihood and General Estimating Equations\" (The [Annals of Statistics](https://www.edgepedia.org/annals-of-statistics), 1994), showing that side information in the form of estimating equations improves the estimators and the ratio confidence intervals.[7](https://doi.org/10.1214/aos/1176325370) Owen's 2001 book consolidated the methodology across means, smooth functions of means, regression models, generalized linear models, and non-identically distributed data.[12](https://www.routledge.com/Empirical-Likelihood/Owen/p/book/9781584880714)\n\nThe idea of maximizing a likelihood over weights on data points has precedents the method built on: a weighting approach under the name of scale-load reasoning appeared in survey sampling, and profile-likelihood confidence intervals for survival probabilities under random right censoring were constructed in work connected with the [Kaplan–Meier estimator](https://www.edgepedia.org/kaplan-meier-estimator) before Owen's general formulation.[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf) • [9](https://www.ms.uky.edu/~mai/research/em.pdf) A related multinomial-weights construction is [Donald B. Rubin](https://www.edgepedia.org/donald-b-rubin)'s Bayesian bootstrap (The Annals of Statistics, 1981).[13](https://doi.org/10.1214/aos/1176345338)\n\n## Variants\n\nSeveral named extensions adapt the core construction:\n\n- **Regression and linear models.** The estimating-equation form covers regression parameters directly, with the log-likelihood and multiplier equations given above.[8](http://staff.ustc.edu.cn/~zwp/teach/nonpar/EL_for_reg.pdf)\n- **Bayesian empirical likelihood.** N. A. Lazar's \"Bayesian empirical likelihood\" (Biometrika, 2003) uses the EL ratio as a likelihood in a posterior, giving likelihood-ratio intervals and tests without parametric assumptions.[14](https://doi.org/10.1093/biomet/90.2.319)\n- **Exponential tilting and the GEL family.** A saddle-point, or exponential tilting, estimator maximizes a tilted profile likelihood under moment constraints; the generalized empirical likelihood (GEL) family, which includes EL, exponential tilting, and continuous updating estimators as special cases, connects EL to generalized minimum contrast estimation and to GMM.[10](https://www.cirje.e.u-tokyo.ac.jp/research/dp/2006/2006cf430.pdf)\n- **Empirical discrepancy statistics.** A general class of empirical discrepancy statistics, including the Cressie–Read discrepancies, generalizes the EL ratio statistic.[15](https://www.ism.ac.jp/editsec/aism/pdf/058_3_0427.pdf)\n- **Adjusted and bias-corrected EL.** Adding a pseudo-observation \\( g_{n+1} = -a_n \\bar{g}_n \\) ensures the constraints are always satisfiable, removing the convex-hull restriction.[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)\n- **Grouped EL for massive data.** A grouped empirical likelihood method divides \\( N \\) observations into \\( n \\) groups with equal weights within groups, reducing the optimization dimension from \\( N \\) to \\( n \\) while retaining first-order asymptotic properties.[4](https://arxiv.org/pdf/2512.08182)\n- **Extrapolated EL.** Extrapolated empirical likelihood addresses the convex-hull-violation problem in moment-condition models, with computation involving an inner search over nuisance parameters and an outer search over \\( \\theta \\).[16](https://orbilu.uni.lu/bitstream/10993/66943/1/DP2025-19%20Extrapolated%20empirical%20likelihood%20as%20a%20solution%20to%20the%20convex-hull-violation%20problem.pdf)\n\n## Applications\n\nEmpirical likelihood is used wherever likelihood-ratio inference is wanted without a parametric model. In survey sampling, empirical likelihood under simple random sampling with or without replacement has been applied, and estimating-equation side information such as auxiliary population data fits naturally into the framework.[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf) In econometrics, the generalized empirical likelihood class serves as an alternative to the generalized method of moments for models defined by moment condition restrictions, including tests of overidentifying restrictions.[17](https://www.annualreviews.org/content/journals/10.1146/annurev-economics-080511-110925) In biostatistics, the ratio with a mean constraint retains a chi-squared limit for right-censored data, and Wilks-type results hold for doubly censored data.[9](https://www.ms.uky.edu/~mai/research/em.pdf) The method has also been extended to time series data beyond the independent, identically distributed setting[18](https://www.sciencedirect.com/science/article/abs/pii/S0378375813002528), and published work applies high-dimensional EL to inference on average treatment effects.[19](https://academic.oup.com/biometrics/article/82/3/ujag149/8788688?searchresult=1)\n\n## Limitations and alternatives\n\nThe main failure modes are well characterized. Unadjusted EL confidence regions are confined to the convex hull of the observed values, a restriction unaffected by Bartlett correction or bootstrap calibration, which bites in small samples.[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf) The chi-squared calibration can fail in nonregular cases, such as when the true parameter lies on the boundary of the parameter space, where the limit may instead be a mixture of chi-squared distributions; nuisance parameters alone do not invalidate it, and the usual chi-squared limit holds under standard regularity conditions.[1](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf) In high dimensions where the dimension grows with sample size, normal-approximation calibration produces type I errors much larger than nominal levels, mainly from underestimation of centralized and normalized ELR quantities; a calibration based on the connection between the ELR and Hotelling's T-square statistic works much better in most situations.[20](https://www.ism.ac.jp/editsec/aism/pdf/10463_2012_Article_384.pdf) [Computation](https://www.edgepedia.org/computation) is also heavier than for competitors: the two-step generalized method of moments estimator is computationally simpler than the EL estimator and is widely used in econometrics[21](https://scholar.harvard.edu/files/imbens/files/empirical_likelihood_estimation_and_consistent_tests_with_conditional_moment_restrictions.pdf), and EL is more computationally intensive than Wald-type methods.[22](http://www.columbia.edu/~im2131/ps/Lecture-notes-EL.pdf) In small samples performance can differ substantially from the large-sample theory, though bootstrap resampling boosts it.[1](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf) The convex-hull problem is fixable by pseudo-data approaches such as adjusted EL.[6](https://www.artowen.su.domains/pubtalks/bayesel-annotated.pdf)\n\n## References\n\n1. [Empirical Likelihood (Annual Review of Statistics and Its Application, 2025 review, Columbia-hosted PDF)](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf)\n2. [Empirical likelihood ratio confidence intervals for a single functional (Owen, 1988, Biometrika)](https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf)\n3. [Empirical Likelihood Methods (Chen & Wu, Handbook of Statistics chapter)](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf)\n4. [Grouped empirical likelihood for massive data (preprint)](https://arxiv.org/pdf/2512.08182)\n5. [Adjusted Empirical Likelihood and its Properties (Chen et al., Journal of Computational and Graphical Statistics)](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)\n6. [Bayesian Empirical Likelihood (annotated talk slides, Art Owen)](https://www.artowen.su.domains/pubtalks/bayesel-annotated.pdf)\n7. [Jin Qin, Jerry Lawless (1994). Empirical Likelihood and General Estimating Equations. The Annals of Statistics.](https://doi.org/10.1214/aos/1176325370)\n8. [A Review on Empirical Likelihood Methods for Regression](http://staff.ustc.edu.cn/~zwp/teach/nonpar/EL_for_reg.pdf)\n9. [Notes on empirical likelihood computation (Mai, Zhou)](https://www.ms.uky.edu/~mai/research/em.pdf)\n10. [Empirical Likelihood Methods in Econometrics: Theory and Practice (Kitamura, CIRJE discussion paper / Cowles version)](https://www.cirje.e.u-tokyo.ac.jp/research/dp/2006/2006cf430.pdf)\n11. [ART B. OWEN (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika.](https://doi.org/10.1093/biomet/75.2.237)\n12. [Empirical Likelihood (Owen, 2001, Chapman & Hall/CRC)](https://www.routledge.com/Empirical-Likelihood/Owen/p/book/9781584880714)\n13. [Donald B. Rubin (1981). The Bayesian Bootstrap. The Annals of Statistics.](https://doi.org/10.1214/aos/1176345338)\n14. [N. A. Lazar (2003). Bayesian empirical likelihood. Biometrika.](https://doi.org/10.1093/biomet/90.2.319)\n15. [Asymptotic results on a general class of empirical statistics (Annals of the Institute of Statistical Mathematics)](https://www.ism.ac.jp/editsec/aism/pdf/058_3_0427.pdf)\n16. [Extrapolated empirical likelihood as a solution to the convex-hull-violation problem (Discussion Paper 2025-19)](https://orbilu.uni.lu/bitstream/10993/66943/1/DP2025-19%20Extrapolated%20empirical%20likelihood%20as%20a%20solution%20to%20the%20convex-hull-violation%20problem.pdf)\n17. [Recent Developments in Empirical Likelihood and Related Methods (Annual Review of Economics)](https://www.annualreviews.org/content/journals/10.1146/annurev-economics-080511-110925)\n18. [A review of empirical likelihood methods for time series (Computational Statistics & Data Analysis)](https://www.sciencedirect.com/science/article/abs/pii/S0378375813002528)\n19. [Multiply robust inference of average treatment effects by high-dimensional empirical likelihood (Biometrics)](https://academic.oup.com/biometrics/article/82/3/ujag149/8788688?searchresult=1)\n20. [Calibration of the empirical likelihood for high-dimensional data (Annals of the Institute of Statistical Mathematics)](https://www.ism.ac.jp/editsec/aism/pdf/10463_2012_Article_384.pdf)\n21. [Empirical Likelihood Estimation and Consistent Tests with Conditional Moment Restrictions (Imbens, Journal of Econometrics)](https://scholar.harvard.edu/files/imbens/files/empirical_likelihood_estimation_and_consistent_tests_with_conditional_moment_restrictions.pdf)\n22. [Introduction to Empirical Likelihood (Columbia lecture notes)](http://www.columbia.edu/~im2131/ps/Lecture-notes-EL.pdf)\n\n---\n*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Statistical inference, estimation, sampling, and testing › Foundations of statistical inference › Statistical inference: overview*\n\n*Initially written Sep 29, 2026 · Edited: Oct 15, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [ "http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf", "https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf", "https://www.ms.uky.edu/~mai/research/em.pdf", "http://www.columbia.edu/~im2131/ps/Lecture-notes-EL.pdf" ], "url": "https://www.edgepedia.org/empirical-likelihood", "markdown_url": "https://www.edgepedia.org/empirical-likelihood.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Empirical likelihood\", Edgepedia (EdgeChat), https://www.edgepedia.org/empirical-likelihood. Edgepedia Community License 1.0.", "credit_md": "\"[Empirical likelihood](https://www.edgepedia.org/empirical-likelihood)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/empirical-likelihood](https://www.edgepedia.org/empirical-likelihood). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Empirical likelihood\", Edgepedia (EdgeChat), https://www.edgepedia.org/empirical-likelihood. Edgepedia Community License 1.0.", "speakable": "Empirical likelihood is a nonparametric statistical method that assigns probability weights to observed data points and maximizes the likelihood under constraints, yielding tests and confidence regions without parametric assumptions." }