# Empirical likelihood

Empirical likelihood is a nonparametric likelihood method in statistics that assigns probability weights to observed data points and maximizes the resulting likelihood under constraints, producing likelihood-ratio tests and confidence regions without assuming a parametric distribution for the data. Because the likelihood is built directly from the sample, the method is self-normalizing, so variance estimation is typically not needed, and the resulting confidence regions have a data-driven shape and respect the range of the observations.<sup>[1](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf)</sup>

| Key fact | Detail |
|---|---|
| What it produces | Likelihood-ratio tests and confidence regions for means, M-estimates, smooth functionals, and parameters defined by estimating equations<sup>[2](https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf)</sup> |
| Core optimization | Maximize \( \prod_{i=1}^{n} p_i \) subject to \( p_i > 0 \), \( \sum_i p_i = 1 \), and moment constraints \( \sum_i p_i g(X_i, \theta) = 0 \)<sup>[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf)</sup> |
| Lagrange weights | \( p_i = \frac{1}{N\{1 + \lambda^{\prime} g(X_i, \theta)\}} \) for a multiplier \( \lambda \), with \( N \) the total number of observations<sup>[4](https://arxiv.org/pdf/2512.08182)</sup> |
| Calibration | \( -2 \log R \) converges to a chi-squared distribution under moment conditions, a nonparametric analog of Wilks's theorem<sup>[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)</sup> |
| Accuracy | Bartlett correction reduces coverage error from \( O(1/n) \) to \( O(1/n^2) \)<sup>[6](https://www.artowen.su.domains/pubtalks/bayesel-annotated.pdf)</sup> |
| Main limitation | Unadjusted regions are confined to the convex hull of the observed data, which fails in small samples<sup>[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)</sup> |

## How it works

The method treats the empirical distribution as a multinomial likelihood over the observed points. Each data point receives a probability weight \( p_i \), and the likelihood is \( L(p) = \prod_{i=1}^{n} p_i \), maximized subject to \( p_i > 0 \) and \( \sum_i p_i = 1 \); without constraints the optimum is \( \hat{p}_i = 1/n \), the empirical distribution function.<sup>[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf)</sup> Side information enters as estimating equations \( \mathrm{E}\{g(X;\theta)\} = 0 \), and the likelihood is maximized over weights satisfying the sample version of these constraints.<sup>[7](https://doi.org/10.1214/aos/1176325370)</sup>

The profile empirical likelihood ratio compares the constrained and unconstrained optima: \( R = \sup\{L(F) \mid \theta(F) = \theta_0,\, F \in \mathcal{F}\} / \sup\{L(F) \mid F \in \mathcal{F}\} \), where \( \mathcal{F} \) is the set of distributions supported on the observed data.<sup>[1](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf)</sup> Owen's 1988 paper defines this ratio function as \( R(F) = L(F)/L(F_n) \), with \( F_n \) the empirical distribution function.<sup>[2](https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf)</sup>

Solving the constrained problem with a [Lagrange multiplier](https://www.edgepedia.org/lagrange-multiplier) gives weights of the form \( p_i = \frac{1}{N\{1 + \lambda^{\prime} g(X_i, \theta)\}} \), with \( N \) the total number of observations.<sup>[4](https://arxiv.org/pdf/2512.08182)</sup> Under moment conditions, when \( g(y,\theta) = y - \theta \), the statistic \( -2W(\theta_0) \) converges in distribution to \( \chi^2_m \) as \( n \to \infty \), the nonparametric analog of Wilks's 1938 result for parametric likelihoods.<sup>[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)</sup>

The regions are range respecting, so interval bounds for a distribution function always lie within \( [0,1] \), unlike conventional normal-theory intervals<sup>[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf)</sup>, and they are invariant under transformations and often outperform asymptotic-normality regions in small samples.<sup>[8](http://staff.ustc.edu.cn/~zwp/teach/nonpar/EL_for_reg.pdf)</sup> The method is Bartlett correctable, which reduces coverage errors from \( O(1/n) \) to \( O(1/n^2) \), and the likelihood-ratio tests have high power.<sup>[6](https://www.artowen.su.domains/pubtalks/bayesel-annotated.pdf)</sup>

## How it is done

A practitioner follows these steps:

1. Choose estimating equations \( g \) that encode the parameter or hypothesis of interest, for example \( g(X;\theta) = X - \theta \) for a mean, or score-type equations in a regression model.
2. For each candidate \( \theta \), maximize the log empirical likelihood under the constraints. The Lagrange multiplier method reduces this maximization to a set of monotone equations for the multiplier \( \lambda \)<sup>[9](https://www.ms.uky.edu/~mai/research/em.pdf)</sup>, solvable by a Newton algorithm, since the inner problem is a low-dimensional convex maximization<sup>[10](https://www.cirje.e.u-tokyo.ac.jp/research/dp/2006/2006cf430.pdf)</sup>; the adjusted-EL variant can be computed with a modified Newton–Raphson algorithm.<sup>[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)</sup>
3. Calibrate the ratio statistic, either by the chi-squared limit or by bootstrap resampling; a hybrid method using the bootstrap to determine critical values of the likelihood ratio was introduced in the 1988 paper.<sup>[2](https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf)</sup>
4. Invert the test to form a confidence region. For a regression parameter, a region of nominal level \( 1 - \alpha \) is \( I_{1-\alpha} = \{\beta: r_n(\beta) \le \chi^2_{p,1-\alpha}\} \), where \( r_n \) is the profile log-likelihood ratio.<sup>[8](http://staff.ustc.edu.cn/~zwp/teach/nonpar/EL_for_reg.pdf)</sup>

## Origin

Art B. Owen introduced the method in "Empirical likelihood ratio confidence intervals for a single functional" ([Biometrika](https://www.edgepedia.org/biometrika), 1988), a paper that coined the term "empirical likelihood" and constructed nonparametric extensions of Wilks's theorem for the sample mean, M-estimates including quantiles, and differentiable statistical functionals.<sup>[11](https://doi.org/10.1093/biomet/75.2.237)</sup> Jin Qin and Jerry Lawless then extended the framework to general estimating equations in "Empirical Likelihood and General Estimating Equations" (The [Annals of Statistics](https://www.edgepedia.org/annals-of-statistics), 1994), showing that side information in the form of estimating equations improves the estimators and the ratio confidence intervals.<sup>[7](https://doi.org/10.1214/aos/1176325370)</sup> Owen's 2001 book consolidated the methodology across means, smooth functions of means, regression models, generalized linear models, and non-identically distributed data.<sup>[12](https://www.routledge.com/Empirical-Likelihood/Owen/p/book/9781584880714)</sup>

The idea of maximizing a likelihood over weights on data points has precedents the method built on: a weighting approach under the name of scale-load reasoning appeared in survey sampling, and profile-likelihood confidence intervals for survival probabilities under random right censoring were constructed in work connected with the [Kaplan–Meier estimator](https://www.edgepedia.org/kaplan-meier-estimator) before Owen's general formulation.<sup>[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf)</sup><sup> • </sup><sup>[9](https://www.ms.uky.edu/~mai/research/em.pdf)</sup> A related multinomial-weights construction is [Donald B. Rubin](https://www.edgepedia.org/donald-b-rubin)'s Bayesian bootstrap (The Annals of Statistics, 1981).<sup>[13](https://doi.org/10.1214/aos/1176345338)</sup>

## Variants

Several named extensions adapt the core construction:

- **Regression and linear models.** The estimating-equation form covers regression parameters directly, with the log-likelihood and multiplier equations given above.<sup>[8](http://staff.ustc.edu.cn/~zwp/teach/nonpar/EL_for_reg.pdf)</sup>
- **Bayesian empirical likelihood.** N. A. Lazar's "Bayesian empirical likelihood" (Biometrika, 2003) uses the EL ratio as a likelihood in a posterior, giving likelihood-ratio intervals and tests without parametric assumptions.<sup>[14](https://doi.org/10.1093/biomet/90.2.319)</sup>
- **Exponential tilting and the GEL family.** A saddle-point, or exponential tilting, estimator maximizes a tilted profile likelihood under moment constraints; the generalized empirical likelihood (GEL) family, which includes EL, exponential tilting, and continuous updating estimators as special cases, connects EL to generalized minimum contrast estimation and to GMM.<sup>[10](https://www.cirje.e.u-tokyo.ac.jp/research/dp/2006/2006cf430.pdf)</sup>
- **Empirical discrepancy statistics.** A general class of empirical discrepancy statistics, including the Cressie–Read discrepancies, generalizes the EL ratio statistic.<sup>[15](https://www.ism.ac.jp/editsec/aism/pdf/058_3_0427.pdf)</sup>
- **Adjusted and bias-corrected EL.** Adding a pseudo-observation \( g_{n+1} = -a_n \bar{g}_n \) ensures the constraints are always satisfiable, removing the convex-hull restriction.<sup>[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)</sup>
- **Grouped EL for massive data.** A grouped empirical likelihood method divides \( N \) observations into \( n \) groups with equal weights within groups, reducing the optimization dimension from \( N \) to \( n \) while retaining first-order asymptotic properties.<sup>[4](https://arxiv.org/pdf/2512.08182)</sup>
- **Extrapolated EL.** Extrapolated empirical likelihood addresses the convex-hull-violation problem in moment-condition models, with computation involving an inner search over nuisance parameters and an outer search over \( \theta \).<sup>[16](https://orbilu.uni.lu/bitstream/10993/66943/1/DP2025-19%20Extrapolated%20empirical%20likelihood%20as%20a%20solution%20to%20the%20convex-hull-violation%20problem.pdf)</sup>

## Applications

Empirical likelihood is used wherever likelihood-ratio inference is wanted without a parametric model. In survey sampling, empirical likelihood under simple random sampling with or without replacement has been applied, and estimating-equation side information such as auxiliary population data fits naturally into the framework.<sup>[3](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf)</sup> In econometrics, the generalized empirical likelihood class serves as an alternative to the generalized method of moments for models defined by moment condition restrictions, including tests of overidentifying restrictions.<sup>[17](https://www.annualreviews.org/content/journals/10.1146/annurev-economics-080511-110925)</sup> In biostatistics, the ratio with a mean constraint retains a chi-squared limit for right-censored data, and Wilks-type results hold for doubly censored data.<sup>[9](https://www.ms.uky.edu/~mai/research/em.pdf)</sup> The method has also been extended to time series data beyond the independent, identically distributed setting<sup>[18](https://www.sciencedirect.com/science/article/abs/pii/S0378375813002528)</sup>, and published work applies high-dimensional EL to inference on average treatment effects.<sup>[19](https://academic.oup.com/biometrics/article/82/3/ujag149/8788688?searchresult=1)</sup>

## Limitations and alternatives

The main failure modes are well characterized. Unadjusted EL confidence regions are confined to the convex hull of the observed values, a restriction unaffected by Bartlett correction or bootstrap calibration, which bites in small samples.<sup>[5](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)</sup> The chi-squared calibration can fail in nonregular cases, such as when the true parameter lies on the boundary of the parameter space, where the limit may instead be a mixture of chi-squared distributions; nuisance parameters alone do not invalidate it, and the usual chi-squared limit holds under standard regularity conditions.<sup>[1](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf)</sup> In high dimensions where the dimension grows with sample size, normal-approximation calibration produces type I errors much larger than nominal levels, mainly from underestimation of centralized and normalized ELR quantities; a calibration based on the connection between the ELR and Hotelling's T-square statistic works much better in most situations.<sup>[20](https://www.ism.ac.jp/editsec/aism/pdf/10463_2012_Article_384.pdf)</sup> [Computation](https://www.edgepedia.org/computation) is also heavier than for competitors: the two-step generalized method of moments estimator is computationally simpler than the EL estimator and is widely used in econometrics<sup>[21](https://scholar.harvard.edu/files/imbens/files/empirical_likelihood_estimation_and_consistent_tests_with_conditional_moment_restrictions.pdf)</sup>, and EL is more computationally intensive than Wald-type methods.<sup>[22](http://www.columbia.edu/~im2131/ps/Lecture-notes-EL.pdf)</sup> In small samples performance can differ substantially from the large-sample theory, though bootstrap resampling boosts it.<sup>[1](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf)</sup> The convex-hull problem is fixable by pseudo-data approaches such as adjusted EL.<sup>[6](https://www.artowen.su.domains/pubtalks/bayesel-annotated.pdf)</sup>

## References

1. [Empirical Likelihood (Annual Review of Statistics and Its Application, 2025 review, Columbia-hosted PDF)](http://www.columbia.edu/~im2131/ps/annurev-statistics.pdf)
2. [Empirical likelihood ratio confidence intervals for a single functional (Owen, 1988, Biometrika)](https://www.ms.uky.edu/~mai/sta709/Owen1988.pdf)
3. [Empirical Likelihood Methods (Chen & Wu, Handbook of Statistics chapter)](https://sas.uwaterloo.ca/~cbwu/paper/08handbook-1.pdf)
4. [Grouped empirical likelihood for massive data (preprint)](https://arxiv.org/pdf/2512.08182)
5. [Adjusted Empirical Likelihood and its Properties (Chen et al., Journal of Computational and Graphical Statistics)](https://www.stat.ubc.ca/~jhchen/paper/JCGS08.pdf)
6. [Bayesian Empirical Likelihood (annotated talk slides, Art Owen)](https://www.artowen.su.domains/pubtalks/bayesel-annotated.pdf)
7. [Jin Qin, Jerry Lawless (1994). Empirical Likelihood and General Estimating Equations. The Annals of Statistics.](https://doi.org/10.1214/aos/1176325370)
8. [A Review on Empirical Likelihood Methods for Regression](http://staff.ustc.edu.cn/~zwp/teach/nonpar/EL_for_reg.pdf)
9. [Notes on empirical likelihood computation (Mai, Zhou)](https://www.ms.uky.edu/~mai/research/em.pdf)
10. [Empirical Likelihood Methods in Econometrics: Theory and Practice (Kitamura, CIRJE discussion paper / Cowles version)](https://www.cirje.e.u-tokyo.ac.jp/research/dp/2006/2006cf430.pdf)
11. [ART B. OWEN (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika.](https://doi.org/10.1093/biomet/75.2.237)
12. [Empirical Likelihood (Owen, 2001, Chapman & Hall/CRC)](https://www.routledge.com/Empirical-Likelihood/Owen/p/book/9781584880714)
13. [Donald B. Rubin (1981). The Bayesian Bootstrap. The Annals of Statistics.](https://doi.org/10.1214/aos/1176345338)
14. [N. A. Lazar (2003). Bayesian empirical likelihood. Biometrika.](https://doi.org/10.1093/biomet/90.2.319)
15. [Asymptotic results on a general class of empirical statistics (Annals of the Institute of Statistical Mathematics)](https://www.ism.ac.jp/editsec/aism/pdf/058_3_0427.pdf)
16. [Extrapolated empirical likelihood as a solution to the convex-hull-violation problem (Discussion Paper 2025-19)](https://orbilu.uni.lu/bitstream/10993/66943/1/DP2025-19%20Extrapolated%20empirical%20likelihood%20as%20a%20solution%20to%20the%20convex-hull-violation%20problem.pdf)
17. [Recent Developments in Empirical Likelihood and Related Methods (Annual Review of Economics)](https://www.annualreviews.org/content/journals/10.1146/annurev-economics-080511-110925)
18. [A review of empirical likelihood methods for time series (Computational Statistics & Data Analysis)](https://www.sciencedirect.com/science/article/abs/pii/S0378375813002528)
19. [Multiply robust inference of average treatment effects by high-dimensional empirical likelihood (Biometrics)](https://academic.oup.com/biometrics/article/82/3/ujag149/8788688?searchresult=1)
20. [Calibration of the empirical likelihood for high-dimensional data (Annals of the Institute of Statistical Mathematics)](https://www.ism.ac.jp/editsec/aism/pdf/10463_2012_Article_384.pdf)
21. [Empirical Likelihood Estimation and Consistent Tests with Conditional Moment Restrictions (Imbens, Journal of Econometrics)](https://scholar.harvard.edu/files/imbens/files/empirical_likelihood_estimation_and_consistent_tests_with_conditional_moment_restrictions.pdf)
22. [Introduction to Empirical Likelihood (Columbia lecture notes)](http://www.columbia.edu/~im2131/ps/Lecture-notes-EL.pdf)

---
*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Statistical inference, estimation, sampling, and testing › Foundations of statistical inference › Statistical inference: overview*

*Initially written Sep 29, 2026 · Edited: Oct 15, 2026*

*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*

License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license