{ "id": "ep9zchjm7s", "slug": "variational-bayesian-inference", "title": "Variational Bayesian inference", "updated": "2026-10-15", "topic_path": [ { "id": "physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical" }, { "id": "physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math" }, { "id": "physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics" }, { "id": "physical.math.statistics.stats_bayesian", "label": "Bayesian statistics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_bayesian" }, { "id": "physical.math.statistics.stats_bayesian.bayes-computation", "label": "Bayesian computation and software", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_bayesian.bayes-computation" }, { "id": "physical.math.statistics.stats_bayesian.bayes-variational-approximate", "label": "Variational and approximate Bayesian methods", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_bayesian.bayes-variational-approximate" } ], "geo": [ { "id": "geo.nongeo.t1946.physical.math.statistics", "label": "Non-geographic · 1946 to 2000: Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t1946", "label": "Non-geographic · 1946 to 2000", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946" }, { "id": "geo.nongeo.t1946.physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical" }, { "id": "geo.nongeo.t1946.physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math" }, { "id": "geo.nongeo.t1946.physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics" } ] }, { "id": "geo.nongeo.t2001.physical.math.statistics", "label": "Non-geographic · 2001 to 2020: Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.physical.math.statistics", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t2001", "label": "Non-geographic · 2001 to 2020", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001" }, { "id": "geo.nongeo.t2001.physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.physical" }, { "id": "geo.nongeo.t2001.physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.physical.math" }, { "id": "geo.nongeo.t2001.physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.physical.math.statistics" } ] } ], "excerpt": "Variational Bayesian inference (VI) approximates an intractable posterior distribution by optimization, maximizing the ELBO to find the closest density in KL divergence; it is faster than MCMC but generally underestimates posterior variance.", "snippet": "Variational Bayesian inference (VI) approximates an intractable posterior distribution by optimization, maximizing the ELBO to find the closest density in KL divergence; it is faster than MCMC but generally underestimates posterior variance.", "node": "physical.math.statistics.stats_bayesian.bayes-variational-approximate", "markdown": "# Variational Bayesian inference\n\nVariational Bayesian inference (VI) approximates an intractable posterior probability distribution by turning inference into optimization: it posits a family of densities and finds the member closest to the target, with closeness measured by Kullback–Leibler (KL) divergence.[1](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773) The output is a density, not samples. VI tends to be faster than [Markov chain Monte Carlo](https://www.edgepedia.org/markov-chain-monte-carlo) (MCMC) sampling, but unlike MCMC it provides no guarantee of asymptotically exact samples from the target density.[1](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773) In many applications it has been observed to run orders of magnitude faster than MCMC for the same approximation accuracy,[2](https://ar5iv.labs.arxiv.org/html/1710.03266)\n\n| Key fact | Detail |\n|---|---|\n| Output | A density \\( q \\) in a chosen family, closest in KL divergence to the posterior[1](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773) |\n| Objective | The ELBO, \\( \\mathcal{L} = \\log p(x) - D_{KL}(q \\| p) \\); maximizing it is equivalent to minimizing the KL divergence[3](https://www.deeplearningbook.org/contents/inference.html) |\n| Classic family | Mean-field factorization \\( q(\\theta) = \\prod_{i} q_{i}(\\theta_{i}) \\)[4](https://robots.ox.ac.uk/~sjrob/Pubs/vbTutorialFinal.pdf) |\n| Classic algorithm | Coordinate ascent (CAVI), which converges to a local optimum sensitive to initialization[1](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773) |\n| Demonstrated scale | 300K Nature, 1.8M New York Times, and 3.8M Wikipedia articles with stochastic variational inference[5](https://www.cs.columbia.edu/~blei/papers/HoffmanBleiWangPaisley2013.pdf) |\n| Known bias | Generally underestimates posterior variance, a consequence of its objective function[1](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773) |\n\n## How it works\n\nThe evidence lower bound (ELBO) is defined as \\( \\mathcal{L}(v, \\theta, q) = \\log p(v; \\theta) - D_{KL}(q(h \\mid v) \\| p(h \\mid v; \\theta)) \\), where \\( q \\) is an arbitrary distribution over the latent variables \\( h \\). It is at most the log probability of the data and equals it only when \\( q \\) is the true posterior, so maximizing the ELBO drives \\( q \\) toward the posterior.[3](https://www.deeplearningbook.org/contents/inference.html) The bound can also be derived from \\( \\log p(x) \\) using [Jensen's inequality](https://www.edgepedia.org/jensens-inequality).[6](https://arxiv.org/pdf/1711.05597v3.pdf)\n\nThe reverse KL is the practical choice: VI minimizes \\( D_{KL}(q \\| p) \\) rather than \\( D_{KL}(p \\| q) \\) because expectations in the ELBO are taken under \\( q_{\\phi} \\), which can be sampled, whereas the forward KL would require expectations under the intractable posterior.[7](https://drawinginferences.com/chapters/vi.html) Algorithms divide into mean-field VB, which factorizes \\( q \\), and fixed-form VB, in which \\( q \\) is a parametric family indexed by a variational parameter \\( \\lambda \\).[8](https://ar5iv.labs.arxiv.org/html/2103.01327)\n\n## How it is done\n\nIn mean-field VB the factorization \\( q(\\theta) = q_{1}(\\theta_{1}) q_{2}(\\theta_{2}) \\) yields coordinate ascent updates \\( q_{1}(\\theta_{1}) \\propto \\exp(\\mathbb{E}_{q_{2}}[\\log p(y, \\theta)]) \\) and \\( q_{2}(\\theta_{2}) \\propto \\exp(\\mathbb{E}_{q_{1}}[\\log p(y, \\theta)]) \\), each factor requiring an expectation over all the other factors; the lower bound increases each iteration.[8](https://ar5iv.labs.arxiv.org/html/2103.01327) In conditionally conjugate models these updates resemble EM: the \"E step\" computes approximate conditionals of local latent variables and the \"M step\" a conditional of the global latent variable.[1](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773) For a global parameter \\( \\lambda \\) and local parameters \\( \\phi_{i} \\), the natural gradient of the ELBO is \\( \\hat{\\nabla}_{\\lambda} \\mathcal{L} = \\mathbb{E}_{\\phi}[\\eta(Z, x)] - \\lambda \\), and updating with it can improve optimization, though under suitable regularity and step-size assumptions it may converge only to a stationary point, not a guaranteed local optimum in general.[5](https://www.cs.columbia.edu/~blei/papers/HoffmanBleiWangPaisley2013.pdf)\n\n**Stochastic variational inference (SVI)** repeatedly subsamples the data to form noisy natural-gradient estimates and follows them with a decreasing step size, \\( \\lambda^{(t)} = \\lambda^{(t-1)} + \\rho_{t} G_{t}^{-1} b_{t}(\\lambda^{(t-1)}) \\), using the Fisher metric; the learning rate must satisfy the Robbins–Monro conditions \\( \\sum_{t} \\rho_{t} = \\infty \\), \\( \\sum_{t} \\rho_{t}^{2} < \\infty \\).[5](https://www.cs.columbia.edu/~blei/papers/HoffmanBleiWangPaisley2013.pdf) • [6](https://arxiv.org/pdf/1711.05597v3.pdf) The reparameterization trick writes \\( z = g_{\\phi}(\\epsilon) \\) with \\( \\epsilon \\sim p(\\epsilon) \\) independent of \\( \\phi \\), for example \\( g_{\\phi}(\\epsilon) = \\mu + \\sigma \\cdot \\epsilon \\) for a Gaussian, letting gradients pass through the expectation; plugging the estimator into any stochastic optimizer (Adam, SGD) gives SVI in systems such as Pyro.[7](https://drawinginferences.com/chapters/vi.html) Automatic differentiation variational inference (ADVI) generalizes this: it transforms constrained latent variables to real coordinate space, estimates the ELBO by [Monte Carlo integration](https://www.edgepedia.org/monte-carlo-integration), and optimizes with stochastic gradient ascent, with no conjugacy assumptions.[9](https://papers.nips.cc/paper_files/paper/2015/file/352fe25daf686bdb4edca223c921acea-Paper.pdf) • [10](https://doi.org/10.48550/arxiv.1603.00788) ADVI has per-iteration complexity \\( O(2N \\cdot M \\cdot K) \\) with \\( M \\) [Monte Carlo](https://www.edgepedia.org/monte-carlo) samples, typically 1 to 10, and \\( O(2B \\cdot M \\cdot K) \\) with minibatch size \\( B \\); its gradient estimator has lower variance than black-box VI's, so a single sample often suffices.[9](https://papers.nips.cc/paper_files/paper/2015/file/352fe25daf686bdb4edca223c921acea-Paper.pdf)\n\n## Origin\n\nThe roots of both MCMC and variational methods lie in statistical physics.[11](https://www.cs.princeton.edu/courses/archive/fall11/cos597C/reading/WainwrightJordan2008.pdf) Neal and Hinton's 1998 treatment of the EM algorithm made important connections between variational bounds and EM.[12](https://doi.org/10.1007/978-94-011-5014-9_12) Michael Jordan and colleagues' 1999 tutorial in *Machine Learning* established variational methods for inference and learning in Bayesian networks and Markov random fields.[13](https://doi.org/10.1023/a:1007665907178) Hagai Attias presented Variational Bayes at NIPS 1999 (proceedings published in 2000) as a practical framework for graphical models, with an algorithm that generalizes EM and has guaranteed convergence.[14](https://proceedings.neurips.cc/paper/1999/file/74563ba21a90da13dacf2a73e3ddefa7-Paper.pdf) Wainwright and Jordan's 2008 monograph in *Foundations and Trends in Machine Learning* unified the field through exponential-family representations and conjugate duality.[15](https://doi.org/10.1561/2200000001) Matt Hoffman and colleagues introduced stochastic variational inference in 2012.[16](https://doi.org/10.48550/arxiv.1206.7051)\n\n## Variants\n\n**Collapsed VI** integrates out variables to speed inference, as in latent-space variational Bayes[17](https://doi.org/10.1109/tpami.2008.157) and fast VI in the conjugate exponential family.[18](https://doi.org/10.48550/arxiv.1206.5162) **Black-box VI** estimates ELBO gradients by sampling when derivatives are unavailable,[19](https://doi.org/10.48550/arxiv.1401.0118) and Wingate and Weber's 2013 automated VI brought stochastic-gradient VI to probabilistic programming.[20](https://doi.org/10.48550/arxiv.1301.1299) **Normalizing flows** enrich a simple \\( q \\) by invertible transforms, first as post-hoc enrichments of diagonal Gaussians[21](https://doi.org/10.48550/arxiv.1505.05770) and later as architectures that scale in latent dimensionality, most notably the Inverse Autoregressive Flow.[22](https://proceedings.neurips.cc/paper_files/paper/2025/file/7ed14a8ca0da9df00d8d0e15617d8e2a-Paper-Conference.pdf) Annealed Flow Transport Monte Carlo combines flows with annealing for sampling.[23](https://doi.org/10.48550/arxiv.2102.07501) [Diffusion](https://www.edgepedia.org/diffusion) implicit variational inference integrates diffusion models with implicit VI.[24](https://www.aimsciences.org/article/doi/10.3934/mfc.2026024) Ξ-VI extends mean-field VI via entropic regularization solved with a multi-marginal Sinkhorn algorithm, with posterior consistency and a [Bernstein–von Mises theorem](https://www.edgepedia.org/bernstein-von-mises-theorem) proved.[25](http://jmlr.org/papers/volume27/24-1057/24-1057.pdf) A 2024 large-scale evaluation benchmarks flow-based VI, stochastic normalizing flows, Annealed Flow Transport, and CRAFT against diffusion-based samplers.[26](https://arxiv.org/abs/2406.07423) A 2025 result shows every full-rank variational implicit posterior transformation can be represented exactly as a forward autoregressive flow augmented with a translation term using the model's prior functions.[22](https://proceedings.neurips.cc/paper_files/paper/2025/file/7ed14a8ca0da9df00d8d0e15617d8e2a-Paper-Conference.pdf)\n\n## Applications\n\n**Topic models** were an application of SVI, which analyzed 300K Nature, 1.8M New York Times, and 3.8M [Wikipedia](https://www.edgepedia.org/wikipedia) articles, sizes batch variational inference could not handle.[5](https://www.cs.columbia.edu/~blei/papers/HoffmanBleiWangPaisley2013.pdf) [Stochastic](https://www.edgepedia.org/stochastic) collapsed variational inference for LDA converged to better held-out likelihood than SVI on the New York Times and Wikipedia datasets.[27](https://jfoulds.informationsystems.umbc.edu/Foulds_SCVB0.pdf) For LDA, a VB coordinate update costs \\( O(m \\cdot k) \\) versus \\( O(m^{2} \\cdot k) \\) for exact collapsed VB, and the bound-tightness gap decreases as \\( O(k^{-1}) + p \\log m / m \\).[28](https://papers.nips.cc/paper_files/paper/2008/file/a49e9411d64ff53eccfdd09ad10a15b3-Paper.pdf) In **generative modeling**, fitting a neural-network recognition model (probabilistic encoder) to the intractable posterior yields the variational auto-encoder.[29](https://papers.baulab.info/papers/Kingma-2013.pdf) ADVI is deployed in the Stan probabilistic programming system and was applied to a dataset with millions of observations.[9](https://papers.nips.cc/paper_files/paper/2015/file/352fe25daf686bdb4edca223c921acea-Paper.pdf) • [10](https://doi.org/10.48550/arxiv.1603.00788)\n\n## Limitations and alternatives\n\nVI generally underestimates posterior variance as a consequence of its objective function, and its relative accuracy versus MCMC remains unknown.[1](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773) The mechanism is the reverse KL: minimizing \\( D_{KL}(q \\| p) \\) yields distributions that avoid regions where \\( p \\) is small, so for multimodal posteriors \\( q \\) tends to find a single mode, whereas the mass-covering \\( D_{KL}(p \\| q) \\) averages across modes; a factorized approximation to a correlated Gaussian is too compact.[30](https://www.cs.cmu.edu/~rsalakhu/10707/Lectures/Lecture_Variational_Inference_2019.pdf) Mean-field Gaussian guides therefore give over-confident credible intervals, an axis-aligned ellipsoid correctly centered but systematically too narrow.[7](https://drawinginferences.com/chapters/vi.html) VB approximates the joint, so individual \\( q_{i}(x_{i}) \\) components can be poor, even remotely unlike, the true marginals, making it harder to debug than algorithms with locally interpretable node states.[4](https://robots.ox.ac.uk/~sjrob/Pubs/vbTutorialFinal.pdf) The ELBO is non-convex, and CAVI only guarantees a local optimum sensitive to initialization.[1](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773)\n\nCompared with the **Laplace approximation**, which is purely local, limited to the Gaussian family, inapplicable to discrete variables, and requires costly Hessian inversion in high dimensions, KL minimization typically approximates the posterior covariance more accurately.[6](https://arxiv.org/pdf/1711.05597v3.pdf) In theory, Gaussian VI's mean error scales as \\( (d/(\\beta n))^{3} \\) versus \\( d/(\\beta n) \\) for Laplace, two orders of magnitude smaller.[31](https://par.nsf.gov/servlets/purl/10633863) [Expectation propagation](https://www.edgepedia.org/expectation-propagation) achieves the same mean and covariance accuracy as Gaussian VI.[31](https://par.nsf.gov/servlets/purl/10633863) Because the reverse KL is mode-seeking, the ELBO is not sensitive to mode collapse; the evidence upper bound (EUBO), tied to the forward KL, is well suited to quantify it.[26](https://arxiv.org/abs/2406.07423)\n\n## References\n\n1. [Variational Inference: A Review for Statisticians (Blei, Kucukelbir, McAuliffe), JASA Vol 112, No 518](https://www.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773)\n2. [α-Variational Inference with Statistical Guarantees (Wang & Blei)](https://ar5iv.labs.arxiv.org/html/1710.03266)\n3. [Deep Learning, Chapter 19: Inference as Optimization (Goodfellow, Bengio, Courville)](https://www.deeplearningbook.org/contents/inference.html)\n4. [A tutorial on variational Bayesian inference (Fox & Roberts, Artificial Intelligence Review, 2011)](https://robots.ox.ac.uk/~sjrob/Pubs/vbTutorialFinal.pdf)\n5. [Stochastic Variational Inference (Hoffman, Blei, Wang, Paisley, 2013, JMLR)](https://www.cs.columbia.edu/~blei/papers/HoffmanBleiWangPaisley2013.pdf)\n6. [Advances in Variational Inference (Zhang, Bütepage, Kjellström, Mandt), IEEE TPAMI](https://arxiv.org/pdf/1711.05597v3.pdf)\n7. [Variational inference – Drawing Inferences (Pyro/SVI chapter)](https://drawinginferences.com/chapters/vi.html)\n8. [A practical tutorial on Variational Bayes (Tran et al.)](https://ar5iv.labs.arxiv.org/html/2103.01327)\n9. [Automatic Variational Inference in Stan (Kucukelbir, Tran, Ranganath, Gelman, Blei, NIPS 2015)](https://papers.nips.cc/paper_files/paper/2015/file/352fe25daf686bdb4edca223c921acea-Paper.pdf)\n10. [Kucukelbir, Alp and colleagues (2016). Automatic Differentiation Variational Inference. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1603.00788)\n11. [Graphical models, exponential families, and variational inference (Wainwright & Jordan, Foundations and Trends in ML, 2008)](https://www.cs.princeton.edu/courses/archive/fall11/cos597C/reading/WainwrightJordan2008.pdf)\n12. [Radford M. Neal, Geoffrey E. Hinton (1998). A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants. .](https://doi.org/10.1007/978-94-011-5014-9_12)\n13. [Michael I. Jordan and colleagues (1999). An Introduction to Variational Methods for Graphical Models. Machine Learning.](https://doi.org/10.1023/a:1007665907178)\n14. [A Variational Bayesian Framework for Graphical Models (H. Attias, NIPS 1999)](https://proceedings.neurips.cc/paper/1999/file/74563ba21a90da13dacf2a73e3ddefa7-Paper.pdf)\n15. [Martin J. Wainwright, Michael I. Jordan (2008). Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning.](https://doi.org/10.1561/2200000001)\n16. [Hoffman, Matt and colleagues (2012). Stochastic Variational Inference. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1206.7051)\n17. [Jaemo Sung, Z. Ghahramani, Sung-Yang Bang (2008). Latent-Space Variational Bayes. IEEE Transactions on Pattern Analysis and Machine Intelligence.](https://doi.org/10.1109/tpami.2008.157)\n18. [Hensman, James, Rattray, Magnus, Lawrence, Neil D. (2012). Fast Variational Inference in the Conjugate Exponential Family. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1206.5162)\n19. [Ranganath, Rajesh, Gerrish, Sean, Blei, David M. (2013). Black Box Variational Inference. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1401.0118)\n20. [Wingate, David, Weber, Theophane (2013). Automated Variational Inference in Probabilistic Programming. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1301.1299)\n21. [Rezende, Danilo Jimenez, Mohamed, Shakir (2015). Variational Inference with Normalizing Flows. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1505.05770)\n22. [Model-Informed Flows for Bayesian Inference (NeurIPS 2025)](https://proceedings.neurips.cc/paper_files/paper/2025/file/7ed14a8ca0da9df00d8d0e15617d8e2a-Paper-Conference.pdf)\n23. [Arbel, Michael, Matthews, Alexander G. D. G., Doucet, Arnaud (2021). Annealed Flow Transport Monte Carlo. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.2102.07501)\n24. [Diffusion implicit variational inference for complex posterior modeling (DIVI)](https://www.aimsciences.org/article/doi/10.3934/mfc.2026024)\n25. [Extending Mean-Field Variational Inference via Entropic Regularization: Theory and Computation (Ξ-VI), JMLR](http://jmlr.org/papers/volume27/24-1057/24-1057.pdf)\n26. [Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling (2024)](https://arxiv.org/abs/2406.07423)\n27. [Stochastic Collapsed Variational Bayesian Inference for LDA (SCVB0)](https://jfoulds.informationsystems.umbc.edu/Foulds_SCVB0.pdf)\n28. [Relative Performance Guarantees for Approximate Inference in Latent Dirichlet Allocation (Teh, Kurihara, Welling, NeurIPS 2008)](https://papers.nips.cc/paper_files/paper/2008/file/a49e9411d64ff53eccfdd09ad10a15b3-Paper.pdf)\n29. [Auto-Encoding Variational Bayes (Kingma, Welling, 2013, ICLR 2014)](https://papers.baulab.info/papers/Kingma-2013.pdf)\n30. [CMU 10-707 lecture notes on Variational Inference](https://www.cs.cmu.edu/~rsalakhu/10707/Lectures/Lecture_Variational_Inference_2019.pdf)\n31. [Approximation accuracy of Gaussian variational inference (error bounds vs Laplace)](https://par.nsf.gov/servlets/purl/10633863)\n\n---\n*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Bayesian statistics › Bayesian computation and software › Variational and approximate Bayesian methods*\n\n*Initially written Sep 29, 2026 · Reviewed: Sep 30, 2026 · Edited: Oct 15, 2026 · Last review: Sep 30, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [ "https://www.cs.columbia.edu/~blei/papers/HoffmanBleiWangPaisley2013.pdf", "https://www.cs.cmu.edu/~rsalakhu/10707/Lectures/Lecture_Variational_Inference_2019.pdf" ], "url": "https://www.edgepedia.org/variational-bayesian-inference", "markdown_url": "https://www.edgepedia.org/variational-bayesian-inference.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Variational Bayesian inference\", Edgepedia (EdgeChat), https://www.edgepedia.org/variational-bayesian-inference. Edgepedia Community License 1.0.", "credit_md": "\"[Variational Bayesian inference](https://www.edgepedia.org/variational-bayesian-inference)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/variational-bayesian-inference](https://www.edgepedia.org/variational-bayesian-inference). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Variational Bayesian inference\", Edgepedia (EdgeChat), https://www.edgepedia.org/variational-bayesian-inference. Edgepedia Community License 1.0.", "speakable": "Variational Bayesian inference approximates an intractable posterior distribution by optimization, maximizing the ELBO to find the closest density in KL divergence; it is faster than MCMC but generally underestimates posterior variance." }