{ "id": "ep6jj33v5q", "slug": "stein-variational-gradient-descent", "title": "Stein variational gradient descent", "updated": "2026-09-30", "topic_path": [ { "id": "technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/topics/technology" }, { "id": "technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing" }, { "id": "technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai" }, { "id": "technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_computation" }, { "id": "technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_methods" } ], "geo": [ { "id": "geo.nongeo.t2001.technology.computing.ai.ml_methods", "label": "Non-geographic · 2001 to 2020: Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_methods", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t2001", "label": "Non-geographic · 2001 to 2020", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001" }, { "id": "geo.nongeo.t2001.technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology" }, { "id": "geo.nongeo.t2001.technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing" }, { "id": "geo.nongeo.t2001.technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_computation" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_methods" } ] } ], "excerpt": "Stein variational gradient descent (SVGD) is a particle-based variational inference algorithm introduced by Qiang Liu and Dilin Wang in 2016, which moves particles to approximate a target distribution.", "snippet": "Stein variational gradient descent (SVGD) is a particle-based variational inference algorithm introduced by Qiang Liu and Dilin Wang in 2016, which moves particles to approximate a target distribution.", "node": "technology.computing.ai.ml_methods", "markdown": "# Stein variational gradient descent\n\nStein variational gradient descent (SVGD) is a particle-based variational inference algorithm that moves a set of particles to approximate a target probability distribution, using Stein identities and kernelized score functions (gradient of the log probability density) to combine gradient ascent with a repulsive force between particles. It was introduced as a general purpose [Bayesian inference](https://www.edgepedia.org/bayesian-inference) algorithm that iteratively transports particles to match the target by functional gradient descent minimizing the KL divergence.[1](https://doi.org/10.48550/arxiv.1608.04471) With a single particle it reduces to gradient ascent for the maximum a posteriori (MAP) estimate; with more particles it becomes a full [Bayesian sampling](https://www.edgepedia.org/bayesian-sampling) method, which makes it more particle-efficient than typical [Monte Carlo](https://www.edgepedia.org/monte-carlo) methods.[1](https://doi.org/10.48550/arxiv.1608.04471) • [2](https://papers.neurips.cc/paper_files/paper/2017/file/17ed8abedc255908be746d245e50263a-Paper.pdf) The update is deterministic, requires only the score function \\( \\nabla_{x}\\log p(x) \\) (no normalizing constant), and in practice a relatively small number of particles, for example several hundreds, suffices.[1](https://doi.org/10.48550/arxiv.1608.04471)\n\n| Key fact | Detail |\n|---|---|\n| What it produces | A set of \\( n \\) particles approximating the target distribution \\( p \\); MAP with \\( n=1 \\), full sampling with more[1](https://doi.org/10.48550/arxiv.1608.04471) |\n| Core update | \\( x_{i}^{l+1} = x_{i}^{l} + \\epsilon\\,\\hat{\\phi}^{*}(x_{i}^{l}) \\), a smoothed gradient term plus a kernel repulsion term[1](https://doi.org/10.48550/arxiv.1608.04471) |\n| Standard kernel | RBF kernel \\( k(x,x') = \\exp(-\\|x-x'\\|^{2}/h) \\) with median-trick bandwidth \\( h = \\mathrm{med}^{2}/\\log n \\)[1](https://doi.org/10.48550/arxiv.1608.04471) |\n| Cost per iteration | \\( O(n^{2}) \\) kernel matrix computation, small next to gradient evaluation for typical \\( n \\)[1](https://doi.org/10.48550/arxiv.1608.04471) |\n| Introduced by | Qiang Liu and Dilin Wang, 2016[1](https://doi.org/10.48550/arxiv.1608.04471) |\n| Best finite-particle rate | Expected kernel Stein discrepancy \\( O(d/\\sqrt{N}) \\) (ICLR 2025)[3](https://proceedings.iclr.cc/paper_files/paper/2025/file/a9f61e75161ceb9350890677b3aa0f0c-Paper-Conference.pdf) |\n| Main failure modes | Mode collapse, variance under-estimation in high dimensions, bandwidth pathologies[1](https://doi.org/10.48550/arxiv.1608.04471) • [4](https://www.cs.toronto.edu/~erdogdu/papers/kernel-bias.pdf) |\n\n## How it works\n\nThe method rests on Stein's identity: for a differentiable test function \\( \\phi \\), \\( \\mathbb{E}_{x\\sim p}[\\mathcal{A}_{p}\\phi(x)] = 0 \\), where the Stein operator is \\( \\mathcal{A}_{p}\\phi(x) = \\nabla_{x}\\log p(x)\\,\\phi(x)^{\\top} + \\nabla_{x}\\phi(x) \\). The identity holds by integration by parts under mild boundary conditions, and the operator depends on \\( p \\) only through the score function, so no normalizing constant is needed.[1](https://doi.org/10.48550/arxiv.1608.04471)\n\nMaximizing the expected Stein operator violation over the unit ball of a reproducing kernel [Hilbert space](https://www.edgepedia.org/hilbert-space) (RKHS) defines the kernelized Stein discrepancy (KSD), \\( D(q,p) = \\|\\phi^{*}_{q,p}\\|_{\\mathcal{H}} \\), with closed-form maximizer \\( \\phi^{*}_{q,p}(\\cdot) = \\mathbb{E}_{x\\sim q}[\\mathcal{A}_{p}k(x,\\cdot)] \\).[1](https://doi.org/10.48550/arxiv.1608.04471) Because the instantaneous decrease in KL divergence under a particle update is proportional to the squared KSD at the current particle measure, the KSD-maximizing direction is the steepest descent direction for KL.[5](https://arxiv.org/html/2510.02067) Evaluating the expectation over the empirical particle distribution gives the update rule below.\n\nThe algorithm can be read in two ways. It is an \"optimization-then-approximate\" procedure: first derive an infinite-dimensional gradient descent, a PDE that solves \\( \\min_{q} \\mathrm{KL}(q\\|p) \\) and converges to \\( p \\) as \\( t\\to\\infty \\), then approximate that gradient flow with particles.[6](https://www.cs.utexas.edu/~lqiang/PDF/svgd_aabi2016.pdf) The population dynamics is a gradient flow of the KL divergence under a metric on the space of distributions induced by the Stein operator, and it is an instance of the [Vlasov equation](https://www.edgepedia.org/vlasov-equation) known in physics.[2](https://papers.neurips.cc/paper_files/paper/2017/file/17ed8abedc255908be746d245e50263a-Paper.pdf) • [6](https://www.cs.utexas.edu/~lqiang/PDF/svgd_aabi2016.pdf)\n\n## How it is done\n\nOne iteration, for particles \\( x_{1}^{l},\\dots,x_{n}^{l} \\), computes\n\n\\[ x_{i}^{l+1} = x_{i}^{l} + \\epsilon\\,\\hat{\\phi}^{*}(x_{i}^{l}), \\qquad \\hat{\\phi}^{*}(x) = \\frac{1}{n}\\sum_{j=1}^{n}\\Big[k(x_{j}^{l},x)\\,\\nabla_{x_{j}^{l}}\\log p(x_{j}^{l}) + \\nabla_{x_{j}^{l}}k(x_{j}^{l},x)\\Big]. \\]\n\nThe first term drives particles toward high-probability areas of \\( p \\) along a smoothed gradient direction; the second acts as a repulsive force that prevents all points from collapsing together into local modes of \\( p \\).[1](https://doi.org/10.48550/arxiv.1608.04471)\n\nA practitioner chooses the target log-density (only the score is needed), a positive definite kernel, the particle count, and the step size. Standard practice uses the [RBF kernel](https://www.edgepedia.org/radial-basis-function-kernel) with bandwidth \\( h = \\mathrm{med}^{2}/\\log n \\), where \\( \\mathrm{med} \\) is the median pairwise distance between the current particles, so \\( h \\) changes adaptively across iterations; the introducing paper's experiments instead used \\( h = 0.002 \\times \\mathrm{med}^{2} \\), following the guidance of Dai and colleagues on particle mirror descent.[1](https://doi.org/10.48550/arxiv.1608.04471) • [7](https://doi.org/10.48550/arxiv.1506.03101) [AdaGrad](https://www.edgepedia.org/adagrad) is used for the step size, and particles are initialized from the prior distribution unless otherwise specified.[1](https://doi.org/10.48550/arxiv.1608.04471)\n\nThe update requires computing the kernel matrix, which costs \\( O(n^{2}) \\); in practice this is small relative to gradient evaluation because a relatively small \\( n \\), for example several hundreds, suffices. For very large \\( n \\), the particle sum can be approximated by subsampling particles or by a random feature expansion of the kernel.[1](https://doi.org/10.48550/arxiv.1608.04471) In the [Bayesian neural network](https://www.edgepedia.org/bayesian-neural-network) experiments of the introducing paper, 20 particles, AdaGrad with momentum, and mini-batches of 100 were used.[1](https://doi.org/10.48550/arxiv.1608.04471)\n\n## Origin\n\nSVGD was introduced by Qiang Liu and Dilin Wang in 2016, both then at the Department of Computer Science, Dartmouth College.[1](https://doi.org/10.48550/arxiv.1608.04471) • [8](https://dl.acm.org/doi/10.5555/3157096.3157362) Its derivation connects the derivative of the KL divergence under smooth transforms with Stein's identity and the kernelized Stein discrepancy, which the authors credit to recently proposed work.[1](https://doi.org/10.48550/arxiv.1608.04471) The KSD itself was proposed in concurrent 2016 works: Liu's overview credits Liu, Lee, and Jordan[9](https://doi.org/10.48550/arxiv.1602.03253) • [6](https://www.cs.utexas.edu/~lqiang/PDF/svgd_aabi2016.pdf), and independent credit is also given to Chwialkowski, Strathmann, and Gretton, with related work by Oates, Girolami, and Chopin on control functionals[10](https://doi.org/10.1111/rssb.12185) • [6](https://www.cs.utexas.edu/~lqiang/PDF/svgd_aabi2016.pdf); published accounts disagree on priority and the disagreement is unresolved. Gorham and Mackey's 2015 work on measuring sample quality with [Stein's method](https://www.edgepedia.org/steins-method) supplied a key precursor result: Theorem 8 of that work establishes when KSD convergence implies weak convergence, for distantly dissipative targets and a class of inverse multi-quadric kernels.[2](https://papers.neurips.cc/paper_files/paper/2017/file/17ed8abedc255908be746d245e50263a-Paper.pdf) Particle mirror descent by Dai, He, Dai, and Song (2015) preceded SVGD and guided its bandwidth choice.[7](https://doi.org/10.48550/arxiv.1506.03101) Liu (2017) gave the first theoretical analysis, establishing weak convergence of the empirical measures and the Vlasov-equation characterization.[2](https://papers.neurips.cc/paper_files/paper/2017/file/17ed8abedc255908be746d245e50263a-Paper.pdf)\n\n## Variants\n\n**Preconditioning.** Matrix-SVGD replaces the scalar kernel with matrix-valued kernels \\( K(x,x') = k(x,x') \\cdot I \\) in the vanilla case, using Hessian or [Fisher information](https://www.edgepedia.org/fisher-information) matrices as preconditioners, with Kronecker-factored (KFAC) approximation for large-scale Fisher matrices in neural networks. On [Bayesian logistic regression](https://www.edgepedia.org/bayesian-logistic-regression) it converged much faster than vanilla SVGD and preconditioned SGLD.[11](https://proceedings.neurips.cc/paper/2019/file/5dcd0ddd3d918c70d380d32bce4e733a-Paper.pdf)\n\n**Stochastic and scalable forms.** [Stochastic](https://www.edgepedia.org/stochastic) variants add noise to the dynamics to aid exploration and efficiency.[12](https://www.jmlr.org/papers/volume24/20-602/20-602.pdf) S-SVGD is a slice-based variant that reduces the per-iteration cost of scalable SVGD.[13](https://arxiv.org/pdf/2202.03297) A Grassmann variant projects the updates to counter high-dimensional mode collapse and variance under-estimation.[14](https://proceedings.mlr.press/v151/liu22a/liu22a.pdf)\n\n**Amortization and adaptive kernels.** Amortized SVGD addresses the inefficiency of repeatedly inferring many target distributions by training a [stochastic neural network](https://www.edgepedia.org/stochastic-neural-network) whose output mimics the SVGD dynamics.[6](https://www.cs.utexas.edu/~lqiang/PDF/svgd_aabi2016.pdf) Ad-SVGD (2025) alternates particle updates with gradient ascent on the KSD to tune kernel bandwidths, reusing the same score gradients so no additional log-density evaluations are needed; it outperformed the median heuristic with no significant runtime difference in experiments with 50 to 200 particles.[5](https://arxiv.org/html/2510.02067) Stein variational evolution strategies combine the SVGD update with gradient-free evolution strategies.[15](https://doi.org/10.48550/arxiv.2410.10390)\n\n## Applications\n\nThe introducing paper demonstrated SVGD on Bayesian neural network regression and classification, using 20 particles with stochastic gradients and mini-batches.[1](https://doi.org/10.48550/arxiv.1608.04471) A NeurIPS 2017 application applied SVGD to learning variational autoencoders without parametric assumptions on the encoder distribution, showing strong performance on unsupervised and semi-supervised problems, including semi-supervised analysis of ImageNet, demonstrating scalability to large datasets.[16](https://papers.nips.cc/paper/2017/file/443dec3062d0286986e21dc0631734c9-Paper.pdf) Citation records show continued derivative work, including a trust-region method for graphical Stein variational inference at UAI 2025 and meta-learning via PAC-Bayesian bounds at IJCAI 2024.[8](https://dl.acm.org/doi/10.5555/3157096.3157362)\n\n## Limitations and alternatives\n\n**Convergence guarantees.** Liu (2017) showed that the population dynamics decreases KL at a rate upper-bounded by the squared Stein discrepancy for sufficiently small step sizes.[2](https://papers.neurips.cc/paper_files/paper/2017/file/17ed8abedc255908be746d245e50263a-Paper.pdf) A 2020 analysis provided the first finite-time results: a descent lemma, rates for the averaged Stein Fisher divergence, and a propagation-of-chaos bound; if the target satisfies the Stein log-[Sobolev inequality](https://www.edgepedia.org/sobolev-inequality) with constant \\( \\lambda > 0 \\), then\n\n\\[ \\mathrm{KL}(\\mu_{t}\\,\\|\\,\\pi) \\le e^{-2\\lambda t}\\,\\mathrm{KL}(\\mu_{0}\\,\\|\\,\\pi), \\]\n\nso KL converges exponentially along the dynamics.[17](https://ar5iv.labs.arxiv.org/html/2006.09797) For finite particles, Shi and Mackey (2022) proved the first finite-particle rate: for sub-Gaussian targets with Lipschitz score, SVGD with \\( n \\) particles drives the kernel Stein discrepancy to zero at an order \\( 1/\\sqrt{\\log\\log n} \\) rate, and reaching KSD \\( \\le \\Delta \\) takes \\( t^{*} = O(1/\\Delta^{2}) \\) rounds, each costing \\( \\Theta(n^{2}) \\) kernel gradient evaluations.[18](https://doi.org/10.48550/arxiv.2211.09721) In 2023, the virtual-particle variants VP-SVGD and GB-SVGD achieved the first oracle complexity guarantees with polynomial dimension dependence, \\( O(d^{4}/\\epsilon^{12}) \\) and \\( O(d^{6}/\\epsilon^{18}) \\) respectively for sub-Gaussian targets[19](https://papers.nips.cc/paper_files/paper/2023/file/9bf1962c5b65a243ee243bb03ff2c506-Paper-Conference.pdf), and Gaussian-SVGD with a bilinear kernel yielded the first uniform-in-time finite-particle result, with linear mean-field convergence to the KL-closest Gaussian for strongly log-concave targets.[20](https://papers.neurips.cc/paper_files/paper/2023/file/c0ae487420ebc8d0ed7c541b4e3f09d4-Paper-Conference.pdf) An ICLR 2025 paper then proved finite-particle rates of \\( O(d/\\sqrt{N}) \\) in expected KSD, near the i.i.d. optimum, and the first Wasserstein-2 rates for non-Gaussian finite-particle SVGD, of the form \\( O(1/N^{\\alpha/d}) \\), though with a curse of dimensionality in the exponent.[3](https://proceedings.iclr.cc/paper_files/paper/2025/file/a9f61e75161ceb9350890677b3aa0f0c-Paper-Conference.pdf)\n\n**Mode collapse and local optima.** As the bandwidth \\( h \\to 0 \\), the repulsive term vanishes and the update reduces to independent gradient-ascent chains, so particles collapse into local modes of the target.[1](https://doi.org/10.48550/arxiv.1608.04471) If particles are initialized on one mode of two well-separated Gaussians, the optimization gets stuck at a local optimum that fails to capture the other mode.[21](https://random-walks.org/book/papers/svgd/svgd.html) Although SVGD can express richer approximate distributions than mean-field variational inference, the optimization is not guaranteed to find them.\n\n**High-dimensional variance bias.** SVGD is asymptotically exact in the infinite-particle, vanishing-step-size limit[13](https://arxiv.org/pdf/2202.03297), but in the finite-particle regime it systematically underestimates marginal variance: for unit Gaussian targets with an RBF kernel, when \\( d,n\\to\\infty \\) with \\( d/n \\to \\gamma > 1 \\), the equilibrium particle variance scales as \\( 1/\\gamma \\)[4](https://www.cs.toronto.edu/~erdogdu/papers/kernel-bias.pdf), whereas the closely related MMD-descent achieves variance approaching 1. The collapse is attributed to high variance from Stein's lemma combined with deterministic bias, the inability to resample particles; when \\( d > n \\), more particles are required as dimensionality increases.[4](https://www.cs.toronto.edu/~erdogdu/papers/kernel-bias.pdf)\n\n**Kernel choice.** The median heuristic \\( h = \\mathrm{med}^{p}/\\log(M-1) \\) lacks theoretical justification and is known to degrade as problem dimensionality increases.[5](https://arxiv.org/html/2510.02067) Geometry analysis yields principled guidelines: nondifferentiable kernels with adjusted tails gave significant performance gains in numerical experiments, though small tail parameters make the dynamics stiff, and for translation-invariant kernels whose \\( h(x) \\) and \\( h'(x) \\) vanish at \\( \\pm\\infty \\), exponential convergence near equilibrium does not hold.[12](https://www.jmlr.org/papers/volume24/20-602/20-602.pdf)\n\n**Comparisons.** With one particle and an RBF kernel, SVGD is exactly MAP gradient ascent; when the kernel is the neural tangent kernel, it corresponds precisely to black-box variational inference in the mean-field limit as \\( n\\to\\infty \\).[1](https://doi.org/10.48550/arxiv.1608.04471) • [22](https://export.arxiv.org/pdf/2004.01822) • [23](https://arxiv.org/abs/2004.01822) On the Covertype dataset (581,012 points, 54 features), stochastic SVGD generally outperformed parallel SGLD, particle mirror descent, and DSVI.[1](https://doi.org/10.48550/arxiv.1608.04471) Its interacting repulsion term may be advantageous over non-interacting Langevin samplers for multimodal potentials, though direct runtime comparisons are difficult because the Stein mean-field limit runs on a fast time scale.[12](https://www.jmlr.org/papers/volume24/20-602/20-602.pdf) A long-time propagation-of-chaos result shows that time-averaged particle marginals become asymptotically independent samples from the target, so SVGD can provide multiple approximately i.i.d. samples, unlike traditional MCMC.[3](https://proceedings.iclr.cc/paper_files/paper/2025/file/a9f61e75161ceb9350890677b3aa0f0c-Paper-Conference.pdf)\n\n## References\n\n1. [Liu, Qiang, Wang, Dilin (2016). Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1608.04471)\n2. [Stein Variational Gradient Descent as Gradient Flow (NIPS 2017)](https://papers.neurips.cc/paper_files/paper/2017/file/17ed8abedc255908be746d245e50263a-Paper.pdf)\n3. [Finite-particle convergence rates for SVGD (ICLR 2025)](https://proceedings.iclr.cc/paper_files/paper/2025/file/a9f61e75161ceb9350890677b3aa0f0c-Paper-Conference.pdf)\n4. [Towards Characterizing the High-dimensional Bias of Kernel-based Particle Inference Algorithms](https://www.cs.toronto.edu/~erdogdu/papers/kernel-bias.pdf)\n5. [Adaptive Kernel Selection for Stein Variational Gradient Descent (Ad-SVGD, 2025)](https://arxiv.org/html/2510.02067)\n6. [Stein Variational Gradient Descent: Theory and Applications (AABI 2016 overview by the author)](https://www.cs.utexas.edu/~lqiang/PDF/svgd_aabi2016.pdf)\n7. [Dai, Bo and colleagues (2015). Provable Bayesian Inference via Particle Mirror Descent. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1506.03101)\n8. [Stein variational Gradient descent | Proceedings of the 30th International Conference on Neural Information Processing Systems (ACM DL record)](https://dl.acm.org/doi/10.5555/3157096.3157362)\n9. [Liu, Qiang, Lee, Jason D., Jordan, Michael I. (2016). A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1602.03253)\n10. [Chris J. Oates, Mark Girolami, Nicolas Chopin (2016). Control Functionals for Monte Carlo Integration. Journal of the Royal Statistical Society Series B (Statistical Methodology).](https://doi.org/10.1111/rssb.12185)\n11. [Stein Variational Gradient Descent with Matrix-Valued Kernels (NeurIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/5dcd0ddd3d918c70d380d32bce4e733a-Paper.pdf)\n12. [On the geometry of Stein variational gradient descent (JMLR)](https://www.jmlr.org/papers/volume24/20-602/20-602.pdf)\n13. [Slice-based SVGD variant paper (S-SVGD, arXiv 2202.03297)](https://arxiv.org/pdf/2202.03297)\n14. [Grassmann Stein Variational Gradient Descent (AISTATS 2022)](https://proceedings.mlr.press/v151/liu22a/liu22a.pdf)\n15. [Braun, Cornelius V., Lange, Robert T., Toussaint, Marc (2024). Stein Variational Evolution Strategies. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.2410.10390)\n16. [VAE Learning via Stein Variational Gradient Descent (NeurIPS 2017)](https://papers.nips.cc/paper/2017/file/443dec3062d0286986e21dc0631734c9-Paper.pdf)\n17. [A Non-Asymptotic Analysis for Stein Variational Gradient Descent (NeurIPS 2020)](https://ar5iv.labs.arxiv.org/html/2006.09797)\n18. [Shi, Jiaxin, Mackey, Lester (2022). A Finite-Particle Convergence Rate for Stein Variational Gradient Descent. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.2211.09721)\n19. [Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation (NeurIPS 2023)](https://papers.nips.cc/paper_files/paper/2023/file/9bf1962c5b65a243ee243bb03ff2c506-Paper-Conference.pdf)\n20. [Towards Understanding the Dynamics of Gaussian–Stein Variational Gradient Descent (NeurIPS 2023)](https://papers.neurips.cc/paper_files/paper/2023/file/c0ae487420ebc8d0ed7c541b4e3f09d4-Paper-Conference.pdf)\n21. [Stein variational gradient descent, Random Walks](https://random-walks.org/book/papers/svgd/svgd.html)\n22. [The equivalence between Stein variational gradient descent and black-box variational inference](https://export.arxiv.org/pdf/2004.01822)\n23. [The equivalence between Stein variational gradient descent and black-box variational inference](https://arxiv.org/abs/2004.01822)\n\n---\n*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data › Machine learning and neural computation › Machine learning methods*\n\n*Initially written Sep 29, 2026 · Reviewed: Sep 30, 2026 · Edited: Sep 30, 2026 · Last review: Sep 30, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [ "https://www.cs.toronto.edu/~erdogdu/papers/kernel-bias.pdf", "https://www.cs.utexas.edu/~lqiang/PDF/svgd_aabi2016.pdf" ], "url": "https://www.edgepedia.org/stein-variational-gradient-descent", "markdown_url": "https://www.edgepedia.org/stein-variational-gradient-descent.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Stein variational gradient descent\", Edgepedia (EdgeChat), https://www.edgepedia.org/stein-variational-gradient-descent. Edgepedia Community License 1.0.", "credit_md": "\"[Stein variational gradient descent](https://www.edgepedia.org/stein-variational-gradient-descent)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/stein-variational-gradient-descent](https://www.edgepedia.org/stein-variational-gradient-descent). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Stein variational gradient descent\", Edgepedia (EdgeChat), https://www.edgepedia.org/stein-variational-gradient-descent. Edgepedia Community License 1.0.", "speakable": "Stein variational gradient descent is a particle-based variational inference algorithm introduced by Qiang Liu and Dilin Wang in 2016, which moves particles to approximate a target distribution." }