# Parameterization and prior choice in MCMC

Parameterization and prior choice in MCMC ([Markov chain Monte Carlo](https://www.edgepedia.org/markov-chain-monte-carlo), a random-sampling method for Bayesian computation) are modeling decisions that, under a consistent reparameterization with the prior transformed accordingly, can leave a model's implied posterior distribution unchanged but reshape the geometry a sampler must traverse, often determining whether the sampler is efficient, biased, or fails to converge at all<sup>[10](https://mc-stan.org/docs/2_35/stan-users-guide/reparameterization.html)</sup>. In hierarchical models in particular, a poor choice of parameterization produces the funnel geometry that defeats fixed-step-size samplers, while a good one can speed sampling by orders of magnitude.

| Fact | Value |
|---|---|
| Reparameterizations change the likelihood | Not at all; they are deterministic transformations of the same model <sup>[1](https://stat.columbia.edu/~gelman/research/published/parameterization.pdf)</sup> |
| Non-centered vs centered, EHMC time per effective sample | 2.94×10⁻² s vs 16.2 s (≈550×) in a 10-dimensional one-way normal benchmark <sup>[2](https://ar5iv.labs.arxiv.org/html/1312.0906)</sup> |
| Rule of thumb | Non-centered when data are sparse per group; centered when data are highly informative <sup>[2](https://ar5iv.labs.arxiv.org/html/1312.0906)</sup><sup> • </sup><sup>[3](https://mc-stan.org/docs/2_31/stan-users-guide/reparameterization.html)</sup> |
| Gibbs convergence relation | Centered and non-centered rates are exactly complementary under flat priors: \(\rho_{\mathrm{nc}} = 1 - \rho_{\mathrm{c}}\) <sup>[4](https://arxiv.org/pdf/0708.3797)</sup> |
| Interleaved HMC robustness | ESS per 1000 gradient evaluations within a factor of two of the best fixed parameterization <sup>[5](https://ar5iv.labs.arxiv.org/html/1906.03028)</sup> |
| π_j-adaptive rule (2026 preprint) | Min-ESS 2381 vs 187 centered and 3327 full non-centered in the σ=1 sparse setting <sup>[6](https://arxiv.org/html/2606.19655)</sup> |
| Non-centering's geometric optimality | Holds exactly only for a log-Gaussian prior on log τ <sup>[7](https://ar5iv.labs.arxiv.org/html/1910.09407)</sup> |

## Why parameterization matters for sampling

A reparameterization is a change of variables applied to a model's parameters. It does not change the likelihood <sup>[1](https://stat.columbia.edu/~gelman/research/published/parameterization.pdf)</sup>. What changes is the shape of the density the sampler works with: its scales along each axis, its correlations, and its curvature.

This matters because practical samplers make geometric assumptions. When the posterior has regions with wildly different scales or curvature, a single global setting cannot serve all of them <sup>[3](https://mc-stan.org/docs/2_31/stan-users-guide/reparameterization.html)</sup>. Parameterization and prior choice are the modeler's main levers for removing those scale disparities before any sampler runs.

## The funnel pathology in hierarchical models

Hierarchical models pool information across groups: inferring the hyperparameters (μ, σ) together with the group-level parameters allows the model to pool data across the groups and reduce their posterior variance <sup>[8](https://mc-stan.org/learn-stan/case-studies/divergences_and_bias.html)</sup>. When data are sparse, the resulting joint density takes the shape of a funnel: a wide mouth where the hierarchical scale is large and the group parameters are weakly constrained, narrowing to a neck of very low scale where shrinkage is strong. Both regions carry comparable probability mass, one through high density with low volume, the other through low density with high volume <sup>[2](https://ar5iv.labs.arxiv.org/html/1312.0906)</sup>.

The mechanical problem is that the density's scale changes with the hierarchical scale parameter. A step size that works well in the wide body of the funnel is too large for the narrow neck, and a step size suited to the neck is inefficient in the body <sup>[3](https://mc-stan.org/docs/2_31/stan-users-guide/reparameterization.html)</sup>. Constant-metric HMC cannot adapt to this rapidly varying curvature, whereas Riemannian-manifold methods are specifically designed to use position-dependent geometry, so constant-metric HMC trajectories become restricted to narrow neighborhoods of the scale parameter and numerical integration becomes unstable deep in the funnel <sup>[7](https://ar5iv.labs.arxiv.org/html/1910.09407)</sup>.

In practice this shows up as divergences and biased inference. In the eight-schools-style model, inferring the hyperparameters μ and σ together with the group-level parameters θ₁,…,θ₈ allows pooling that reduces posterior variance, but the same pooling squeezes the posterior into the funnel geometry; the fix is reimplementing the model with a non-centered parameterization <sup>[8](https://mc-stan.org/learn-stan/case-studies/divergences_and_bias.html)</sup>.

## Centered vs non-centered parameterization

The **centered parameterization (CP)** declares group-level parameters θ directly, with a prior θ ~ Normal(μ, σ). The **non-centered parameterization (NCP)** writes θ = μ + σ·η with η ~ Normal(0, 1), so the hierarchical dependence is a deterministic transformation rather than a correlated prior. The transform follows Papaspiliopoulos, Roberts, and Sköld (2007) <sup>[3](https://mc-stan.org/docs/2_31/stan-users-guide/reparameterization.html)</sup>.

The choice is governed by how strongly the data inform each group. With little data, the CP's correlation among θ, μ, and σ is extreme (Neal's funnel being the extreme case with no data), and the NCP is preferable; with a lot of data, the CP is more efficient <sup>[3](https://mc-stan.org/docs/2_31/stan-users-guide/reparameterization.html)</sup>. Betancourt and Girolami put it sharply: when the centered parameterization is best, there is often enough data that the partial pooling of hierarchical models is not needed in the first place <sup>[2](https://ar5iv.labs.arxiv.org/html/1312.0906)</sup>. In condition-number terms, the NCP suits weak evidence and the CP suits strong evidence <sup>[5](https://ar5iv.labs.arxiv.org/html/1906.03028)</sup>.

Two refinements matter. First, the CP can have a computational edge for Gibbs samplers: conditional independence of the data from the hyperparameters given θ can make [Gibbs sampling](https://www.edgepedia.org/gibbs-sampling) implementable for the CP but not the NCP, and partially non-centered parameterizations (Meng and van Dyk) occupy continua between the two <sup>[4](https://arxiv.org/pdf/0708.3797)</sup><sup> • </sup><sup>[11](https://mc-stan.org/docs/2_28/stan-users-guide/reparameterization.html)</sup>. Second, partial or incomplete reparameterizations can induce funnel geometry of their own, so half-measures need care <sup>[7](https://ar5iv.labs.arxiv.org/html/1910.09407)</sup>.

## By the numbers

The quantified gains are large. In a 10-dimensional one-way normal benchmark, Euclidean HMC took 2.94×10⁻² seconds per effective sample under the NCP versus 16.2 s under the CP, a gap of roughly 550×. For comparison, random-walk [Metropolis](https://www.edgepedia.org/metropolis) under the CP took 1220 s per effective sample and Gibbs 297 s; both improved dramatically under the NCP, to 1.95 s and 1.44 s respectively <sup>[2](https://ar5iv.labs.arxiv.org/html/1312.0906)</sup>. Even under centered pathologies, Euclidean HMC is at least an order of magnitude more efficient than random-walk Metropolis and Gibbs <sup>[2](https://ar5iv.labs.arxiv.org/html/1312.0906)</sup>.

A 2026 preprint studying the pooling factor reports that non-centering won minimum ESS in every simulated cell (σ ∈ {1, 2}, n_j ∈ {3, 50}), and that the centered σ geometry retains a funnel that non-centering removes regardless of group size. Its π_j-adaptive rule, which sets per-group centering weights from the prior fraction π_j, reached min-ESS 2381 at σ=1 with sparse data (π_j ≈ 0.8), far above 187 for the CP and 1071 for half non-centering, though below 3327 for full non-centering <sup>[6](https://arxiv.org/html/2606.19655)</sup>.

## Priors as geometry, not just regularization

Priors are usually discussed as regularizers, but they also determine sampling geometry. The standard non-centering trick, θ = μ + σ·η with η ~ Normal(0,1), is geometrically optimal only for a log-Gaussian prior density on λ = log τ; for other scale priors, such as a half-Gaussian, non-centering may be suboptimal <sup>[7](https://ar5iv.labs.arxiv.org/html/1910.09407)</sup>. In other words, the reparameterization and the prior are a matched pair: change the prior on the hierarchical scale and the geometric argument for non-centering has to be re-derived, not assumed.

Prior informativeness also interacts with the CP/NCP choice directly. The NCP yields better geometry for weakly informative data and worse geometry for strongly informative data, the inverse of the CP <sup>[7](https://ar5iv.labs.arxiv.org/html/1910.09407)</sup>. Recent preconditioning work restates the same dependence: hierarchical varying-effects models might require centered or non-centered parameterizations depending on how informative the prior and data are <sup>[9](https://arxiv.org/html/2603.18845v1)</sup>.

## How it compares with sampler choice

Reparameterization and sampler choice are complementary levers, and their effects are not interchangeable. On the sampler side, EHMC beats random-walk Metropolis and Gibbs under both parameterizations, by at least an order of magnitude even in the unfavorable centered case <sup>[2](https://ar5iv.labs.arxiv.org/html/1312.0906)</sup>. On the parameterization side, the biggest wins come from matching the parameterization to the data regime.

For Gibbs specifically, the two parameterizations are exact complements under flat priors in a Gaussian hierarchical model: the convergence rates satisfy \(\rho_{\mathrm{nc}} = 1 - \rho_{\mathrm{c}}\), so when one mixes very slowly the other performs extremely well. Under proper priors both converge faster and the exact complementarity no longer holds <sup>[4](https://arxiv.org/pdf/0708.3797)</sup>. This complementarity, plus the CP's conditional-independence advantage for Gibbs implementation <sup>[4](https://arxiv.org/pdf/0708.3797)</sup>, means non-centering does not transfer its HMC advantage unchanged to Gibbs.

A caution cuts the other way: bias from funnel geometry can be hard to observe when the [Markov chain](https://www.edgepedia.org/markov-chain) is slow and MCMC estimators are noisy, as is common with random-walk Metropolis and Gibbs samplers <sup>[8](https://mc-stan.org/learn-stan/case-studies/divergences_and_bias.html)</sup>. A sampler that mixes slowly and noisily can hide the very pathology that a geometry-sensitive sampler would expose as divergences.

## References

1. [Parameterization and Bayesian Modeling (Gelman et al., JASA 2004)](https://stat.columbia.edu/~gelman/research/published/parameterization.pdf)
2. [Hamiltonian Monte Carlo for Hierarchical Models (Betancourt & Girolami)](https://ar5iv.labs.arxiv.org/html/1312.0906)
3. [Reparameterization | Stan User's Guide](https://mc-stan.org/docs/2_31/stan-users-guide/reparameterization.html)
4. [Papaspiliopoulos, Roberts & Sköld: Centering and Noncentering Parameterizations and Gibbs Convergence](https://arxiv.org/pdf/0708.3797)
5. [Automatic Reparameterisation of Probabilistic Programs (Gorinova, Moore, Sarkar)](https://ar5iv.labs.arxiv.org/html/1906.03028)
6. [A Flat Connection: The Pooling Factor and the Geometry of Centring in Hierarchical MCMC (Bindoff, 2026 preprint)](https://arxiv.org/html/2606.19655)
7. [Incomplete Reparameterizations and Equivalent Metrics (Betancourt 2019)](https://ar5iv.labs.arxiv.org/html/1910.09407)
8. [Diagnosing Biased Inference with Divergences (Stan case study)](https://mc-stan.org/learn-stan/case-studies/divergences_and_bias.html)
9. [Preconditioning Hamiltonian Monte Carlo by Minimizing Fisher Divergence (2026 preprint)](https://arxiv.org/html/2603.18845v1)
10. [Reparameterization and Change of Variables](https://mc-stan.org/docs/2_35/stan-users-guide/reparameterization.html)
11. [24.7 Reparameterization, Stan User’s Guide](https://mc-stan.org/docs/2_28/stan-users-guide/reparameterization.html)

---
*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Bayesian statistics › Bayesian computation and software › Markov chain Monte Carlo for Bayes › MCMC practice: model parameterization and priors for sampling*

*Initially written Sep 17, 2026 · Reviewed: Sep 30, 2026 · Edited: Sep 30, 2026; Oct 11, 2026 · Last review: Sep 30, 2026*

*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*

License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license