{
 "id": "eprkybxzw0",
 "slug": "spline-regression",
 "title": "Spline regression",
 "updated": "2026-09-30",
 "topic_path": [
  {
   "id": "physical",
   "label": "Physical world and mathematics",
   "api_url": "https://www.edgepedia.org/api/v1/topics/physical"
  },
  {
   "id": "physical.math",
   "label": "Mathematics and statistics",
   "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math"
  },
  {
   "id": "physical.math.statistics",
   "label": "Statistics and probability",
   "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics"
  },
  {
   "id": "physical.math.statistics.stats_statistical_inference",
   "label": "Statistical inference, estimation, sampling, and testing",
   "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference"
  },
  {
   "id": "physical.math.statistics.stats_statistical_inference.regression-analysis",
   "label": "Regression analysis",
   "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.regression-analysis"
  },
  {
   "id": "physical.math.statistics.stats_statistical_inference.regression-analysis.spline-and-basis-expansion-regression",
   "label": "Spline and basis-expansion regression",
   "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.regression-analysis.spline-and-basis-expansion-regression"
  }
 ],
 "geo": [
  {
   "id": "geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference.regression-analysis",
   "label": "Non-geographic · 1800 to 1945: Regression analysis",
   "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference.regression-analysis",
   "path": [
    {
     "id": "geo.nongeo",
     "label": "Non-geographic",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo"
    },
    {
     "id": "geo.nongeo.t1800",
     "label": "Non-geographic · 1800 to 1945",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800"
    },
    {
     "id": "geo.nongeo.t1800.physical",
     "label": "Physical world and mathematics",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical"
    },
    {
     "id": "geo.nongeo.t1800.physical.math",
     "label": "Mathematics and statistics",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math"
    },
    {
     "id": "geo.nongeo.t1800.physical.math.statistics",
     "label": "Statistics and probability",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math.statistics"
    },
    {
     "id": "geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference",
     "label": "Statistical inference, estimation, sampling, and testing",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference"
    },
    {
     "id": "geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference.regression-analysis",
     "label": "Regression analysis",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference.regression-analysis"
    }
   ]
  },
  {
   "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference.regression-analysis",
   "label": "Non-geographic · 1946 to 2000: Regression analysis",
   "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference.regression-analysis",
   "path": [
    {
     "id": "geo.nongeo",
     "label": "Non-geographic",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo"
    },
    {
     "id": "geo.nongeo.t1946",
     "label": "Non-geographic · 1946 to 2000",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946"
    },
    {
     "id": "geo.nongeo.t1946.physical",
     "label": "Physical world and mathematics",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical"
    },
    {
     "id": "geo.nongeo.t1946.physical.math",
     "label": "Mathematics and statistics",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math"
    },
    {
     "id": "geo.nongeo.t1946.physical.math.statistics",
     "label": "Statistics and probability",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics"
    },
    {
     "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference",
     "label": "Statistical inference, estimation, sampling, and testing",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference"
    },
    {
     "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference.regression-analysis",
     "label": "Regression analysis",
     "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference.regression-analysis"
    }
   ]
  }
 ],
 "excerpt": "Spline regression fits piecewise polynomials, typically cubic, constrained to join smoothly at knots to model nonlinear relationships; variants include smoothing splines, P-splines, natural splines, and Friedman's MARS.",
 "snippet": "Spline regression fits piecewise polynomials, typically cubic, constrained to join smoothly at knots to model nonlinear relationships; variants include smoothing splines, P-splines, natural splines, and Friedman's MARS.",
 "node": "physical.math.statistics.stats_statistical_inference.regression-analysis.spline-and-basis-expansion-regression",
 "markdown": "# Spline regression\n\nSpline regression fits a piecewise polynomial function to data, with the polynomial pieces constrained to join smoothly at fixed points called knots, in order to model nonlinear relationships between a predictor and a response without committing to a single global polynomial or parametric form. \n\n| Key fact | Detail |\n|---|---|\n| What a knot is | A predictor value where two polynomial pieces meet; smoothness constraints (continuity of the function and its derivatives) are imposed there.[1][3] |\n| Truncated power basis | A degree \\( p \\) spline is a global polynomial of degree \\( p \\) plus shifted hinge terms \\( (x - t_{i})_{+}^{p} \\) of degree \\( p \\) at each knot \\( t_{i} \\).[3] |\n| Smoothing spline | The minimizer of the penalized least-squares criterion is a natural cubic spline with knots at the unique data values; \\( \\lambda \\to \\infty \\) yields the least-squares line.[4] |\n| Knot counts | Rules of thumb: about \\( n^{1/5} - 1 \\) inner knots for regression splines; about \\( \\min(n/4, 35) \\) or 20–40 knots for penalized splines.[5] |\n| P-splines | A large, fixed set of B-spline knots plus a difference penalty on adjacent coefficients, introduced by Eilers and Marx in 1996, largely removes knot placement as a modeling decision.[6] |\n| Boundary behavior | Ordinary splines have high variance at the extremes of the predictor; natural splines, constrained to be linear beyond the extreme knots, give more stable estimates and narrower confidence intervals there.[4] |\n| GAMs | Generalized additive models use spline terms within a penalized likelihood framework, fit by backfitting or related algorithms, and are widely used in medical and epidemiological research.[7][8] |\n\n## How it works\n\nA spline of degree \\( p \\) divides the predictor range at knots into intervals, fitting a separate polynomial of degree p on each, and imposes continuity conditions at every knot so the joined curve looks smooth rather than kinked. In the truncated power basis the fit is written as a global polynomial plus one hinge function per knot:[3]\n\n Each hinge term contributes nothing on one side of its knot and behaves like a degree-\\( p \\) polynomial on the other, so adding terms changes the curve only from that knot onward. The truncated power basis and the B-spline basis span the same spline space with the same knots and give the same fit, apart from computational accuracy; B-splines are preferred numerically because each basis function has minimal support, meaning it is nonzero over only a few adjacent knot intervals.[5][9] B-splines are defined through recurrence relations, which avoids the divided differences of the traditional truncated-power definition.[10]\n\nRegression splines and smoothing splines differ in where the knots sit and how wiggliness is controlled. A regression spline uses a modest number of chosen knots and unpenalized least squares. A smoothing spline places a knot at every distinct data point and controls wiggliness by penalizing the integrated squared second derivative, \\( \\int \\{ f^{\\prime\\prime}(t) \\}^{2} \\, dt \\); the minimizer is a natural cubic spline shrunk by the tuning parameter \\( \\lambda \\).[4][7] As \\( \\lambda \\to \\infty \\) the solution tends to the least-squares line, and as \\( \\lambda \\to 0 \\) it tends to the interpolating natural spline, so \\( \\lambda \\) traces the bias–variance tradeoff continuously.[4]\n\n## How it is done\n\nThe practitioner chooses the degree, the number and placement of knots (or a penalty), and the smoothing parameter. Cubic splines are the common default: Ruppert found the cubic order sufficient, Dierckx recommended \\( k = 4 \\) in the order notation for computational efficiency and constraint implementation, and Wegman and Wright noted \\( k = 4 \\) is the smallest order giving visual smoothness.[5] A cubic spline with \\( K \\) knots uses \\( K + 4 \\) degrees of freedom, an intercept plus \\( 3 + K \\) predictors.[4]\n\nFor knot counts, asymptotic results for regression splines suggest a number of inner knots growing slowly with sample size, with information criteria or cross-validation as alternatives; for penalized splines, published rules of thumb are about \\( \\min(n/4, 35) \\) knots or 20–40 knots.[5]\n\nThe smoothing parameter is chosen by a criterion. [Generalized cross-validation](https://www.edgepedia.org/generalized-cross-validation) (GCV) improves on ordinary cross-validation both on asymptotic grounds and in computational ease; Eilers and Marx proposed related criteria for choosing the optimal penalty in P-splines.[4][6]\n\n## Origin\n\nThe name \"spline function\" refers to piecewise polynomial functions, after the draftsman's mechanical spline, and B-splines appeared in a 1946 paper on the approximation of equidistant data by analytic functions, the same paper in which splines were introduced.[4][13] The difference formulation of B-splines generalizes to divided differences on arbitrary points.[13] The B-spline basis is used for computational implementation of spline regression, with the recursion\n\n[7]\n\nSpline smoothing in its modern form is a smoothing-spline problem with an explicit solution.[14][15] Craven and Wahba's 1979 paper in Numerische Mathematik introduced GCV for choosing the smoothing parameter,[16] and Wahba showed in 1978, in the Journal of the Royal Statistical Society Series B, that spline smoothing is equivalent to Bayesian estimation with a partially improper prior.[17]\n\n## Variants\n\n**Penalized splines** come in two families. P-splines, introduced by Paul H. C. Eilers and Brian D. Marx in a 1996 Statistical Science paper, combine regression on a relatively large number of B-splines with a penalty on (higher-order) differences of adjacent coefficients, which tunes smoothness continuously and eliminates problems with missing support; the idea of penalized spline smoothing traces back to O'Sullivan (1986), and the difference penalty connects to the familiar integral-of-squared-second-derivative penalty and to Whittaker's 1922 graduation method as a zeroth-order case.[6][19][20][7] The second family uses truncated power functions with knots at quantiles of the predictor and a ridge penalty, as in the Semiparametric Regression framework of David Ruppert, M. P. Wand, and R. J. Carroll, published by [Cambridge University Press](https://www.edgepedia.org/cambridge-university-press) in 2003.[21]\n\n**Natural splines** add boundary constraints requiring linearity beyond the extreme knots, trading interior flexibility for stable extrapolation.[4] **Thin plate regression splines**, introduced by [Simon N. Wood](https://www.edgepedia.org/simon-n-wood) in a 2003 JRSS-B paper, are low-rank smoothers built by transformation and truncation of the thin plate spline basis, optimal in the sense of minimum perturbation of the thin plate smoothing problem for a given basis dimension; they remove the knot placement problem and support multidimensional smooth terms in GAMs.[22] **MARS** (multivariate adaptive regression splines), introduced by [Jerome H. Friedman](https://www.edgepedia.org/jerome-h-friedman) in a 1991 [Annals of Statistics](https://www.edgepedia.org/annals-of-statistics) paper, models high-dimensional data as an expansion in product spline basis functions whose number, product degree, and knot locations are determined automatically from the data; unlike recursive partitioning it produces continuous models with continuous derivatives.[23] P-splines have been extended to varying-coefficient models, quantile and expectile smoothing, periodic data, shape constraints, and tensor products of B-splines for multiple dimensions.[24] Adaptive generalized P-splines for functional data over complex domains were developed by Anna De Magistris, Elvira Romano, and Rosanna Campagna in a 2025 Statistics and Computing article, via a blockwise generalized SVD framework.[34]\n\n## Applications\n\nGeneralized additive models allow flexible functional dependence of a response on covariates by summing spline terms, fit through a penalized likelihood; in GAMs with smoothing splines, backfitting repeatedly updates each predictor's function while holding the others fixed, and the Python package pygam implements this approach.[7][8] Such GAMs are widely applied in medical and epidemiological research.[8] Wood's thin plate regression splines provide a way of incorporating multidimensional smooth terms into GAMs with well-founded model selection via GCV, generalized maximum likelihood, or AIC.[22] P-spline applications demonstrated in the introducing paper include nonparametric logistic regression, density estimation, and scatterplot smoothing.[6]\n\n## Limitations and alternatives\n\nRegression and smoothing splines share the same asymptotic convergence rates under optimal smoothing parameters, though for very large data sets practical results may differ.[4]\n\nCompared with kernel smoothers, splines are computationally faster and, once the basis functions are calculated, simpler, and they let the analyst control curvature directly; kernels are easier to program but slower to run.[30] For local constant and local linear kernel regression the optimal bandwidth is of order \\( n^{-1/5} \\), trading bias against variance, and local constant (Nadaraya–Watson) regression suffers additional bias near the boundaries, which local linear regression avoids.[12] No published head-to-head benchmark has settled comparisons with Gaussian processes or tree-based methods, and Runge-type oscillation is not documented as a spline failure mode in the published literature; the boundary-variance and extrapolation problems above are the documented failure modes.\n\n## References\n\n---\n*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Statistical inference, estimation, sampling, and testing › Regression analysis › Spline and basis-expansion regression*\n\n*Initially written Sep 29, 2026 · Reviewed: Sep 30, 2026 · Edited: Sep 30, 2026 · Last review: Sep 30, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n",
 "same_as": [],
 "url": "https://www.edgepedia.org/spline-regression",
 "markdown_url": "https://www.edgepedia.org/spline-regression.md",
 "license": {
  "name": "Edgepedia Community License 1.0",
  "url": "https://www.edgepedia.org/edgepedia/license",
  "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.",
  "spdx": "LicenseRef-Edgepedia-Community-1.0"
 },
 "credit": "\"Spline regression\", Edgepedia (EdgeChat), https://www.edgepedia.org/spline-regression. Edgepedia Community License 1.0.",
 "credit_md": "\"[Spline regression](https://www.edgepedia.org/spline-regression)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/spline-regression](https://www.edgepedia.org/spline-regression). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).",
 "credit_html": "\"<a href=\"https://www.edgepedia.org/spline-regression\">Spline regression</a>\", Edgepedia (EdgeChat), <a href=\"https://www.edgepedia.org/spline-regression\">https://www.edgepedia.org/spline-regression</a>. <a href=\"https://www.edgepedia.org/edgepedia/license\">Edgepedia Community License 1.0</a>.",
 "speakable": "Spline regression fits piecewise polynomials, typically cubic, constrained to join smoothly at knots to model nonlinear relationships; variants include smoothing splines, P-splines, natural splines, and Friedman's MARS."
}