# Classification of states in Markov chains

Classification of states is the taxonomy that sorts the states of a countable-state, discrete-time [Markov chain](https://www.edgepedia.org/markov-chain) into communicating classes, recurrence types, and periods. The classification matters because it records which classes the chain can never leave, whether returns to a state are certain, how long returns take on average, and at which time steps returns are even possible.<sup>[1](https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf)</sup><sup> • </sup><sup>[2](https://ocw.mit.edu/courses/6-262-discrete-stochastic-processes-spring-2011/3558b08622765d26c2b0a7d2eeeac885_MIT6_262S11_chap03.pdf)</sup><sup> • </sup><sup>[3](http://www.statslab.cam.ac.uk/%7Errw1/markov/M.pdf)</sup>

| Key fact | Statement |
|---|---|
| Recurrence | State i is recurrent if \(P_i(T_i < \infty) = 1\), transient otherwise<sup>[1](https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf)</sup> |
| Sum criterion | i is recurrent iff \(\sum_{n} p_{ii}^{(n)} = \infty\)<sup>[4](https://gordanz.github.io/stochastic-book/classification-of-states.html)</sup> |
| Positive vs null | Positive recurrent means \(E_{i}(T_{i}) < \infty\); recurrent with infinite mean return time is null recurrent<sup>[1](https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf)</sup> |
| Class solidarity | Recurrence type and period are shared by all states in a communicating class<sup>[5](http://www.columbia.edu/~ww2040/4701Sum07/4701-06-Notes-MCII.pdf)</sup><sup> • </sup><sup>[6](https://www.probabilitycourse.com/chapter11/11_2_4_classification_of_states.php)</sup> |
| Finite chains | Every recurrent state of a finite chain is positive recurrent; the null case never arises<sup>[2](https://ocw.mit.edu/courses/6-262-discrete-stochastic-processes-spring-2011/3558b08622765d26c2b0a7d2eeeac885_MIT6_262S11_chap03.pdf)</sup><sup> • </sup><sup>[7](https://www.sjsu.edu/faculty/guangliang.chen/Math263/lec3classification.pdf)</sup> |
| Random walks | Simple symmetric random walk is recurrent in dimensions 1 and 2, transient in dimension 3 and higher<sup>[4](https://gordanz.github.io/stochastic-book/classification-of-states.html)</sup> |
| Period | d(i) = gcd of the return-time set; the simple random walk has period 2<sup>[1](https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf)</sup><sup> • </sup><sup>[8](https://mpaldridge.github.io/math2750/S07-classes.html)</sup> |

## Communicating classes, irreducibility, closed and absorbing classes

Two states i and j communicate (written i ↔ j) when each is accessible from the other.<sup>[9](https://reference.wolframcloud.com/cloudplatform/ref/MarkovProcessProperties.html)</sup> Communicating classes are sets of states accessible from each other.<sup>[9](https://reference.wolframcloud.com/cloudplatform/ref/MarkovProcessProperties.html)</sup>

A set B of states is closed if no state in B leads outside B; once the chain enters a closed set, it stays there forever. A state whose singleton set {i} is closed is called absorbing.<sup>[1](https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf)</sup> Within a communicating class, states are either all transient or all recurrent; every recurrent class is closed, and every finite closed class is recurrent.<sup>[10](https://www.math.umd.edu/~mariakc/REU2022/REU2022_MarkovChains.pdf)</sup>

In practice, communicating classes can be computed directly from the transition matrix; software such as Wolfram's MarkovProcessProperties reports communicating classes, recurrent classes, transient classes, periods, and whether the chain is "primitive", meaning irreducible and aperiodic.<sup>[9](https://reference.wolframcloud.com/cloudplatform/ref/MarkovProcessProperties.html)</sup>

## Recurrence and transience

 Let \(T_{i}\) be the first return time to state i. State i is recurrent if \(P_{i}(T_{i} < \infty) = 1\), and transient otherwise, that is, when there is a positive probability of never returning.<sup>[1](https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf)</sup> Equivalently, a recurrent state is visited infinitely often with probability 1, \(P_{i}(V_{i} = \infty) = 1\).<sup>[3](http://www.statslab.cam.ac.uk/%7Errw1/markov/M.pdf)</sup>

The Recurrence Criterion turns this into a computable test on the transition matrix: state i is recurrent if and only if the sum of return probabilities \(\sum_{n \in \mathbb{N}} p_{ii}^{(n)}\) diverges; if the return probability \(f_i = P_i(T_i < \infty)\) is below 1, the expected number of visits to i is \(f_i/(1 - f_i)\), which is finite.<sup>[4](https://gordanz.github.io/stochastic-book/classification-of-states.html)</sup> The dichotomy is sharp: \(P_i(T_i < \infty) = 1\) forces the sum to diverge, while \(P_i(T_i < \infty) < 1\) forces it to converge.<sup>[10](https://www.math.umd.edu/~mariakc/REU2022/REU2022_MarkovChains.pdf)</sup>

For finite-state chains there is a shortcut: state i is recurrent exactly when it is accessible from every state accessible from it (i → j implies j → i).<sup>[2](https://ocw.mit.edu/courses/6-262-discrete-stochastic-processes-spring-2011/3558b08622765d26c2b0a7d2eeeac885_MIT6_262S11_chap03.pdf)</sup>

## Positive and null recurrence

A recurrent state is positive recurrent if the expected return time \(E(\tau_{jj})\) is finite, and null recurrent if it is recurrent with \(E(\tau_{jj}) = \infty\). Positive recurrence implies recurrence, but not conversely.<sup>[5](http://www.columbia.edu/~ww2040/4701Sum07/4701-06-Notes-MCII.pdf)</sup> A useful equivalent test: state i is null if and only if \(\lim_{n \to \infty} p_{ii}^{(n)} = 0\).<sup>[11](https://www.math.purdue.edu/%7Estindel/teaching/ma532/slides/discrete-markov-chains.pdf)</sup>

Both recurrence types are class properties: if i ↔ j and i is positive recurrent, then j is positive recurrent as well, and likewise for null recurrence. All states in a communication class are together positive recurrent, null recurrent, or transient; in an irreducible chain the whole chain shares one type.<sup>[5](http://www.columbia.edu/~ww2040/4701Sum07/4701-06-Notes-MCII.pdf)</sup><sup> • </sup><sup>[7](https://www.sjsu.edu/faculty/guangliang.chen/Math263/lec3classification.pdf)</sup> A reducible chain can therefore mix types across its classes, but never within one.

The positive/null distinction only exists for countably infinite state spaces. In a finite Markov chain, all recurrent states are positive recurrent.<sup>[2](https://ocw.mit.edu/courses/6-262-discrete-stochastic-processes-spring-2011/3558b08622765d26c2b0a7d2eeeac885_MIT6_262S11_chap03.pdf)</sup><sup> • </sup><sup>[7](https://www.sjsu.edu/faculty/guangliang.chen/Math263/lec3classification.pdf)</sup>

A stationary-measure test completes the toolkit. Every irreducible finite-state Markov chain is positive recurrent and has a unique stationary distribution, the probability solution to \(\pi = \pi \cdot P\); finding such a probability solution is a practical test for positive recurrence.<sup>[5](http://www.columbia.edu/~ww2040/4701Sum07/4701-06-Notes-MCII.pdf)</sup> For a general irreducible recurrent chain, an invariant measure \(x\) satisfying \(x = x \cdot P\) exists and is unique up to a multiplicative constant; the chain is positive if \(\sum_i x_i < \infty\) and null if \(\sum_i x_i = \infty\).<sup>[11](https://www.math.purdue.edu/%7Estindel/teaching/ma532/slides/discrete-markov-chains.pdf)</sup>

## Period and aperiodicity

The period of state i is the greatest common divisor of its return-time set \(R(i) = \{ n \in \mathbb{N} : p_{ii}^{(n)} > 0 \}\); state i is aperiodic if \(d(i) = 1\).<sup>[1](https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf)</sup> Equivalently, \(d(i)\) is the largest integer d such that \(p_{ii}^{(n)} = 0\) whenever n is not divisible by d.<sup>[6](https://www.probabilitycourse.com/chapter11/11_2_4_classification_of_states.php)</sup>

Period is a class property: all states in a communicating class have the same period, called the period of the class, so an irreducible chain has a single common period.<sup>[6](https://www.probabilitycourse.com/chapter11/11_2_4_classification_of_states.php)</sup><sup> • </sup><sup>[8](https://mpaldridge.github.io/math2750/S07-classes.html)</sup> Two quick tests cover most examples. If \(p_{ii} > 0\), the state has a self-transition and is aperiodic, since 1 is co-prime to every integer. More generally, if there exist co-prime numbers l and m with \(p_{ii}^{(l)} > 0\) and \(p_{ii}^{(m)} > 0\), then i is aperiodic.<sup>[6](https://www.probabilitycourse.com/chapter11/11_2_4_classification_of_states.php)</sup> For the simple random walk with \(p \notin \{0,1\}\), returns occur only at even times (\(p_{ii}^{(n)} = 0\) for odd n, while \(p_{ii}^{(2)} = 2 \cdot p \cdot q > 0\)), so all states have period \(\gcd\{2,4,6,\ldots\} = 2\).<sup>[8](https://mpaldridge.github.io/math2750/S07-classes.html)</sup>

One convention differs between references: when the return set R(i) is empty, so no return is possible at all, the UT Austin notes set d(i) = 1 (aperiodic by convention)<sup>[1](https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf)</sup>, while ProbabilityCourse.com sets d(i) = ∞<sup>[6](https://www.probabilitycourse.com/chapter11/11_2_4_classification_of_states.php)</sup>.

Aperiodicity matters downstream because convergence theorems require it. Gallager calls a class that is both recurrent and aperiodic an ergodic class, and a chain consisting of one ergodic class an ergodic chain.<sup>[2](https://ocw.mit.edu/courses/6-262-discrete-stochastic-processes-spring-2011/3558b08622765d26c2b0a7d2eeeac885_MIT6_262S11_chap03.pdf)</sup>

## By the numbers: random walks on Z, Z², Z³

Pólya's theorem draws the dimensional line: the simple symmetric random walk is recurrent for \(d = 1\) and \(d = 2\), but transient for \(d \geq 3\).<sup>[4](https://gordanz.github.io/stochastic-book/classification-of-states.html)</sup> The asymptotics behind the theorem show why. In dimension 2, the return probability at even times satisfies \(p^{(2m)} \sim 1/(\pi \cdot m)\), and the harmonic series \(\sum 1/m\) diverges, so the sum criterion gives recurrence. In dimension 3 and higher, \(p^{(2m)} \sim C \cdot (3/m)^{3/2}\) for some constant \(C\), a convergent p-series, so the walk is transient.<sup>[4](https://gordanz.github.io/stochastic-book/classification-of-states.html)</sup>

On the one-dimensional lattice Z, the picture depends on the drift. For the simple random walk with parameters p and q = 1−p, the states are periodic with period 2, transient if p ≠ 1/2, and null recurrent if p = 1/2.<sup>[11](https://www.math.purdue.edu/%7Estindel/teaching/ma532/slides/discrete-markov-chains.pdf)</sup> The symmetric walk on Z is thus recurrent but with infinite expected return time, while any bias destroys recurrence entirely. In 3D or higher, all states of the symmetric random walk are transient.<sup>[7](https://www.sjsu.edu/faculty/guangliang.chen/Math263/lec3classification.pdf)</sup>

## Reducible chains: a worked example and what changes

The gambler's ruin chain on states {0, 1, …, a} shows how a reducible chain mixes classifications. The absorbing states 0 and a are trivially positive recurrent, with period 1, while all intermediate states are transient.<sup>[4](https://gordanz.github.io/stochastic-book/classification-of-states.html)</sup> The example also illustrates period mixing across classes: the transient interior carries period 2, while the absorbing endpoints have period 1.<sup>[4](https://gordanz.github.io/stochastic-book/classification-of-states.html)</sup>

What irreducibility guarantees is uniformity, not any particular type. Since positive recurrence, null recurrence, and transience are communication class properties, an irreducible chain shares a single type throughout<sup>[5](http://www.columbia.edu/~ww2040/4701Sum07/4701-06-Notes-MCII.pdf)</sup><sup> • </sup><sup>[7](https://www.sjsu.edu/faculty/guangliang.chen/Math263/lec3classification.pdf)</sup>; in a reducible chain, each closed recurrent class carries its own type and period, and transient states sit outside them.<sup>[10](https://www.math.umd.edu/~mariakc/REU2022/REU2022_MarkovChains.pdf)</sup>

## How it compares with hitting times, stationary distributions and convergence

The classification is the entry condition for the sibling topics. For stationary distributions, a positive recurrent chain has a unique stationary distribution π, while a null recurrent or transient chain has none, and the limits of the n-step probabilities are then all 0 with probability 1.<sup>[12](http://www.columbia.edu/~ks20/4106-18-Fall/Notes-MCII.pdf)</sup> [Existence](https://www.edgepedia.org/existence) of a stationary distribution is therefore exactly as strong as positive recurrence in the irreducible case.<sup>[5](http://www.columbia.edu/~ww2040/4701Sum07/4701-06-Notes-MCII.pdf)</sup>

For convergence to equilibrium, the convergence theorem requires all three classifications at once: an ergodic Markov chain, meaning irreducible, aperiodic, and positive recurrent, with invariant distribution \(\pi\), satisfies \(P(X_{n} = j) \to \pi_{j}\) for any initial distribution.<sup>[3](http://www.statslab.cam.ac.uk/%7Errw1/markov/M.pdf)</sup>

## References

1. Lecture 8: Classification of states, UT Austin. https://web.ma.utexas.edu/users/gordanz/notes/classification_of_states_color.pdf
2. Discrete Stochastic Processes, Chapter 3: Finite-State Markov Chains, MIT OCW (Gallager). https://ocw.mit.edu/courses/6-262-discrete-stochastic-processes-spring-2011/3558b08622765d26c2b0a7d2eeeac885_MIT6_262S11_chap03.pdf
3. Markov Chains, Cambridge lecture notes (R. R. Weber). http://www.statslab.cam.ac.uk/%7Errw1/markov/M.pdf
4. Chapter 6: Classification of States, Introduction to Stochastic Processes (Zitković). https://gordanz.github.io/stochastic-book/classification-of-states.html
5. Communication classes and irreducibility for Markov chains, Columbia. http://www.columbia.edu/~ww2040/4701Sum07/4701-06-Notes-MCII.pdf
6. Classification of States, ProbabilityCourse.com. https://www.probabilitycourse.com/chapter11/11_2_4_classification_of_states.php
7. Classification of States, SJSU Math 263. https://www.sjsu.edu/faculty/guangliang.chen/Math263/lec3classification.pdf
8. Section 7 Class structure, MATH2750 Introduction to Markov Processes. https://mpaldridge.github.io/math2750/S07-classes.html
9. MarkovProcessProperties, Wolfram Documentation. https://reference.wolframcloud.com/cloudplatform/ref/MarkovProcessProperties.html
10. Markov Chains, UMD REU 2022 notes. https://www.math.umd.edu/~mariakc/REU2022/REU2022_MarkovChains.pdf
11. Discrete time Markov chains, Purdue MA 532 slides. https://www.math.purdue.edu/%7Estindel/teaching/ma532/slides/discrete-markov-chains.pdf
12. Limiting distribution for a Markov chain, Columbia lecture notes. http://www.columbia.edu/~ks20/4106-18-Fall/Notes-MCII.pdf

---
*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Stochastic processes › Markov chains and processes › Discrete-time Markov chains › Classification of states*

*Initially written Sep 17, 2026 · Reviewed: Sep 30, 2026 · Edited: Oct 11, 2026 · Last review: Sep 30, 2026*

*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*

License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license