Markov Chain Simulator
Build a transition matrix and explore how probabilities move between states. Simulate a Markov chain over multiple steps, inspect each probability distribution, visualize the results, and estimate a stationary distribution.
Build your Markov chain
Choose the number of states and enter the transition probabilities.
Transition matrix P
Initial probability distribution
Enter the probability of starting in each state. These values must also add up to 1.
Simulation results
The distribution after each transition is calculated using v(n + 1) = v(n)P.
Current probability distribution
Probability distribution by state
Probability over time
Transition history
Each row shows the probability distribution after the indicated number of transitions.
| Step | State 1 | State 2 | State 3 |
|---|---|---|---|
| 0 | 0.3333 | 0.3333 | 0.3333 |
Stationary distribution estimate
The simulator uses repeated multiplication to test whether a distribution approaches a fixed point. It also checks the stationary equation πP = π for the estimated result.
Calculation steps
Run the simulation to see the matrix multiplication and probability calculations.
What is a Markov chain?
A Markov chain is a stochastic process that moves between a finite or countable collection of possible states. Its defining feature is the Markov property: the probability of the next state depends on the current state, not on the full sequence of states visited before it.
For example, a simple weather model might have three states: Sunny, Cloudy, and Rainy. A transition matrix describes the probability of tomorrow's weather given today's weather. Repeated matrix multiplication then predicts the probability distribution over future days.
The transition matrix
A Markov chain with n states can be represented by an n × n transition matrix P. In this simulator, the entry Pij represents the probability of moving from state i to state j.
Every entry must be nonnegative, and every row must sum to one. A matrix satisfying these conditions is called a row-stochastic matrix.
Calculating future probabilities
The initial probability distribution is represented by a row vector:
The probability distribution after one transition is obtained by multiplying the initial distribution by the transition matrix:
After k transitions, the distribution is:
This is the matrix-based foundation of Markov chain simulation. The simulator applies the transition matrix repeatedly and displays the resulting probabilities at each step.
Stationary distributions
A stationary distribution is a probability vector π that remains unchanged after a transition:
If a chain has a stationary distribution, it describes a probability distribution that is preserved by the transition process. For a finite, irreducible, and aperiodic Markov chain, the distribution from any valid initial state converges to a unique stationary distribution. Other chains may have multiple stationary distributions or may oscillate rather than converge.
Common applications of Markov chains
- Weather forecasting: modeling transitions between weather conditions.
- Finance: modeling changes between credit ratings or market regimes.
- Web navigation: modeling movement between web pages and ranking pages.
- Queueing systems: analyzing customers moving through service states.
- Reliability: modeling equipment moving between working and failed states.
- Biology: representing changes between genetic or biological states.
Frequently asked questions
What is a Markov chain?
A Markov chain is a mathematical model that describes a sequence of transitions between states. The probability of the next state depends only on the current state, rather than the complete history of previous states. This is known as the Markov property.
What is a transition matrix?
A transition matrix contains the probabilities of moving from one state to another. In this simulator, each row represents the current state and each column represents the next state. Every entry must be between zero and one, and each row must sum to one.
How do you calculate the next state distribution?
Using the row-vector convention in this simulator, multiply the current probability distribution by the transition matrix: v(n+1) = v(n)P. Repeating this operation calculates the distribution after multiple transitions.
What is a stationary distribution?
A stationary distribution is a probability vector that remains unchanged after applying the transition matrix. It satisfies πP = π. A stationary distribution may exist even when the simulated distribution does not converge to it from every starting distribution.
What is the difference between a state and a transition?
A state represents a possible condition or position in a system, such as sunny weather or rainy weather. A transition represents a possible movement from one state to another, with a probability assigned to that movement.
Where are Markov chains used?
Markov chains are used in probability theory, weather modeling, finance, queueing systems, inventory management, genetics, reliability analysis, board games, and algorithms such as PageRank.
Does this simulator support any number of states?
This interactive simulator supports transition matrices with two to five states. It is designed for learning and exploring finite-state Markov chains rather than solving very large stochastic models.
Related matrix calculators and simulators
Explore more tools for matrix operations, transformations, and probability applications.