4.19 Transition matrices and Markov chains

4.19 Transition matrices and Markov chains Flashcards

IB Maths: Applications and Interpretation HL: Flashcards

Studying this course? Keep your progress in one place.

Flip each card. Be honest: the ones you mark Not yet come back first.

12 cards

Tap a card to flip it. Swipe right if you knew it, left if not.

See all 12 cards
What is a transition matrix?
A square matrix of probabilities of moving between states; TijT_{ij} is the probability of moving from state jj to state ii.
Which sums to 1 in TT: rows or columns?
Each column.
State the formula for the state matrix after nn transitions.
sn=Tns0\mathbf{s}_n=T^n\mathbf{s}_0
Order of multiplication for s1\mathbf{s}_1?
s1=Ts0\mathbf{s}_1=T\mathbf{s}_0 (matrix on the left, state column on the right).
What does a transition diagram show?
Nodes for states and arrows labelled with the transition probabilities (including loops).
What is a regular Markov chain?
One where some power of TT has all entries positive.
What is the steady state?
The state matrix s\mathbf{s} with Ts=sT\mathbf{s}=\mathbf{s}; it no longer changes.
How do you find the steady state exactly?
Solve Ts=sT\mathbf{s}=\mathbf{s} together with the entries summing to 1.
How do you find the long-term probabilities with your GDC?
Calculate Tns0T^n\mathbf{s}_0 for a large nn until the values stop changing.
Link between steady state and eigenvalues?
The steady state is the eigenvector for eigenvalue 11, scaled so its entries sum to 1.
Does the steady state of a regular chain depend on s0\mathbf{s}_0?
No, it is the same whatever the starting state.
What does (Tn)ij(T^n)_{ij} represent?
The probability of going from state jj to state ii in exactly nn steps.

Flashcards