4.19 Transition matrices and Markov chains Flashcards
Studying this course? Keep your progress in one place.

Flip each card. Be honest: the ones you mark Not yet come back first.
12 cards
Tap a card to flip it. Swipe right if you knew it, left if not.
See all 12 cards
- What is a transition matrix?
- A square matrix of probabilities of moving between states; is the probability of moving from state to state .
- Which sums to 1 in : rows or columns?
- Each column.
- State the formula for the state matrix after transitions.
- Order of multiplication for ?
- (matrix on the left, state column on the right).
- What does a transition diagram show?
- Nodes for states and arrows labelled with the transition probabilities (including loops).
- What is a regular Markov chain?
- One where some power of has all entries positive.
- What is the steady state?
- The state matrix with ; it no longer changes.
- How do you find the steady state exactly?
- Solve together with the entries summing to 1.
- How do you find the long-term probabilities with your GDC?
- Calculate for a large until the values stop changing.
- Link between steady state and eigenvalues?
- The steady state is the eigenvector for eigenvalue , scaled so its entries sum to 1.
- Does the steady state of a regular chain depend on ?
- No, it is the same whatever the starting state.
- What does represent?
- The probability of going from state to state in exactly steps.