Probability generating functions

Probability generating functions

Spec 7.1–3 · Edexcel A-Level Further Maths: Revision notes

Studying this course? Keep your progress in one place.

Key facts

  • The probability generating function is GX(t)=E(tX)=∑P(X=r)tr\mathrm{G}_X(t)=\mathrm{E}(t^X)=\sum\mathrm{P}(X=r)t^r: the coefficient of trt^r is P(X=r)\mathrm{P}(X=r).
  • Always GX(1)=1\mathrm{G}_X(1)=1: use it as a check.
  • E(X)=GX′(1)\mathrm{E}(X)=\mathrm{G}_X'(1) and Var(X)=GX′′(1)+GX′(1)−[GX′(1)]2\mathrm{Var}(X)=\mathrm{G}_X''(1)+\mathrm{G}_X'(1)-\left[\mathrm{G}_X'(1)\right]^2.
  • For independent XX and YY: GX+Y(t)=GX(t) GY(t)\mathrm{G}_{X+Y}(t)=\mathrm{G}_X(t)\,\mathrm{G}_Y(t).
  • Know the PGFs of the binomial, Poisson, geometric and negative binomial distributions.

Definition and basic properties

A PGF is a power series whose coefficients are the probabilities.

For a discrete random variable XX taking non-negative integer values, the probability generating function (PGF) is GX(t)=E(tX)=∑rP(X=r) tr\mathrm{G}_X(t)=\mathrm{E}(t^X)=\sum_r\mathrm{P}(X=r)\,t^r.

The coefficient of trt^r is P(X=r)\mathrm{P}(X=r), so you can read off probabilities from a PGF written as a polynomial or power series.

Since probabilities sum to 1, GX(1)=1\mathrm{G}_X(1)=1 always. Check this every time: if it fails, an error has crept in.

For P(X=0)=0.2\mathrm{P}(X=0)=0.2, P(X=1)=0.5\mathrm{P}(X=1)=0.5, P(X=2)=0.3\mathrm{P}(X=2)=0.3: GX(t)=0.2+0.5t+0.3t2\mathrm{G}_X(t)=0.2+0.5t+0.3t^2.

00.10.20.30.40.5012rP(X = r)
The coefficients of 0.2 + 0.5t + 0.3t² are the probabilities.
  • Probability generating functionGX(t)=E(tX)\mathrm{G}_X(t)=\mathrm{E}(t^X)

A PGF is GX(t)=0.1+0.4t+0.3t2+kt3\mathrm{G}_X(t)=0.1+0.4t+0.3t^2+kt^3. What is kk?

Mean and variance from the PGF

Differentiate and set t=1t=1: the first derivative gives the mean, and the second gives the variance.

Differentiating term by term, GX′(t)=∑r P(X=r)tr−1\mathrm{G}_X'(t)=\sum r\,\mathrm{P}(X=r)t^{r-1}, so at t=1t=1 the mean is E(X)=GX′(1)\mathrm{E}(X)=\mathrm{G}_X'(1).

Differentiating again gives GX′′(1)=E[X(X−1)]=E(X2)−E(X)\mathrm{G}_X''(1)=\mathrm{E}[X(X-1)]=\mathrm{E}(X^2)-\mathrm{E}(X). Hence Var(X)=GX′′(1)+GX′(1)−[GX′(1)]2\mathrm{Var}(X)=\mathrm{G}_X''(1)+\mathrm{G}_X'(1)-\left[\mathrm{G}_X'(1)\right]^2.

Proofs of these standard results may be asked for in the exam. Do not use G′′(1)\mathrm{G}''(1) as the variance, and do not forget to subtract [G′(1)]2[\mathrm{G}'(1)]^2.

0.20.40.60.811.2123456xyt = 4/3(1, 1)G(t) = t/(4 − 3t)tangent at t = 1
G(t) = t/(4 − 3t) for Geo(¼). The gradient of the tangent at t = 1 is G′(1) = 4, the mean.
  • E(X)\mathrm{E}(X)GX′(1)\mathrm{G}_X'(1)
  • Var(X)\mathrm{Var}(X)GX′′(1)+GX′(1)−[GX′(1)]2\mathrm{G}_X''(1)+\mathrm{G}_X'(1)-\left[\mathrm{G}_X'(1)\right]^2

Worked example

Find E(X)\mathrm{E}(X) and Var(X)\mathrm{Var}(X) when GX(t)=0.2+0.5t+0.3t2\mathrm{G}_X(t)=0.2+0.5t+0.3t^2.

If GX′(1)=3\mathrm{G}_X'(1)=3 and GX′′(1)=10\mathrm{G}_X''(1)=10, what is Var(X)\mathrm{Var}(X)?

PGFs of standard distributions

Each PGF comes straight from the definition, using the binomial theorem, the exponential series or a geometric series.

You must be able to derive each PGF from the definition and use it for the mean and variance.

  • Binomial B(n,p)\mathrm{B}(n,p): G(t)=(q+pt)n\mathrm{G}(t)=(q+pt)^n, from the binomial theorem.
  • Poisson Po(λ)\mathrm{Po}(\lambda): G(t)=eλ(t−1)\mathrm{G}(t)=\mathrm{e}^{\lambda(t-1)}, from ∑e−λ(λt)rr!\sum\mathrm{e}^{-\lambda}\frac{(\lambda t)^r}{r!}.
  • Geometric (trials to the first success): G(t)=pt1−qt\mathrm{G}(t)=\dfrac{pt}{1-qt}, from a geometric series, valid for ∣qt∣<1|qt|<1.
  • Negative binomial (trials to the rrth success): G(t)=(pt1−qt)r\mathrm{G}(t)=\left(\dfrac{pt}{1-qt}\right)^r, a sum of rr independent geometric variables.

The geometric PGF counts trials including the success, not failures.

0.10.20.30.40.50.60.70.80.911.11.20.20.40.60.811.21.41.61.82xy(1, 1)Po(2): e^(2(t−1))B(3, 0.5): (0.5 + 0.5t)³
Illustrative: the PGFs of Po(2) and B(3, 0.5). Every PGF equals 1 at t = 1.

Binomial B(n, p)

  • (q+pt)n(q+pt)^n

Poisson Po(λ)

  • eλ(t−1)\mathrm{e}^{\lambda(t-1)}

Geometric

  • pt1−qt\dfrac{pt}{1-qt}

Negative binomial

  • (pt1−qt)r\left(\dfrac{pt}{1-qt}\right)^r

Which PGF belongs to a Po(3)\mathrm{Po}(3) variable?

Sums of independent variables

For independent variables, the PGFs multiply.

If XX and YY are independent, then E(tX+Y)=E(tX)E(tY)\mathrm{E}(t^{X+Y})=\mathrm{E}(t^X)\mathrm{E}(t^Y), so GX+Y(t)=GX(t)×GY(t)\mathrm{G}_{X+Y}(t)=\mathrm{G}_X(t)\times\mathrm{G}_Y(t). Derivation is not required.

It extends to any number of independent variables: the sum of nn independent copies of XX has PGF [GX(t)]n[\mathrm{G}_X(t)]^n.

Adding PGFs instead of multiplying them is a common error.

0.10.20.30.40.50.60.70.80.911.11.20.20.40.60.811.21.4xy(1, 1)G_X(t) = ½ + ½tG_(X+Y)(t) = (½ + ½t)²
Illustrative: for two independent Bernoulli(½) variables, G(X+Y) = (G_X)². Both curves equal 1 at t = 1.
  • X, YX,\ Y independentGX+Y(t)=GX(t) GY(t)\mathrm{G}_{X+Y}(t)=\mathrm{G}_X(t)\,\mathrm{G}_Y(t)
  • nn independent copies of XX[GX(t)]n[\mathrm{G}_X(t)]^n

Worked example

Find the PGF of B(3,0.4)\mathrm{B}(3,0.4) plus an independent Po(2)\mathrm{Po}(2), and identify the sum of independent Po(λ)\mathrm{Po}(\lambda) and Po(μ)\mathrm{Po}(\mu).

XX and YY are independent with PGFs GX(t)\mathrm{G}_X(t) and GY(t)\mathrm{G}_Y(t). What is the PGF of X+YX+Y?

Worked example: geometric

Write the PGF, differentiate twice, then use the standard formulae.

Take X∼Geo(p)X\sim\mathrm{Geo}(p) with p=14p=\frac14, so q=34q=\frac34. The PGF is G(t)=t4−3t\mathrm{G}(t)=\dfrac{t}{4-3t}.

Simplify G′(t)\mathrm{G}'(t) before differentiating a second time; it saves quotient-rule errors. The result agrees with E(X)=1p\mathrm{E}(X)=\frac1p and Var(X)=qp2\mathrm{Var}(X)=\frac{q}{p^2}.

For the trials NN to the third success, N=X1+X2+X3N=X_1+X_2+X_3 has PGF [GX]3[\mathrm{G}_X]^3.

00.050.10.150.20.25123456rP(X = r)
Geometric with p = ¼: P(X = r) = ¼(¾)^(r−1), falling by a factor of ¾ each trial. Values to 4 d.p.
  • Geo(14)\mathrm{Geo}\left(\frac14\right)G(t)=t4−3t\mathrm{G}(t)=\dfrac{t}{4-3t}

Worked example

For X∼Geo(14)X\sim\mathrm{Geo}\left(\frac14\right), find E(X)\mathrm{E}(X) and Var(X)\mathrm{Var}(X), then the mean of the number of trials NN to the third success.

For a geometric variable with p=12p=\frac12, what is E(X)\mathrm{E}(X)?

Try an exam question

The random variable XX has probability generating function GX(t)=(0.2+0.8t)3\mathrm{G}_X(t)=(0.2+0.8t)^3. Use the PGF to find E(X)\mathrm{E}(X) and Var(X)\mathrm{Var}(X).

[5 marks]

That's the notes covered.

Carry on to the next subtopic.

Try a question

Find GX(t)\mathrm{G}_X(t).

1 mark, marked against the mark scheme

Revision notes