Topic 5 of 7

Moment generating functions

Step 22 · Probability theory topics

Pack a distribution into its moment generating function, read off the mean and variance by differentiating, and use these functions to find distributions of sums.

Defining the moment generating function

The moment generating function, or MGF, of X is $M(t) = E(e^{tX})$ . It packs every moment E(Xⁿ) into one function, and two distributions with the same MGF are the same distribution.

Example: A coin

X is 1 with probability p and 0 otherwise. Find its MGF.

$E(e^{tX}) = (1 - p)e^{0} + pe^{t}$ .

Answer $M(t) = 1 - p + pe^{t}$

Example: An exponential

X has density $\lambda e^{-\lambda x}$ for x ≥ 0. Find its MGF.

$\int_{0}^{\infty} e^{tx}\lambda e^{-\lambda x}\,dx = \frac{\lambda}{\lambda - t}$ , for t < λ.

Answer $M(t) = \frac{\lambda}{\lambda - t}$

Distribution	MGF
Binomial(n, p)	$(1 - p + pe^{t})^{n}$
Poisson(λ)	$e^{\lambda(e^{t} - 1)}$
Exponential(λ)	$\frac{\lambda}{\lambda - t}$
Normal(μ, σ²)	$e^{\mu t + \sigma^{2}t^{2}/2}$

Moments from the MGF

The nth derivative of M at t = 0 is the nth moment, E(Xⁿ). So the mean is M′(0), and the variance is M″(0) − M′(0)².

E(X^{n}) = M^{(n)}(0)

Example: Exponential moments

Use $M(t) = \frac{\lambda}{\lambda - t}$ to find E(X) and Var(X).

$M'(t) = \frac{\lambda}{(\lambda - t)^{2}}$ , so E(X) = 1/λ.
$M''(t) = \frac{2\lambda}{(\lambda - t)^{3}}$ , so E(X²) = 2/λ².
Var(X) = 2/λ² − 1/λ².

AnswerMean $\frac{1}{\lambda}$ , variance $\frac{1}{\lambda^{2}}$

Example: A Poisson mean

Use $M(t) = e^{\lambda(e^{t} - 1)}$ to find the mean.

$M'(t) = \lambda e^{t}M(t)$ , and M(0) = 1.

Answerλ

Sums of independent variables

If X and Y are independent, the MGF of X + Y is the product of their MGFs. Multiplying MGFs is far easier than working out the distribution of a sum directly.

M_{X+Y}(t) = M_X(t)\,M_Y(t)

Example: Binomial from coins

Add n independent coin flips, each 1 with probability p.

Each has MGF $1 - p + pe^{t}$ .
The sum has MGF $(1 - p + pe^{t})^{n}$ : binomial(n, p).

AnswerA binomial(n, p) count

Example: Poisson sums

X is Poisson(2) and Y is Poisson(3), and they are independent. What is the distribution of X + Y?

$e^{2(e^{t} - 1)} \cdot e^{3(e^{t} - 1)} = e^{5(e^{t} - 1)}$ .

AnswerPoisson(5)

Try it yourself

Work each one on paper first, then open the answer to check.

What is M(0) for any random variable?
Show answer
1
What distribution has MGF $(0.5 + 0.5e^{t})^{10}$ ?
Show answer
Binomial with n = 10 and p = 0.5
What are the mean and variance of a variable with MGF $e^{2t + 8t^{2}}$ ?
Show answer
Mean 2, variance 16
M′(0) = 3 and M″(0) = 13. What is Var(X)?
Show answer
4
X and Y are independent with MGFs M₁ and M₂. What is the MGF of X + Y?
Show answer
M₁(t) M₂(t)
X is Poisson(1) and Y is Poisson(4), and they are independent. What is X + Y?
Show answer
Poisson(5)