Topic 3 of 7

Expectation, variance, and covariance

Step 22 · Probability theory topics

Compute expected values and variances of discrete and continuous random variables, use their rules for sums and scaling, and measure how two variables move together.

Expected value

The expected value E(X) is the probability-weighted average of X: a sum for discrete variables, an integral for continuous ones. Expectation is linear: E(aX + bY + c) = aE(X) + bE(Y) + c, whether or not X and Y are independent.

E(X) = \sum x\,p(x) \qquad E(X) = \int x f(x)\,dx

Example: A density

f(x) = 2x on [0, 1]. Find E(X).

$\int_{0}^{1} x \cdot 2x\,dx = \frac{2}{3}$ .

Answer $\frac{2}{3}$

Example: Linearity

Roll 10 dice. What is the expected total?

Each die has expected value 3.5.
Expected values add: 10 × 3.5.

Answer35

Example: Indicators

10 people throw their hats in a pile and each takes one at random. How many expect to get their own hat?

Let Xᵢ be 1 if person i gets their own hat, so E(Xᵢ) = 1/10.
Add the 10 expected values.

Answer1, on average

Variance

The variance measures spread: the expected squared distance from the mean. A shortcut is E(X²) − (E X)². Scaling X by a multiplies the variance by a², and for independent variables variances add.

\text{Var}(X) = E\left[(X - \mu)^{2}\right] = E(X^{2}) - \mu^{2}

Example: A die

Find the variance of one fair die roll.

E(X²) = (1 + 4 + 9 + 16 + 25 + 36) ÷ 6, which is $\frac{91}{6}$ .
$\frac{91}{6} - 3.5^{2} = \frac{35}{12}$ .

Answer $\frac{35}{12}$ , about 2.92

Example: The rules

Var(X) = 4. Find Var(3X + 5) and SD(3X + 5).

Adding 5 moves X without spreading it.
Multiplying by 3 multiplies the variance by 9.

AnswerVariance 36, SD 6

Covariance and correlation

Cov(X, Y) = E(XY) − E(X)E(Y) measures how X and Y move together: positive when they rise together, negative when one rises as the other falls. The correlation divides by both SDs, giving a number from −1 to 1.

\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y) + 2\,\text{Cov}(X, Y)

Example: A small table

X and Y are each 0 or 1, with P(1, 1) = 0.4, P(1, 0) = 0.1, P(0, 1) = 0.1 and P(0, 0) = 0.4. Find Cov(X, Y).

E(XY) = 0.4, and E(X) = E(Y) = 0.5.
0.4 − 0.25 = 0.15.

Answer0.15: they tend to agree

Try it yourself

Work each one on paper first, then open the answer to check.

E(X) = 4 and E(Y) = −1. What is E(2X + 3Y)?
Show answer
5
X is 0 or 10 with equal chances. What is Var(X)?
Show answer
25
Var(X) = 9. What is SD(−2X)?
Show answer
6
X and Y are independent with variances 4 and 5. What is Var(X − Y)?
Show answer
9
E(XY) = 7, E(X) = 2 and E(Y) = 3. What is Cov(X, Y)?
Show answer
1
X is uniform on [0, 1]. What is Var(X)?
Show answer
$\frac{1}{12}$