The variance of a mixture

Suppose $X$ is a mixture distribution that is the result of mixing a family of conditional distributions indexed by a parameter random variable $\Theta$ . The uncertainty in the parameter variable $\Theta$ has the effect of increasing the unconditional variance of the mixture $X$ . Thus, $Var(X)$ is not simply the weighted average of the conditional variance $Var(X \lvert \Theta)$ . The unconditional variance $Var(X)$ is the sum of two components. They are:

$\displaystyle Var(X)=E[Var(X \lvert \Theta)]+Var[E(X \lvert \Theta)]$

The above relationship is called the law of total variance, which is the proper way of computing the unconditional variance $Var(X)$ . The first component $E[Var(X \lvert \Theta)]$ is called the expected value of conditional variances, which is the weighted average of the conditional variances. The second component $Var[E(X \lvert \Theta)]$ is called the variance of the conditional means, which represents the additional variance as a result of the uncertainty in the parameter $\Theta$ .

We use an example of a two-point mixture to illustrate the law of total variance. The example is followed by a proof of the total law of variance.

Example
Let $U$ be the uniform distribution on the unit interval $(0, 1)$ . Suppose that a large population of insureds is composed of “high risk” and “low risk” individuals. The proportion of insured classified as “low risk” is $p$ where $0<p<1$ . The random loss amount $X$ of a “low risk” insured is $U$ . The random loss amount $X$ of a “high risk” insured is $U$ shifted by a positive constant $w>0$ , i.e. $w+U$ . What is the variance of the loss amount of an insured randomly selected from this population?

For convenience, we use $\Theta$ as a parameter to indicate the risk class ( $\Theta=1$ is “low risk” and $\Theta=2$ is “high risk”). The following shows the relevant conditional distributional quantities of $X$ .

$\displaystyle E(X \lvert \Theta=1)=\frac{1}{2} \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ E(X \lvert \Theta=2)=w+\frac{1}{2}$

$\displaystyle Var(X \lvert \Theta=1)=\frac{1}{12} \ \ \ \ \ \ \ \ \ \ \ \ Var(X \lvert \Theta=2)=\frac{1}{12}$

The unconditional mean loss is the weighted average of the conditional mean loss amounts. However, the same idea does not work for variance.

$\displaystyle \begin{aligned}E[X]&=p \times E(X \lvert \Theta=1)+(1-p) \times E(X \lvert \Theta=2) \\&=p \times \frac{1}{2}+(1-p) \times (w+\frac{1}{2}) \\&=\frac{1}{2}+(1-p) \ w \end{aligned}$

$\displaystyle \begin{aligned}Var[X]&\ne p \times Var(X \lvert \Theta=1)+(1-p) \times Var(X \lvert \Theta=2)=\frac{1}{12} \end{aligned}$

The conditional variance is the same for both risk classes since the “high risk” loss is a shifted distribution of the “low rosk” loss. However, the unconditional variance is more than $\frac{1}{12}$ since the mean loss for the two casses are different (heterogeneous risks across the classes). The uncertainty in the risk classes (i.e. uncertainty in the parameter $\Theta$ ) introduces additional variance in the loss for a randomly selected insured. The unconditional variance $Var(X)$ is the sum of the following two components:

$\displaystyle \begin{aligned}E[Var(X \lvert \Theta)]&=p \times Var(X \lvert \Theta=1)+(1-p) \times Var(X \lvert \Theta=2) \\&=p \times \frac{1}{12}+(1-p) \times \frac{1}{12} \\&=\frac{1}{12} \end{aligned}$

$\displaystyle \begin{aligned}Var[E(X \lvert \Theta)]&=p \times E(X \lvert \Theta=1)^2+(1-p) \times E(X \lvert \Theta=2)^2 \\&\ \ \ -\biggl(p \times E(X \lvert \Theta=1)+(1-p) \times E(X \lvert \Theta=2)\biggr)^2 \\&=p \times \biggl(\frac{1}{2}\biggr)^2+(1-p) \times \biggl(w+\frac{1}{2}\biggr)^2 \\&\ \ \ -\biggl(p \times \frac{1}{2}+(1-p) \times (w+\frac{1}{2})\biggr)^2 \\&=p \ (1-p) \ w^2 \end{aligned}$

$\displaystyle \begin{aligned}Var(X)&=E[Var(X \lvert \Theta)]+Var[E(X \lvert \Theta)] \\&=\frac{1}{12}+p \ (1-p) \ w^2 \end{aligned}$

The additional variance is in the amount of $p(1-p)w^2$ . This is the variance of the conditional means of the risk classes. Note that $w$ is the additional mean loss for a “high risk” insured. The higher the additional mean loss $w$ , the more heterogeneous in risk between the two classes, hence the larger the dispersion in unconditional loss.

The total law of variance gives the unconditional variance of a random variable $X$ that is indexed by another random variable $\Theta$ . The unconditional variance of $X$ is the sum of two components, namely, the expected value of conditional variances and the variance of the conditional means. The formula is:

$\displaystyle Var(X)=E[Var(X \lvert \Theta)]+Var[E(X \lvert \Theta)]$

The following is the derivation of the formula:

$\displaystyle \begin{aligned}Var[X]&=E[X^2]-E[X]^2 \\&=E[E(X^2 \lvert \Theta)]-E[E(X \lvert \Theta)]^2 \\&=E\left\{Var(X \lvert \Theta)+E(X \lvert \Theta)^2 \right\}-E[E(X \lvert \Theta)]^2 \\&=E[Var(X \lvert \Theta)]+E[E(X \lvert \Theta)^2]-E[E(X \lvert \Theta)]^2 \\&=E[Var(X \lvert \Theta)]+Var[E(X \lvert \Theta)] \end{aligned}$

Additional Practice
See this blog post for practice problems on mixture distributions.

Reference

Klugman S.A., Panjer H. H., Wilmot G. E. Loss Models, From Data to Decisions, Second Edition., Wiley-Interscience, a John Wiley & Sons, Inc., New York, 2004

2 thoughts on “The variance of a mixture”

Kevin on November 18, 2011 at 9:27 am said:

I was struggling with this concept, but your example and explanation really helped me to finally understand. Thank you!

Reply ↓
Andrew on February 22, 2012 at 3:54 pm said:

Wonderful explanation, the best online tutorial on the law of total variance by far!

Reply ↓

Applied Probability and Statistics

Applications in Actuarial Science and Enterprise Risk Management

The variance of a mixture

2 thoughts on “The variance of a mixture”

Leave a Reply Cancel reply

Follow “Applied Probability and Statistics”

Share this:

Like this:

2 thoughts on “The variance of a mixture”

Leave a Reply Cancel reply

Follow “Applied Probability and Statistics”