ThmDex – An index of mathematical definitions, results, and conjectures.

Result R3547 on D2013: Random real number central moment

Expectation minimises second central moment for random real number

Formulation 0

Let

$X \in \text{Random}(\mathbb{R})$ be a D3161: Random real number such that

(i)	$\begin{equation} \mathbb{E} \|X\|^2 < \infty \end{equation}$

Let

$\lambda \in \mathbb{R}$ be a D993: Real number.

Then

$\begin{equation} \mathbb{E} |X - \mathbb{E} X|^2 \leq \mathbb{E} |X - \lambda|^2 \end{equation}$

Formulation 1

Let

$X \in \text{Random}(\mathbb{R})$ be a D3161: Random real number such that

(i)	$\begin{equation} \mathbb{E} \|X\|^2 < \infty \end{equation}$

Let

$\lambda \in \mathbb{R}$ be a D993: Real number.

Then

$\begin{equation} \text{Var} X \leq \mathbb{E} |X - \lambda|^2 \end{equation}$

Formulation 2

Let

$X \in \text{Random}(\mathbb{R})$ be a D3161: Random real number such that

(i)	$\begin{equation} \mathbb{E} \|X\|^2 < \infty \end{equation}$

Then

$\begin{equation} \mathbb{E} X = \underset{\lambda \in \mathbb{R}}{\text{arg min}} \; \mathbb{E} |X - \lambda|^2 \end{equation}$

Also known as

Expectation minimises mean square error, Expectation minimises unconditional L2-distance

Proofs

Proof 0

Let

$X \in \text{Random}(\mathbb{R})$ be a D3161: Random real number such that

(i)	$\begin{equation} \mathbb{E} \|X\|^2 < \infty \end{equation}$

Let

$\lambda \in \mathbb{R}$ be a D993: Real number.

Using R3824: Bias-variance partition of mean square error, we have

$\begin{equation} \mathbb{E} |X - \lambda|^2 = \text{Var} X + (\mathbb{E} X - \lambda)^2 \end{equation}$ The term

$\text{Var} X$ is a constant, so to minimize the right-hand side it is sufficient to minimize the expression

$(\mathbb{E} X - \lambda)^2$ . The function

$\begin{equation} \lambda \to (\mathbb{E} X - \lambda)^2 \end{equation}$ is nonnegative and attains (the minimum) value

$0$ at

$\lambda = \mathbb{E} X$ . Hence

$\begin{equation} \begin{split} \mathbb{E} |X - \lambda|^2 & = \text{Var} X + (\mathbb{E} X - \lambda)^2 \\ & \geq \text{Var} X \\ & = \mathbb{E} |X - \mathbb{E} X|^2 \\ \end{split} \end{equation}$

$\square$