ThmDex – An index of mathematical definitions, results, and conjectures.
 ▼ Set of symbols ▼ Alphabet ▼ Deduction system ▼ Theory ▼ Zermelo-Fraenkel set theory ▼ Set ▼ Binary cartesian set product ▼ Binary relation ▼ Map ▼ Function ▼ Measure ▼ Real measure ▼ Euclidean real measure ▼ Complex measure ▼ Basic measure ▼ Unsigned basic measure ▼ Unsigned basic integral measure ▼ Radon-Nikodym derivative ▼ Kullback-Leibler divergence ▼ Mutual information ▼ Discrete random variable mutual information
Definition D5724
Simple random variable mutual information

Let $P = (\Omega, \mathcal{F}, \mathbb{P})$ be a D1159: Probability space such that
 (i) $X : \Omega \to \mathcal{X}$ and $Y : \Omega \to \mathcal{Y}$ are each a D5723: Simple random variable on $P$
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty)$.
The mutual information of $(X, Y)$ in base $a$ is the D4767: Unsigned real number $$I(X; Y) : = \sum_{x \in \mathcal{X}} \sum_{y \in \mathcal{Y}} \mathbb{P}(X = x, Y = y) \log_a \frac{\mathbb{P}(X = x, Y = y)}{\mathbb{P}(X = x) \mathbb{P}(Y = y)}$$

Let $X \in \text{Random}(\mathcal{X})$ and $Y \in \text{Random}(\mathcal{Y})$ each be a D5723: Simple random variable such that
 (i) $$\forall \, x \in \mathcal{X} : p(x ) : = \mathbb{P}(X = x)$$ (ii) $$\forall \, y \in \mathcal{Y} : p(y) : = \mathbb{P}(Y = y)$$ (iii) $$\forall \, x \in \mathcal{X}, y \in \mathcal{Y} : p(x, y) : = \mathbb{P}(X = x, Y = y)$$
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty)$.
The mutual information of $(X, Y)$ in base $a$ is the D4767: Unsigned real number $$I(X; Y) : = \sum_{x \in \mathcal{X}} \sum_{y \in \mathcal{Y}} p(x, y) \log_a \frac{p(x, y)}{p(x) p(y)}$$
Results
 ▶ Joint entropy formula for simple mutual information ▶ Mutual information of a simple random variable with respect to itself ▶ Symmetry of simple mutual information