Measure

Paul Schrimpf

2025-09-10

References

• Song (2021) chapter 1
• Watt (2022) chapter 8
• Spanos (2019) chapters 1 & 2
• Pollard (2002)
• Tao (2011)

Measures

Why Measure Theory?

1. Simplifies some arguments
   • Example from Pollard (2002), define independence as factorization of distribution functions \[ P(X \leq x \cap Y \leq y) = P(X \leq x) P(Y \leq y) \]
   • If \(X_1,X_2,X_3, X_4\) are independent, show that \[ Y = X_1 X_2 \log\left(\frac{X_1^2 + X_2^3}{|X_1| + |X_2|}\right) \] is independent of \[ Z = sin\left(X_3 + X_3^2 + X_3X_4 + X_4^2 \right) \]

\[ \def\R{\mathbb{R}} \def\B{\mathscr{B}} \]

Why Measure Theory?

1. Simplifies some arguments

2. Unifies treatment

   • discrete vs continuous
   • uni- vs multi-variate

3. Resolves some difficulties with infinity

One difficulty with infinity in probability is defining probability over uncountably infinite sets. In probability, we start with a sample space, say \(\Omega\), and then say a probability is a mapping from subsets of \(\Omega\) to \([0,1]\) satisfying some conditions. One question is can we define such a probability on all subsets of \(\Omega\). If \(\Omega\) is uncountable, then the answer no. We have to limit our attention to certain well behaved subsets. These turn out to be exactly a \(\sigma\)-field, which we now define.

Another important difficulty with infinity is that the limit of bounded Riemann integrable functions need not be Riemann integrable. This concerns us because limits of random variables, which are functions, are essential for asymptotic theory and practical statistics.

Measure Space

1. A set \(\Omega\)
2. A collection of subsets, \(\mathscr{F}\), of \(\Omega\) that is a \(\sigma\)-field (aka \(\sigma\)-algebra) , that is
   1. \(\Omega \in \mathscr{F}\)
   2. If \(A \in \mathscr{F}\), then \(A^c \in \mathscr{F}\)
   3. If \(A_1, A_2, ... \in \mathscr{F}\), then \(\cup_{j=1}^\infty A_j \in \mathscr{F}\)
3. A measure, \(\mu: \mathcal{F} \to [0, \infty]\) s.t.
   1. \(\mu(\emptyset) = 0\)
   2. If \(A_1, A_2, ... \in \mathscr{F}\) are pairwise disjoint, then \(\mu\left(\cup_{j=1}^\infty A_j \right) = \sum_{j=1}^\infty \mu(A_j)\)

Why do we need sigma fields? We need them because we cannot assign a sensible measure to all subsets. Construct the set \(V\) as follows. Let \(x \sim y\) iff \(x-y \in \mathbb{Q}\). Form \(V\) by choosing one element from each equivalence class in \([0,1)\). For \(x,y \in [0,1]\), let \(+_1\) be addition modulo \(1\), so \(x +_1 y = x+y\) if \(x+y \in [0,1]\) and \(x+y - 1\) otherwise. Translation invariance implies \(\mu(V +_1 x) = \mu(V)\), but \(\mu\left([0,1)\right) = \mu\left( \cup_{q \in \mathbb{Q} \cap [0,1)} V +_1 q \right) = \sum_{q \in \mathbb{Q} \cap [0,1)} \mu(V +_1 q)\) and then either \(\mu\left([0,1))\right) = 0\) or \(\infty\).

Examples of sigma fields.

Generated \(\sigma\)-field

Given \(\mathcal{C} \subseteq 2^\Omega\), the \(\sigma\)-field generated by \(\mathcal{C}\) is the intersection of all \(\sigma\)-fields containing \(\mathcal{C}\)

Exercise

If \(\Omega = \{a, b, c\}\) and \(\mathcal{C} = \{ \{a, b\} \}\), what is \(\sigma(\mathcal{C})\)?

Carathéodary’s Extension Theorem

Theorem

If \(\mathcal{C}\) is a ring of sets, i.e.

• \(\emptyset \in \mathcal{C}\)
• For all \(A,B \in \mathcal{C}\), \(A \cup B \in \mathcal{C}\) and \(A \setminus B \in \mathcal{C}\)

and \(\mu: \mathcal{C} \to \R\) is a pre-measure, i.e.

• \(\mu(\emptyset) = 0\)
• For any \(A = \cup_{i=1}^\infty A_i\) where \(A_i\) are disjoint, \(\mu(A) = \sum_{i=1}^\infty \mu(A_i)\)

then there exists a measure \(\mu': \sigma(\mathcal{C}) \to \R\).

This ensures existence of measures. It is not too difficult to specify a pre-measure on a ring of sets, and this theorem ensures that measures extends to a \(\sigma\)-field.

If \(\mu\) is \(\sigma\)-finite, then the extension is unique.

Finite Measure

• Measure \(\mu\) is finite if \(\mu(\Omega)\) is finite
• \(\mu\) is \(\sigma\)-finite if \(\exists\) \(\{A_n\}_{n=1}^\infty \in \mathscr{F}\) s.t. \(\mu(A_n)\) is finite \(\forall n\) and \(\cup_{n=1}^\infty A_n = \Omega\)

Exercise

Let \(\Omega\) be countable with any \(\mathscr{F}\), define \(\mu(A)\) as the number of elements of \(A\). Show \(\mu\) is \(\sigma\) finite.

Lebesgue Measure

Theorem

There exists a unique \(\sigma\)-finite measure \(\mu\) on \((\mathbf{R},\mathscr{B}(\mathbf{R}))\) such that for any \(a\leq b\) with \(a,b\in \mathbf{R}\), \[ \mu ((a,b])=b-a \]

Measurable Function

• Given a topology on \(\Omega\), the Borel \(\sigma\)-field, \(\mathscr{B}(\Omega)\), is the smallest \(\sigma\)-field containing all open subsets of \(\Omega\)

When working with \(R^d\), we will generally use \(\B(\R^d)\) as our \(\sigma\)-field.

• \(f: \Omega \to \R\) is (\(\mathscr{F}\)-)measurable if \(\forall\) \(B \in \B(\R)\), \(f^{-1}(B) \in \mathscr{F}\)

Measurability and Continuity

Lemma

If \(f: \R^n \to \R^k\) is continuous, then \(f\) is \(\B(\R^n)\) measurable.

To prove this, we show another useful claim. Let \((X,\mathcal{A})\) and \((Y, \sigma(\mathcal{C}))\) be measure spaces. Then, \(f:X \to Y\) is measurable if \(f^{-1}(C) \in \mathcal{A}\) for all \(C \in \mathcal{C}\). The lemma on the slide immediately follows since \(\B(\R^k)\) is generated by open sets and continuous pre-images of open-sets are open.

To prove the claim, note that \(\mathcal{D} = \{ B \in Y : f^{-1}(B) \in \mathcal{A}\}\) is a \(\sigma\)-field on \(Y\). Also, if \(f^{-1}(C) \in \mathcal{A}\) for all \(C \in \mathcal{C}\), then \(\mathcal{C} \subseteq \mathcal{D}\). Thus, \(\sigma(\mathcal{C}) \subseteq \mathcal{D}\), so \(f\) is measurable.

Corollary

If \(f: \Omega \to \R\) is measurable and \(g: \R \to \R\) is continuous, then \(g \circ f\) is measurable.

Corollary

If \(f: \Omega \to \R\) and \(g: \Omega \to \R\) are measurable, then \(f+g\) is measurable.

Integration

Simple Functions

• Assume \(\mu(\Omega) < \infty\)
• \(f\) is a simple function if \(f = \sum_{j=1}^n a_j 1\{\omega \in E_j \}\) for \(a_j \in \R\) and \(E_j \in \mathscr{F}\)
• Integral of simple functions: \[ \int f d \mu = \sum_{j=1}^n a_j \mu(E_j) \]

Exercise

Show that if \(f\) is simple, then \(f\) is measurable.

Exercise

Show that if \(f\) and \(g\) are simple, then

1. If \(f \geq 0\) a.e., then \(\int f d\mu \geq 0\)

2. \(\forall a \in \R\) , \(\int a f d\mu = a \int f d\mu\)

3. \(\int (f + g) d\mu = \int f d\mu + \int g d \mu\)

Bounded Functions

• Let \(E\) be such that \(\mu(E)<\infty\)
• Let \(f\) be bounded function and \(f(x) = 0 \forall x \in E^c\)
• Define: \[ \int f d\mu \equiv \sup_{\varphi \leq f: \varphi \text{ simple}} \int \varphi d\mu =\inf_{\varphi \geq f: \varphi \text{ simple}} \int \varphi d\mu \]

Exercise

Show the second equality above.

Exercise

Show that for all bounded \(f\) and \(g\) that vanish outside a finite measure set,

1. If \(f \geq 0\) a.e., then \(\int f d\mu \geq 0\)

2. \(\forall a \in \R\) , \(\int a f d\mu = a \int f d\mu\)

3. \(\int (f + g) d\mu = \int f d\mu + \int g d \mu\)

Nonnegative Functions

• If \(f \geq 0\), define \[ \int fd\mu =\sup_{f_n \leq f \text{ simple, bounded+}} \int f_{n}d\mu \]

Exercise

Show that for all \(f \geq 0\),

1. If \(f \geq 0\) a.e., then \(\int f d\mu \geq 0\)

2. \(\forall a \in \R\) , \(\int a f d\mu = a \int f d\mu\)

3. \(\int (f + g) d\mu = \int f d\mu + \int g d \mu\)

Measurable Functions

• If \(f\) is measurable, let \(f^{+} = \max\{f, 0\}\) and \(f^{-} = \max\{-f, 0\}\) and define the Lebesgue integral

\[ \int f d\mu = \int f^{+} d\mu - \int f^{-} d\mu \]

Exercise

Show that for all measurable \(f\)

1. If \(f \geq 0\) a.e., then \(\int f d\mu \geq 0\)

2. \(\forall a \in \R\) , \(\int a f d\mu = a \int f d\mu\)

3. \(\int (f + g) d\mu = \int f d\mu + \int g d \mu\)

Convergence Theorems

Sequences of Sets

• \(\{E_n\}_{n \geq 1} \in \mathscr{F}\)
  • increasing if \(E_1 \subset E_2 \subset ...\)
  • decreasing if \(E_1 \supset E_2 \supset ...\)
  • monotone if either increasing or decreasing
• For increasing \(E_n\), define \(\lim_{n \to \infty} E_n =\cup_{n=1}^\infty E_n\)
• For decreasing \(E_n\), define \(\lim_{n \to \infty} E_n =\cap_{n=1}^\infty E_n\)

Continuity of Measure

Lemma

Suppose that \(\{E_{n}\}\) is a monotone sequence of sets. Then \[ \mu \left( \lim_{n\rightarrow \infty}E_{n}\right) =\lim_{n\rightarrow \infty }\mu (E_{n}). \]

To see what this has to do with “continuity”, remember that one way to define continuity of functions is that for all \(x_n \to x\), \(f(x_n) \to x\).

Monotone Convergence Theorem

Lemma

If \(f_n:\Omega \to \mathbf{R}\) are measurable, \(f_{n}\geq 0\), and for each \(\omega \in \Omega\), \(f_{n}(\omega )\uparrow f(\omega )\), then \(\int f_{n}d\mu \uparrow \int fd\mu\) as \(n\rightarrow \infty\)

Exercise

Suppose \(g_n: \Omega \to \R\) are measurable and \(g_n \geq 0\). Show that \[ \int \sum_{n=1}^\infty g_n d\mu = \sum_{n=1}^\infty \int g_n d\mu \]

How do we know \(f = \lim_{n\to \infty} f_n\) is measurable?

Note that half open intervals \((-\infty, c)\) generate \(\B(\R)\), so it suffices to show that \(f^{-1}((-\infty, c))\) is in \(\mathscr{F}\). Since \(f_n\) are measurable, \(f_n^{-1}((-\infty,c)) \in \mathscr{F}\) for all \(n\). Also, \[ f_n^{-1}((-\infty,c)) = \{x: f_n(x) < c\} \] and since \(f_n\) are increasing, \(f_n^{-1}((-\infty,c)) \supset f_m^{-1}((-\infty, c))\) for an \(m \geq n\). Moreover, \[ f^{-1}((-\infty,c)) = \{x : f(x) < c \} = \cup_{n=1}^\infty \{x: f_n(x) < c\} \] and hence is in \(\mathscr{F}\).

Fatou’s Lemma

Lemma

If \(f_n:\Omega \to \R\) are measurable, \(f_{n}\geq 0\), then \[ \int \left( \text{liminf}_{n\rightarrow \infty }f_{n}d\mu \right) \leq \text{liminf}_{n\rightarrow \infty }\int f_{n}d\mu \]

\(\text{liminf}_{n\rightarrow \infty} x_n = \lim_{n \to \infty} \left(\inf m \geq n x_m \right)\)

Dominated Convergence Theorem

Lemma

If \(f_n:\Omega \to \R\) are measurable, and for each \(\omega \in \Omega\), \(f_{n}(\omega )\rightarrow f(\omega ).\) Furthermore, for some \(g\geq 0\) such that \(\int gd\mu <\infty\), \(|f_{n}|\leq g\) for each \(n\geq 1\). Then, \(\int f_{n}d\mu \rightarrow \int fd\mu\)

Exercise

Show that if \(g_n: \Omega \to \R\) are measurable and \(\sum_{n=1}^\infty \int |g_n| d\mu < \infty\), then \(\sum_{n=1}^\infty |g_n(x)| < \infty\) almost everywhere and if \(g(x) = \sum_{n=1}^\infty g_n(x)\), then \[ \lim_{N \to \infty} \int \left\vert g(x) - \sum_{n=1}^N g_n(x) \right\vert d\mu(x) = 0 \]

Radon-Nikodym Derivatives

Recall that if \(f(x) = \frac{dF}{dx}(x)\), then \(F(x) - F(0) = \int_0^x f(x) dx\). The idea of this section is to generalize this relationship to measures.

Suppose you have a measure \(\mu\). You can build another measure be taking a measurable, nonnegative function and defining \(\nu(A) = \int_A f(x) d\mu(x)\)

Now, let’s try to reverse this exercise. Given measures \(\mu\) and \(\nu\), can we find \(f\) such that \(\nu = f \mu\)? One needed condition is if \(\mu(A) = 0\), then we must have \(\nu(A) = \int_A f d\mu = 0\).

Absolute Continuity

• Measure \(\nu\) is absolutely continuous with respect to \(\mu\) if for \(A \in \mathscr{F}\), \(\mu(A) = 0\) implies \(\nu(A) = 0\)
  • denotate as \(\nu \ll \mu\)
  • \(\mu\) is called a dominating measure

Radon-Nikodym Derivative

Theorem

Let \((\Omega,\mathscr{F},\mu)\) be a measure space, and let \(\nu\) and \(\mu\) be \(\sigma\)-finite measures defined on \(\mathscr{F}\) and \(\nu \ll \mu\). Then there is a nonnegative measurable function \(f\) such that for each set \(A\in \mathscr{F}\), \[ \nu (A)=\int_{A}fd\mu \] For any such \(f\) and \(g\), \(\mu (\{\omega \in \Omega:f(\omega )\neq g(\omega )\})=0\)

Denote \(f = \frac{d\nu}{d\mu}\)

Exercise: show coincides with usual derivative?

References

Pollard, David. 2002. A User’s Guide to Measure Theoretic Probability. 8. Cambridge University Press.

Song, Kyunchul. 2021. “Introduction to Econometrics.”

Spanos, Aris. 2019. Probability Theory and Statistical Inference: Empirical Modeling with Observational Data. Cambridge University Press. https://ubc.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fb9MwELZY-0Jf2PghyjYUQHvgoVtju3aMxhCMVYCExANCEy-W7dhTtZGgtEPsv-fOTqq2CE1IVaUkl8S9XO8-5-4-E8Lo4Xi04RPMhFmTO1vKwLgqXZAA-51g3FhPmdgs1Sm61pi1dH4e-WBzXGqpwJbLN-juF4vXuGbMK5hkG3ezRfqyEHzSI_23n87Pvy9ft0AUKyAYxbWCIKoCzuAKN-BiggkmaUv_1B3strvjvOXn_GsQAzIw80twSOCsFvNVhgZsXYI57Qx2blWz2bqzp20Em94jv7sfe1Ef4pJZiPzA4PDNRONTYKubo9RmuGSJxHo5sGT4tCpKM_CWNWlthAdsmhR1wN6jquDMpKxt0vfYUbFD7vjqPhl8XhLFzh8QdfzDNJcnXxpwK7FM9yZLbAGZqcoMkXAkkjZX2ceuM_H4KJ7ykHybnn09_TBqV3MYGcBkVIy8tWESCiqFoZgcZNQoNrbIMAe4KJSA9SQzygrKnVKc5la4EuCOY4FLEzh7RHpVXfnHJFMyGA-eCSzQcxO8KgB9F85MbBF86eyQPF95KPrXVcw8z_WaXm4RUhzAK7tNCPnrQOjF8qlrZ2udSufk5v1erkjVTXulf8gOk-3on4l0RGO6LEf6xvGQ7IM5aTfD7xwTzwDyBAB1qTApSofkWWdoOt6jLfPVZ-9OY688U_zJf4xll9wFlBgL63K1R3qL5trvR5KKp-2_7A8NQRm-.

Tao, Terence. 2011. An Introduction to Measure Theory. Vol. 126. American Mathematical Society Providence.

Watt, Mitchell. 2022. “Math Camp for Incoming Doctoral Students 2022.” https://www.mitchellwatt.com/files/mathcampnotes22.pdf.