Convergence in Distribution

Paul Schrimpf

2024-12-14

Central Limit Theorem

Levy’s Continuity Theorem

Lemma 2.1 (Levy’s Continuity Theorem)

$X_n \indist X$ iff $\Er[e^{i t'X_n} ] \to \Er[e^{i t' X} ]$ for all $t \in \R^d$

see Döbler (2022) for a short proof
$\Er[e^{i t' X}] \equiv \varphi(t)$ is the characteristic function of $X$

Law of Large Numbers Revisited

Lemma 2.2 (Weak Law of Large Numbers)

If $X_1, ..., X_n$ are i.i.d. with $\Er[|X_1|] < \infty$ , then $\frac{1}{n} \sum_{i=1}^n X_i \inprob \Er[X_1]$

Theorem 2.2 (non-iid WLLN)

If $\Er[X_i]=0$ , $\Er[X_i X_j] = 0$ for all $i \neq j$ and $\frac{1}{n} \max_{1 \leq j \leq n} \Er[X_j^2] \to 0$ , then $\frac{1}{n} \sum_{i=1}^n X_i \inprob 0$

Delta Method: Example

What is the asymptotic distribution of $\hat{\sigma} = \sqrt{\frac{1}{n} \sum_{i=1}^n \left(x_i - \frac{1}{n} \sum_{j=1}^n x_j \right)^2}?$

i. non i.d. Central Limit Theorem

Theorem 2.5 (Lindeberg’s Theorem)

Assume that for each $n$ , $X_{n,1}, ..., X_{n,k(n)}$ are independent with $\Er[X_{nj}] = 0$ , and $\frac{1}{k(n)} \sum_{j=1}^{k(n)} \Er[X_{nj}^2] = 1$ and for any $\epsilon>0$ , $\lim_{n \to \infty} \frac{1}{k(n)} \sum_{j=1}^{k(n)} \Er\left[ X_{nj}^2 1\{|X_{nj}|>\epsilon \sqrt{k(n)} \right] = 0$ Then, $\frac{1}{\sqrt{k(n)}} \sum_{j=1}^{k(n)} X_{n,j} \indist N(0,1)$

Weak Berry-Esseen Theorem

Weak Berry-Esseen Theorem

Let $X_i$ be i.i.d with $\Er[X]=0$ , $\Er[X^2]=1$ and $\Er[|X|^3]$ finite. Let $\varphi$ be smooth with its first three derivatives uniformly bounded, and let $G \sim N(0,1)$ . Then $\left\vert \Er\left[ \varphi\left( \frac{1}{\sqrt{n}} \sum_{i=1}^n X_i \right) \right] - \Er\left[\varphi(G)\right] \right\vert \leq C \frac{\Er[|X|^3]}{\sqrt{n}} \sup_{x \in \R} |\varphi'''(x)|$

This is based on Tao (2010) https://terrytao.wordpress.com/2010/01/05/254a-notes-2-the-central-limit-theorem/#berry

Proof

Let $G_i \sim N(0,1)$ . Define $Z_{n,i} = \frac{X_1 + \cdots X_i + G_{i+1} + \cdots G_n}{\sqrt{n}},$ so that $Z_{n,n} = _{i=1}^n X_i $ and $Z_{n,0} \equiv G$ .

With this notation we have $\Er\left[\varphi(Z_{n,n}) - \varphi(Z_{n,0}) \right] = - \sum_{i=0}^{n-i} \Er\left[ \varphi(Z_{n,i}) - \varphi(Z_{n,i+1})\right]$ and we will show that $\Er\left[ \varphi(Z_{n,i}) - \varphi(Z_{n,i+1})\right]$ is small.

Let $S_{n,i} = Z_{n,i} - \frac{G_{i+1}}{\sqrt{n}} = Z_{n,i+1} - \frac{X_{i+1}}/\sqrt{n}$ and take a second order Taylor expansion of $\varphi$ around $S_{n,i}$ , giving $\varphi(Z_{n,i}) = \varphi(S_{n,i}) + \varphi'(S_{n,i}) \frac{G_{i+1}}{\sqrt{n}} + \frac{1}{2}\varphi''(S_{n,i}) \frac{G_{i+1}^2}{n} + O\left( \frac{|G_{i+1}|^3}{n^{3/2}} \sup \varphi'''(x) \right)$ and $\varphi(Z_{n,i+i}) = \varphi(S_{n,i}) + \varphi'(S_{n,i}) \frac{X_{i+1}}{\sqrt{n}} + \frac{1}{2}\varphi''(S_{n,i}) \frac{X_{i+1}^2}{n} + O\left( \frac{|X_{i+1}|^3}{n^{3/2}} \sup \varphi'''(x) \right).$

We have assumed that $\Er[X_i] = \Er[G_i]$ and $\Er[X_i^2] = \Er[G_i^2]$ , so $\left\vert \Er\left[ \varphi(Z_{n,i}) - \varphi(Z_{n,i+1})\right] \right\vert \leq C( \frac{|G_{i+1}|^3 + |X_{i+1}|^3}{n^{3/2}} \sup \varphi'''(x) \right)$ for some constant $C$ that depends on $\varphi$ , but not on $n$ or $i$ . We can conclude that $\Er\left[\varphi(Z_{n,n}) - \varphi(Z_{n,0}) \right] \leq n C( \frac{|G_{i+1}|^3 + |X_{i+1}|^3}{n^{3/2}} \sup \varphi'''(x) \right).$

Berry-Esseen Theorem

Berry-Esseen Theorem

If $X_i$ are i.i.d. with $\Er[X] = 0$ and $\var(X)=1$ , then $\sup_{z \in \R} \left\vert P\left(\left[\frac{1}{\sqrt{n}} \sum_{i=1}^n X_i\right] \leq z \right) - \Phi(z) \right\vert \leq 0.5 \Er[|X|^3]/\sqrt{n}$ where $\Phi$ is the normal CDF.

Multivariate Berry-Esseen Theorem

If $X_i \in \R^d$ are i.i.d. with $\Er[X] = 0$ and $\var(X)=I_d$ , then $\sup_{A \subset \R^d, \text{convex}} \left\vert P\left(\frac{1}{\sqrt{n}} \sum_{i=1}^n X_i \in A \right) - P(N(0,I_d) \in A) \right\vert \leq (42 d^{1/4} + 16) \Er[\Vert X \Vert ^3]/\sqrt{n}$

Convergence in Distribution Paul Schrimpf 2024-12-14

Convergence in Distribution
Convergence in Distribution
Reading
Convergence in Distribution
Relation to Convergence in Probability
Slutsky’s Lemma
Central Limit Theorem
Levy’s Continuity Theorem
Law of Large Numbers Revisited
Central Limit Theorem
using PlotlyLight,...
CDF
Size Distortion
Histogram
Cramér-Wold Device
Multivariate Central Limit Theorem
Delta Method
Delta Method: Example
Continuous Mapping Theorem
Continuous Mapping Theorem: Example
i. non i.d. Central Limit Theorem
i. non i.d. Central Limit Theorem
Characterizing Convergence in Distribution
Characterizing Convergence in Distribution
Characterizing Convergence in Distribution
Non-asymptotic
Weak Berry-Esseen Theorem
Berry-Esseen Theorem
Simulated Illustration of Berry-Esseen CLT
Simulated Illustration of Berry-Esseen CLT : Slack Bounds
References