Midterm Solutions 2023

Problem 1

$$$$

Suppose $u_{i,k}\in \{0,1\}$ for $k=1,...,K$, and $Y_i=\sum _{k=1}^K{u}_{i,k}$. Observations for different $i$ are independent and identically distributed. $Y_1,...,Y_n$ are observed, but $u_{i,k}$ are not. For the same $i$ and $j\ne k$, $u_{ij}$ and $u_{ik}$ are independent (but may not be identically distributed).

$\sigma$ fields

For $K=2$, what is $\sigma (Y)$?

Solution. The range of $Y$ is $\{0,1,2\}$. $\sigma (Y)$ will consist of the preimages of each of these values and all their intersections and unions. That is, $$\sigma (Y)=\left\{\begin{array}{c}\mathrm{\varnothing },\{(0,0)\},\{(1,1)\},\{(0,1),(1,0)\},\\ \{(0,0),(0,1),(1,0)\},\{(0,1),(1,0),(1,1)\}\\ \{(0,0),(1,1)\},\{(0,0),(0,1),(1,0),(1,1)\}\end{array}\right\}$$

Identification

1. Let ${\theta }_k=P(u_{i,k}=1)$ show that $\theta =({\theta }_1,...,{\theta }_K)$ is not identified by finding an observationally equivalent $\tilde{\theta }$.

Solution. Since addition is commutative, $\theta$ is observationally equivalent to any permutation of its values. For example, with $K=2$, $\theta =({\theta }_1,{\theta }_2)$ is observationally equivalent to $\tilde{\theta }=({\theta }_2,{\theta }_1)$ because $$\begin{array}{rl}{P}_{\theta }(Y=0)=& (1-{\theta }_1)(1-{\theta }_2)=P_{\tilde{\theta }}(Y=0)\\ P_{\theta }(Y=1)=& thet{a}_1(1-{\theta }_2)+{\theta }_2(1-{\theta }_1)=P_{\tilde{\theta }}(Y=1)\\ P_{\theta }(Y=2)=& {\theta }_1{\theta }_2=P_{\tilde{\theta }}(Y=2).\end{array}$$

2. Show that if ${\theta }_1={\theta }_2=\cdots ={\theta }_K=\vartheta$, then $\vartheta$ is identified.

Solution. In this case, the expectation of $Y$ is $\mathrm{E}[Y]=K\vartheta$, so $\vartheta$ is identified by $\mathrm{E}[Y]/K$.

Estimation

1. Assuming ${\theta }_1={\theta }_2=\cdots ={\theta }_K=\vartheta$, find the maximum likelihood estimator¹ for $\vartheta$, and show whether it is unbiased.

Solution. The density of $Y$ with respect to a uniform discrete measure on $\{0,...,K\}$ is $$f(Y_i;\theta )=(\genfrac{}{}{0ex}{}{K}{Y_i}){\theta }^{Y_i}(1-\theta {)}^{K-Y_i}$$ The loglikelihood is then $$\log \ell (\theta ;Y)=\sum _{i=1}^n(\log (\genfrac{}{}{0ex}{}{K}{Y_i})+Y_i\log \theta +(K-Y_i)\log (1-\theta ))$$ Solving the first order condition for $\hat{\theta }$ gives: $$\begin{array}{rl}0=& \sum \frac{Y_i}{\hat{\theta }}-\frac{K-Y_i}{1-\hat{\theta }}\\ \hat{\theta }=\frac{1}{nK}\sum _{i=1}^n{Y}_i\end{array}$$

This estimator is unbiased. $$\mathrm{E}[\hat{\theta }]=\frac{1}{nK}\sum _{i=1}^n\mathrm{E}[Y_i]=\vartheta .$$

2. Show whether or not the maximum likelihood estimator is consistent.

Solution. Note that $\mathrm{E}[|Y_i|]=\mathrm{E}[Y_i]=K\vartheta$ is finite, and $Y_i$ is iid. Therefore, by the law of large numbers, $$\frac{1}{n}\sum _{i=1}^n{Y}_i\stackrel{p}{\to }\mathrm{E}[Y_i]=K\vartheta$$ so $$\frac{1}{nK}\sum _{i=1}^n{Y}_i\stackrel{p}{\to }\vartheta .$$

Testing

For this part, still assume ${\theta }_1={\theta }_2=\cdots ={\theta }_K=\vartheta$.

1. For find the most powerful test for testing $H_0:\vartheta ={\vartheta }_0$ against $H_1:\vartheta ={\vartheta }_a$ where ${\vartheta }_a>{\vartheta }_0$.

Solution. By the Neyman-Pearson Lemma, the likelihood ratio is most powerful. Let’s describe the critical region for this test. The likelihood ratio is $$\begin{array}{rl}lr({\vartheta }_a,{\vartheta }_0;Y)=& \frac{\prod _i{\vartheta }_a^{Y_i}(1-{\vartheta }_a{)}^{K-Y_i}}{\prod _i{\vartheta }_0^{Y_i}(1-{\vartheta }_0{)}^{K-Y_i}}\\ =& \frac{{\vartheta }_a^{\sum Y_i}(1-{\vartheta }_a{)}^{nK-\sum Y_i}}{{\vartheta }_0^{\sum Y_i}(1-{\vartheta }_0{)}^{nK-\sum Y_i}}\\ =& {\left(\frac{{\vartheta }_a}{{\vartheta }_0}\right)}^{\sum Y_i}{\left(\frac{1-{\vartheta }_a}{1-{\vartheta }_0}\right)}^{nK-\sum Y_i}\end{array}$$ When ${\vartheta }_a>{\vartheta }_0$, so that $\frac{{\vartheta }_a}{{\vartheta }_0}>0$ and $\frac{1-{\vartheta }_a}{1-{\vartheta }_0}<1$, the likelihood ratio is an increasing function of $\sum Y_i$ and does not depend on the data in any other way. The critical region for a test of size $\alpha$ will be $$C=\{Y_i:\sum Y_i>c^{\ast }(\alpha ,{\theta }_0)\}$$ where $P_{{\theta }_0}(\sum Y_i>c^{\ast }({\theta }_0))=\alpha$.

2. Is this test also most powerful against the alternative $H_1:\vartheta >varthet{a}_0$? (Hint: does the critical region depend on ${\vartheta }_a$?

Solution. In the previous part, we saw that the critical region is the same for any ${\vartheta }_a>{\vartheta }_0$. Hence, the test is uniformly most powerful again $H_1:\vartheta >{\vartheta }_0$.

Problem 2

Suppose you have a linear model with grouped data $$y_{ij}=x_{ij}^{\prime }\beta +u_{ij}$$ where there are $j=1,...,J$ groups with $i=1,...,n_j$ individuals each, and $x_{ij}\in {\mathbb{R}}^k$. For example, $j$ could be firms, and $i$ could index workers in the firms. Throughout, assume that observations are independent across $j$, $\mathrm{E}[u_{ij}]=0$, and $\mathrm{E}[x_{ij}{u}_{ij}]=0$. $y$ and $x$ are observed, but $u$ is not. Assume that $\mathrm{E}[x_{ij}{x}_{ij}^{\prime }]$ has rank $K$.

Identification

1. Show that $\beta$ is identified by explicitly writing $\beta$ as a function of the distribution of $x_{ij}$ and $y_{ij}$.

Solution. We can identify $\beta$ by regression, $\beta =\mathrm{E}[x_{ij}{x}_{ij}^{\prime }{]}^{-1}\mathrm{E}[x_{i}j{y}_{i}j]$.

2. Suppose that instead of observing $(y_{ij},x_{ij})$ for each $i$ and $j$, you only observe group averages, ${\overline{y}}_j=\frac{1}{n_j}\sum _{i=1}^{n_j}{y}_{ij}$ and ${\overline{x}}_j=\frac{1}{n_j}\sum _{i=1}^{n_j}{x}_{ij}$. Can $\beta$ still be identified?

Solution. Yes, $\beta$ can still be identified by $$\beta =\mathrm{E}[{\overline{x}}_j{\overline{x}}_j^{\prime }{]}^{-1}\mathrm{E}[{\overline{x}}_j{\overline{y}}_j]$$ as long as $\mathrm{E}[{\overline{x}}_j{\overline{x}}_j^{\prime }]$ is nonsingular.

Estimation

Continue to assume that you only observe group averages. Construct a sample analogue estimator for $\beta$ based on your answer to 2.b.2.²

1. Show whether your estimator is unbiased.

Solution. The estimator is $$\hat{\beta }={\left(\sum _j{\overline{x}}_j{\overline{x}}_j^{\prime }\right)}^{-1}\sum _j{\overline{x}}_j{\overline{y}}_j$$ Substituting the model in for ${\overline{y}}_j$, we have $$\hat{\beta }={\left(\sum _j{\overline{x}}_j{\overline{x}}_j^{\prime }\right)}^{-1}\sum _j{\overline{x}}_j({\overline{x}}_j^{\prime }\beta +{\overline{u}}_j)$$ so $$\mathrm{E}[\hat{\beta }]=\beta +\mathrm{E}[{\left(\sum _j{\overline{x}}_j{\overline{x}}_j^{\prime }\right)}^{-1}\sum _j{\overline{x}}_j({\overline{x}}_j^{\prime }{\overline{u}}_j)].$$ We are assuming that $\mathrm{E}[x_{ij}{u}_{ij}]=0$, so $\mathrm{E}[{\overline{x}}_j\overline{u_j}]=0$ as well. However, this does not imply that $$\mathrm{E}[{\left(\sum _j{\overline{x}}_j{\overline{x}}_j^{\prime }\right)}^{-1}\sum _j{\overline{x}}_j({\overline{x}}_j^{\prime }{\overline{u}}_j)]=0,$$ so $\hat{\beta }$ is biased.

For $\hat{\beta }$ to be unbiased, we would need the stronger assumption that $\mathrm{E}[{\overline{u}}_j|{\overline{x}}_j]=0$.

2. Assume that $\mathrm{E}[\Vert {\overline{x}}_j{\overline{x}}_j^{\prime }{\Vert }_2^2]\le M$ and $\mathrm{E}[\Vert {\overline{x}}_j{u}_j{\Vert }_2^2]\le M$ for all $j$. Show whether or not your estimator is consistent as $J\to \mathrm{\infty }$.

Solution. In this question and in the distribution question, there is some complication because $\mathrm{E}[{\overline{x}}_j{\overline{x}}_j^{\prime }]$ potentially varies with $j$. A really great answer recognizes this and addresses it in some way. An answer that correctly uses the law of large number and Slutsky’s lemma is also okay.

These assumptions imply that the law of large numbers applies to $\frac{1}{J}\sum {\overline{x}}_j{\overline{x}}_j^{\prime }$ and $\frac{1}{J}\sum {\overline{x}}_j{\overline{u}}_j$. We can show this using Markov’s inequality, but doing so is not necessary for full credit. Using Markov’s inequality we have $$\begin{array}{rl}P(\Vert \frac{1}{J}\sum ({\overline{x}}_j{\overline{x}}_j^{\prime }-\mathrm{E}[{\overline{x}}_j{\overline{x}}_j^{\prime }])\Vert >ϵ)& \le \frac{\mathrm{E}[\Vert \frac{1}{J}\sum ({\overline{x}}_j{\overline{x}}_j^{\prime }-\mathrm{E}[{\overline{x}}_j{\overline{x}}_j]){\Vert }^2]}{{ϵ}^2}\\ & \le \frac{\frac{1}{J}\sum \mathrm{E}[\Vert {\overline{x}}_j{\overline{x}}_j^{\prime }{\Vert }^2]}{{ϵ}^2}\\ & \le \frac{M}{J{ϵ}^2}\end{array}$$ so $\frac{1}{J}\sum {\overline{x}}_j{\overline{x}}_j^{\prime }\stackrel{p}{\to }\frac{1}{J}\sum \mathrm{E}[{\overline{x}}_j{\overline{x}}_j^{\prime }]$. An identical argument shows that $\frac{1}{J}\sum {\overline{x}}_j{\overline{u}}_j\stackrel{p}{\to }0$.

Therefore, $$\begin{array}{rl}\hat{\beta }=& \beta +{\left(\frac{1}{J}\sum _j{\overline{x}}_j{\overline{x}}_j^{\prime }\right)}^{-1}\frac{1}{J}\sum _j{\overline{x}}_j{\overline{u}}_j\\ \stackrel{p}{\to }& \beta \end{array}$$ provided that $\frac{1}{J}\sum \mathrm{E}[{\overline{x}}_j{\overline{x}}_j^{\prime }]$ is invertible for all $J$ large enough. It would be sufficient to assume $\underset{J\to \mathrm{\infty }}{lim}\frac{1}{J}\sum \mathrm{E}[{\overline{x}}_j{\overline{x}}_j^{\prime }]=C$ exists and $C$ is invertible, but weaker conditions are possible as well.

Efficiency

Assume that $\mathrm{E}[u_{ij}{u}_{\ell k}]=\{\begin{array}{ll}{\sigma }^2& \text{ if }i=\ell \text{ and }j=k\\ 0& \text{ otherwise}\end{array}$.

1. Suppose you observe $y_{ij}$ and $x_{ij}$ for each $i$ and $j$. What is the minimal variance unbiased estimator for $c^{\prime }\beta$ that is a linear function of $y$?

Solution. In this case, all the assumptions of the Gauss Markov thereom are met, so ordinary least squares in the best linear unbiased estimator.

2. Suppose you only observe group averages ${\overline{y}}_j$ and ${\overline{x}}_j$. What is the minimal variance unbiased estimator for $c^{\prime }\beta$ that is a linear function of $\overline{y}$?

Solution. Now, the Gauss Markov theorem does not directly apply because $var({\overline{u}}_j)=\frac{{\sigma }^n}{n_j}$ is not the same for all $j$. However, we can transform the model to make the variance constant, $$\sqrt{n_j}{\overline{y}}_j=\sqrt{n_j}{\overline{x}}_j\beta +\underset{{\tilde{u}}_j}{\underbrace{\sqrt{n_j}{\overline{u}}_j}}$$ Now $\mathrm{E}[\tilde{u}{\tilde{u}}^{\prime }]={\sigma }^2{I}_J$, so the Gauss Markov theorem applies and the best linear unbiased estimator is $${\hat{\beta }}^{WLS}={\left(\sum _j{n}_j{\overline{x}}_j{\overline{x}}_j^{\prime }\right)}^{-1}\sum _j{n}_j{\overline{x}}_j{\overline{y}}_j$$

Distribution

Continue to assume that you only observe group averages. Let ${\hat{\beta }}^{WLS}={\left(\sum _{j=1}^J{n}_j{\overline{x}}_j{\overline{x}}_j^{\prime }\right)}^{-1}\left(\sum _{j=1}^J{n}_j{\overline{x}}_j{\overline{y}}_j\right)$. Show that $$\sqrt{J}({\hat{\beta }}^{WLS}-\beta )=\sqrt{J}{\left(\sum _{j=1}^J{n}_j{\overline{x}}_j{\overline{x}}_j^{\prime }\right)}^{-1}\left(\sum _{j=1}^J{n}_j{\overline{x}}_j{\overline{u}}_j\right)$$ converges in distributions and compute the limiting distribution. State any additional assumptions that you need to show convergence.

Solution. As above, assuming $\mathrm{E}[\Vert n_j{\overline{x}}_j{\overline{x}}_j^{\prime }{\Vert }_2^2]$ is finite is sufficient for a law of large numbers to apply to $\frac{1}{J}\sum _j{n}_j{\overline{x}}_j{\overline{x}}_j^{\prime }$.

We already have assumptions that imply $\mathrm{E}[n_j{\overline{x}}_j{\overline{u}}_j]=0$. If we also assume that this has a variance, then a central limit theorem would apply. It would be an acceptable (and the expected) answer to just assume $\mathrm{E}[n_j^2{\overline{x}}_j{\overline{x}}_j^{\prime }{\overline{u}}_j^2]=\mathrm{\Omega }$ for all $j$ and proceed.

Alternatively, we could assume $\mathrm{E}[u_{ij}^2|x_{ij}]={\sigma }^2$ for all $i$ and $j$ and $u_{ij}$ are independent across $i$ and $j$. Then, $$\begin{array}{rl}\mathrm{E}[n_j{\overline{x}}_j{\overline{x}}_j^{\prime }{\overline{u}}_j^2]=& \mathrm{E}[n_j{\overline{x}}_j{\overline{x}}_j^{\prime }\mathrm{E}[n_j{\overline{u}}_j^2|{\overline{x}}_j]]\\ =& \mathrm{E}[n_j{\overline{x}}_j{\overline{x}}_j^{\prime }]{\sigma }^2\end{array}$$ If we also assume $\mathrm{E}[x_{ij}{x}_{ij}^{\prime }]=C$ is the same for all $i$ and $j$ and observations are independent, then $\mathrm{E}[n_j{\overline{x}}_j{\overline{x}}_j^{\prime }]=C$ for all $j$.

In that case, by the central limit theorem, $$\frac{1}{\sqrt{J}}\sum _{j=1}^J{\overline{x}}_j{\overline{u}}_j\stackrel{d}{\to }N(0,\mathrm{E}[x_{ij}{x}_{ij}^{\prime }]{\sigma }^2)$$ and therefore, $$\sqrt{J}(\hat{\beta }-\beta )\stackrel{d}{\to }N(0,\mathrm{E}[x_{ij}{x}_{ij}^{\prime }{]}^{-1}{\sigma }^2).$$

Definitions and Results

• Measure and Probability:

  • A collection of subsets, $\mathcal{F}$, of $\mathrm{\Omega }$ is a $\sigma$-field , if

    1. $\mathrm{\Omega }\in \mathcal{F}$
    2. If $A\in \mathcal{F}$, then $A^c\in \mathcal{F}$
    3. If $A_1,A_2,...\in \mathcal{F}$, then ${\cup }_{j=1}^{\mathrm{\infty }}{A}_j\in \mathcal{F}$

  • A measure, $\mu :\mathcal{F}\to [0,\mathrm{\infty }]$ s.t.

    1. $\mu (\mathrm{\varnothing })=0$
    2. If $A_1,A_2,...\in \mathcal{F}$ are pairwise disjoint, then $\mu \left({\cup }_{j=1}^{\mathrm{\infty }}{A}_j\right)=\sum _{j=1}^{\mathrm{\infty }}\mu (A_j)$

  • Lesbegue integral is

    1. Positive: if $f\ge 0$ a.e., then $\int fd\mu \ge 0$
    2. Linear: $\int (af+bg)d\mu =a\int fd\mu +b\int gd\mu$

  • Radon-Nikodym derivative: if $\nu \ll \mu$, then $\mathrm{\exists }$ nonnegative measureable function, $\frac{d\nu }{d\mu }$, s.t. $$\nu (A)={\int }_A\frac{d\nu }{d\mu }d\mu$$

  • Monotone convergence: If $f_n:\mathrm{\Omega }\to \mathbf{R}$ are measurable, $f_n\ge 0$, and for each $\omega \in \mathrm{\Omega }$, $f_n(\omega )\uparrow f(\omega )$, then $\int f_{n}d\mu \uparrow \int fd\mu$ as $n\to \mathrm{\infty }$

  • Dominated converegence: If $f_n:\mathrm{\Omega }\to \mathbf{R}$ are measurable, and for each $\omega \in \mathrm{\Omega }$, $f_n(\omega )\to f(\omega ).$ Furthermore, for some $g\ge 0$ such that $\int gd\mu <\mathrm{\infty }$, $|f_n|\le g$ for each $n\ge 1$. Then, $\int f_{n}d\mu \to \int fd\mu$

  • Markov’s inequality: $P(|X|>ϵ)\le \frac{\mathrm{E}[|X{|}^k]}{{ϵ}^k}$ $\mathrm{\forall }ϵ>0,k>0$

  • Jensen’s inequality: if $g$ is convex, then $g(\mathrm{E}[X])\le \mathrm{E}[g(X)]$

  • Cauchy-Schwarz inequality: ${(\mathrm{E}[XY])}^2\le \mathrm{E}[X^2]\mathrm{E}[Y^2]$

  • $\sigma (X)$ is $\sigma$-field generated by $X$, it is

    • smallest $\sigma$-field w.r.t. which $X$ is measurable
    • $\sigma (X)=\{X^{-1}(B):B\in \mathcal{B}(\mathbb{R})\}$

  • $\sigma (W)\subset \sigma (X)$ iff $\mathrm{\exists }$ $g$ s.t. $W=g(X)$

  • Events $A_1,...,A_m$ are independent if for any sub-collection $A_{i_1},...,A_{i_s}$ $$P\left({\cap }_{j=1}^s{A}_{i_j}\right)=\prod _{j=1}^{s}P(A_{i_j})$$ $\sigma$-fields are independent if this is true for any events from them. Random variables are independent if their $\sigma$-fields are.

  • Conditional expection of $Y$ given $\sigma$-field $\mathcal{G}$ satisfies ${\int }_A\mathrm{E}[Y|\mathcal{G}]dP={\int }_{A}YdP$ $\mathrm{\forall }A\in \mathcal{G}$

• Identification $X$ observed, distribution $P_X$, probability model $\mathcal{P}$

  • ${\theta }_0\in {\mathbb{R}}^k$ is identified in $\mathcal{P}$ if there exists a known $\psi :\mathcal{P}\to {\mathbb{R}}^k$ s.t. ${\theta }_0=\psi (P_X)$
  • $\mathcal{P}=\{P(\cdot ;s):s\in S\}$, two structures $s$ and $\tilde{s}$ in $S$ are observationally equivalent if they imply the same distribution for the observed data, i.e. $$P(B;s)=P(B;\tilde{s})$$ for all $B\in \sigma (X)$.
  • Let $\lambda :S\to {\mathbb{R}}^k$, $\theta$ is observationally equivalent to $\tilde{\theta }$ if $\mathrm{\exists }s,\tilde{s}\in S$ that are observationally equivalent and $\theta =\lambda (s)$ and $\tilde{\theta }=\lambda (\tilde{s})$
  • $s_0\in S$ is identified if there is no $s\ne s_0$ that is observationally equivalent to $s_0$
  • ${\theta }_0$ is identified (in $S$) if there is no observationally equivalent $\theta \ne {\theta }_0$

• Cramér-Rao Bound: in the parametric model $P_X\in \{P_{\theta }:\theta \in {\mathbb{R}}^d\}$ wiht likelihood $\ell (\theta ;x)$, if appropriate derivatives and integrals can be interchanged, then for any unbiased estimator $\tau (X)$, $${\mathrm{Var}}_{\theta }(\tau (X))\ge I(\theta {)}^{-1}$$ where $I(\theta )=\int s(x,\theta )s(x,\theta {)}^{\prime }d{P}_{\theta }(x)=\mathrm{E}[H(x,\theta )]$ and $s(x,\theta )=\frac{\partial \log \ell (\theta ;x)}{\partial \theta }$

• Hypothesis testing:

  • $P(\text{reject }{H}_0|P_x\in {\mathcal{P}}_0)$=Type I error rate $=P_x(C)$
  • $P(\text{fail to reject }{H}_0|P_x\in {\mathcal{P}}_1)$=Type II error rate
  • $P(\text{reject }{H}_0|P_x\in {\mathcal{P}}_1)$ = power
  • $\underset{P_x\in {\mathcal{P}}_0}{sup}{P}_x(C)$ = size of test
  • Neyman-Pearson Lemma: Let $\mathrm{\Theta }=\{0,1\}$, $f_0$ and $f_1$ be densities of $P_0$ and $P_1$, $\tau (x)=f_1(x)/f_0(x)$ and $C^{\ast }=\{x\in X:\tau (x)>c\}$. Then among all tests $C$ s.t. $P_0(C)=P_0(C^{\ast })$, $C^{\ast }$ is most powerful.

• Projection: $P_{L}y\in L$ is the projection of $y$ on $L$ if $$\Vert y-P_{L}y\Vert =\underset{w\in L}{inf}\Vert y-w\Vert$$

  1. $P_{L}y$ exists, is unique, and is a linear function of $y$
  2. For any $y_1^{\ast }\in L$, $y_1^{\ast }=P_{L}y$ iff $y-y_1^{\ast }\perp L$
  3. $G=P_L$ iff $Gy=y\mathrm{\forall }y\in L$ and $Gy=0\mathrm{\forall }y\in L^{\perp }$
  4. Linear $G:V\to V$ is a projection map onto its range, $\mathcal{R}(G)$, iff $G$ is idempotent and symmetric.

• Gauss-Markov: $Y=\theta +u$ with $\theta \in L\subset {\mathbb{R}}^n$, a known subspace. If $\mathrm{E}[u]=0$ and $\mathrm{E}[u{u}^{\prime }]={\sigma }^2{I}_n$, then the best linear unbiased estimator (BLUE) of $a^{\prime }\theta =a^{\prime }\hat{\theta }$ where $\hat{\theta }=P_{L}y$

• Convergence in probability:

  • $X_1,X_2,...$ converge in probability to $Y$ if $\mathrm{\forall }ϵ>0$, $\underset{n\to \mathrm{\infty }}{lim}P(\Vert X_n-Y\Vert >ϵ)=0$
  • If $\underset{n\to \mathrm{\infty }}{lim}\mathrm{E}[{\Vert X_n-Y\Vert }^p]\to 0$, then $X_n\stackrel{p}{\to }Y$
  • If $X_n\stackrel{p}{\to }X$, and $f$ continuous, then $f(X_n)\stackrel{p}{\to }f(X)$
  • Weak LLN: if $X_1,...,X_n$ are i.i.d. and $\mathrm{E}[X^2]$ exists, then $\frac{1}{n}\sum _{i=1}^n{X}_i\stackrel{p}{\to }\mathrm{E}[X]$
  • $X_n=O_p(b_n)$ if $\mathrm{\forall }ϵ>0$ $\mathrm{\exists }{M}_{ϵ}$ s.t. $limsupP(\frac{\Vert X_n\Vert }{b_n}\ge M_{ϵ})<ϵ$
  • $X_n=o_p(b_n)$ if $\frac{X_n}{b_n}\stackrel{p}{\to }0$

• Convergence in distribution:

  • $X_1,X_2,...$ converge in distribution to $X$ if $\mathrm{\forall }f\in {\mathcal{C}}_b$, $\mathrm{E}[f(X_n)]\to \mathrm{E}[f(X)]$
  • If $X_n\stackrel{d}{\to }X$ and $g$ is continuous, then $g(X_n)\stackrel{d}{\to }g(X)$
  • Slutsky’s lemma: if $Y_n\stackrel{p}{\to }c$ and $X_n\stackrel{d}{\to }X$ and $g$ is continuious, then $g(Y_n,X_n)\stackrel{d}{\to }g(c,X)$
  • Levy’s Continuity Theorem: $X_n\stackrel{d}{\to }X$ iff $\mathrm{E}[e^{i{t}^{\prime }{X}_n}]\to \mathrm{E}[e^{i{t}^{\prime }X}]\mathrm{\forall }t$
  • CLT: if $X_1,...,X_n$ are i.i.d. with $\mathrm{E}[X_1]=\mu$ and $\mathrm{Var}(X_1)={\sigma }^2$, then $\frac{1}{\sqrt{n}}\sum _{i=1}^n\frac{X_i-\mu }{\sigma }\stackrel{d}{\to }N(0,1)$
  • Delta Method: suppose $\sqrt{n}(\hat{\theta }-{\theta }_0)\stackrel{d}{\to }S$ and $h$ is differentiable, then $\sqrt{n}(h(\hat{\theta })-h({\theta }_0))\stackrel{d}{\to }{\mathrm{\nabla }}_h({\theta }_0)S$

Footnotes

1. If you cannot find the maximum likelihood estimator, show whether $\overline{Y}/K=\frac{1}{nK}\sum _{i=1}^n{Y}_i$ is unbiased and consistent for partial credit.

2. If you could not answer that part, suppose you had shown that $\beta =\mathrm{E}[{\overline{x}}_j{\overline{x}}_j^{\prime }{]}^{-1}\mathrm{E}[{\overline{x}}_j{\overline{y}}_j]$.