ECON 626: Problem Set 4

Published

October 29, 2026

\[ \def\R{{\mathbb{R}}} \def\Er{{\mathrm{E}}} \def\var{{\mathrm{Var}}} \def\cov{{\mathrm{Cov}}} \newcommand\norm[1]{\left\lVert#1\right\rVert} \]

Problem 1: Sufficient Statistics and Efficiency via Projection

Read chapter 6, especially 6.2 and 6.3 of Song (2021).

In the lecture slides (Projection Geometry), we saw that in any complete inner product space \(V\), the projection \(P_L y\) of a vector \(y\) onto a subspace \(L\) is the unique element in \(L\) such that the residual is orthogonal to \(L\): \(y - P_L y \perp L\).

Here, let \(V = \mathcal{L}^2\) be the Hilbert space of square-integrable random variables with inner product \(\langle Y_1, Y_2 \rangle = \Er[Y_1 Y_2]\) and norm \(\norm{Y} = \sqrt{\Er[Y^2]}\). When \(L = \mathcal{L}^2(\sigma(T))\) is the subspace of random variables that are functions of a statistic \(T\), the projection map \(P_L\) is the conditional expectation: \(P_L Y = \Er[Y \mid T]\).

Now suppose \(X_1, \dots, X_n\) are i.i.d. \(\text{Uniform}(0,\theta)\) with \(\theta > 0\).

  1. Show that \(T = \max_{1 \leq i \leq n} X_i\) is a sufficient statistic for \(\theta\).

  2. Consider the crude unbiased estimator \(W = 2\bar{X} = \frac{2}{n}\sum_{i=1}^n X_i\). According to the Rao-Blackwell Theorem, we can improve \(W\) by projecting it onto the subspace \(L = \mathcal{L}^2(\sigma(T))\) generated by the sufficient statistic. Show that this projection is: \[ \hat{\theta} \equiv P_L(2\bar{X}) = \Er[2\bar{X} \mid T] = \frac{n+1}{n} T. \] Hint: Given \(\max_{1 \leq j \leq n} X_j = t\), for each \(i\), \(X_i = t\) with probability \(1/n\), and with probability \(\frac{n-1}{n}\), \(X_i \sim \text{Uniform}(0, t)\).

  3. By the Projection Theorem from the slides, the decomposition \(W = P_L W + (W - P_L W)\) has orthogonal components: \(P_L W \perp (W - P_L W)\).

    1. Use this orthogonality to deduce the Pythagorean variance decomposition: \[ \var(2\bar{X}) = \var(\hat{\theta}) + \var(2\bar{X} - \hat{\theta}), \] and explain why this implies that the projection \(\hat{\theta}\) must have weakly smaller variance than \(2\bar{X}\).

    2. Verify this explicitly by computing \(\var(2\bar{X})\) and \(\var(\hat{\theta})\).

  4. Does this model satisfy the regularity conditions for the Cramér-Rao lower bound? Why or why not? Explain why the Lehmann-Scheffé Theorem nevertheless guarantees that the projected estimator \(\hat{\theta}\) is the unique minimum variance unbiased estimator.

Problem 2: Cramér-Rao Bounds and Semiparametric Efficiency

In this problem, we explore how understanding the Cramér-Rao lower bound as a projection can help extend the theory to semiparametric efficiency bounds.

  1. The Cramér-Rao bound is a 1-dimensional projection: Consider a parametric model with scalar parameter \(\theta\), score \(s(X) = \frac{\partial \log f(X;\theta)}{\partial \theta}\), and Fisher information \(I = \Er[s(X)^2] > 0\). Let \(\hat{\theta}\) be any unbiased estimator of \(\theta\).

    1. Differentiating the unbiasedness identity \(\Er_\theta[\hat{\theta}] = \theta\) under the integral sign, show that \[ \langle \hat{\theta} - \theta, s(X) \rangle \equiv \Er[(\hat{\theta} - \theta) s(X)] = 1. \]

    2. In the Hilbert space \(V = L^2_0(P)\) of mean-zero random variables, let \[ L = \text{span}\{s(X)\} = \{c s(X) : c \in \R\} \] be the 1-dimensional subspace spanned by the score. Using the formula for projection onto a line from the slides, show that the projection of the estimation error \(\hat{\theta} - \theta\) onto \(L\) is \[ P_L(\hat{\theta} - \theta) = \frac{\langle \hat{\theta} - \theta, s(X) \rangle}{\norm{s(X)}^2} s(X) = \frac{s(X)}{I}. \]

    3. Use the Pythagorean identity, \[ \norm{\hat{\theta} - \theta}^2 = \norm{P_L(\hat{\theta} - \theta)}^2 + \norm{(\hat{\theta} - \theta) - P_L(\hat{\theta} - \theta)}^2, \] to deduce that \(\var(\hat{\theta}) \geq \norm{P_L(\hat{\theta} - \theta)}^2 = \frac{1}{I}\). Conclude that the Cramér-Rao lower bound is achieved if and only if the residual is zero, i.e., \(\hat{\theta} - \theta = \frac{s(X)}{I}\) almost surely.

  2. Parametric submodels and the hardest direction: Now consider estimating the population mean \(\mu = \Er[X]\) in a semiparametric model where the density \(f(x)\) is unknown. Consider a 1-dimensional parametric submodel (a curve through \(f\) in the space of densities): \[ f_t(x) = f(x)\big(1 + t \cdot s(x)\big), \quad t \in (-\epsilon, \epsilon), \] where \(s(x)\) is a score satisfying \(\Er[s(X)] = 0\) and \(\Er[s(X)^2] > 0\).

    1. Show that the derivative of the mean along this submodel at \(t = 0\) is \[ \mu'(0) \equiv \left. \frac{d}{dt} \Er_{f_t}[X] \right|_{t=0} = \Er[(X - \mu) s(X)] = \langle \phi, s \rangle, \] where \(\phi(X) = X - \mu\) is the influence function (gradient).

    2. Along this submodel, the Cramér-Rao lower bound for estimating \(\mu\) is \[ CR(s) = \frac{\langle \phi, s \rangle^2}{\norm{s}^2}. \] Notice that \(CR(s) = \norm{P_{\text{span}\{s\}} \phi}^2\): the Cramér-Rao bound along submodel \(s\) is simply the squared length of the projection of the gradient \(\phi\) onto the 1D subspace spanned by \(s\)!

    3. The semiparametric efficiency bound is the bound for the hardest (least favorable) submodel: \[ V^* = \sup_{s \neq 0} CR(s) = \sup_{s \neq 0} \norm{P_{\text{span}\{s\}} \phi}^2. \] Show that \(V^* = \norm{\phi}^2 = \var(X)\), and explain why the sample mean \(\bar{X}\) achieves this bound.

  3. Restricted models and projection onto the tangent space: Now suppose we observe \(W = (Y, Z)'\) and wish to estimate \(\mu = \Er[Y]\), but economic theory provides the restriction that \(\Er[Z] = 0\).

    1. Show that for any submodel \(f_t(w) = f(w)(1 + t \cdot s(w))\) satisfying \(\Er_t[Z] = 0\) for all \(t\), the score \(s(W)\) must be orthogonal to \(Z\): \[ \langle Z, s \rangle = \Er[Z s(W)] = 0. \] The subspace of all such scores is the tangent space \(\mathcal{T} = \{s \in L^2_0(P) : s \perp Z\}\).

    2. The unrestricted gradient is \(\phi(W) = Y - \mu\). Using the projection formula from the slides (Theorem 1.4: \(P_{L \cap L_0^\perp} = P_L - P_{L_0}\)), show that the orthogonal projection of \(\phi(W)\) onto the tangent space \(\mathcal{T} = (\text{span}\{Z\})^\perp\) is \[ \phi^*(W) \equiv P_\mathcal{T} \phi = (Y - \mu) - \frac{\cov(Y, Z)}{\var(Z)} Z. \]

    3. Deduce that the restricted semiparametric efficiency bound is the squared length of this projection: \[ V^* = \sup_{s \neq 0, s \in \mathcal{T}} CR(s) = \norm{P_\mathcal{T} \phi}^2 = \var(Y) - \frac{\cov(Y, Z)^2}{\var(Z)} = \var(Y)(1 - \rho^2), \] where \(\rho = \text{Corr}(Y, Z)\).

    4. How does the geometric fact that \(\norm{P_\mathcal{T} \phi}^2 \leq \norm{\phi}^2\) translate into intuition about the value of model restrictions? Verify that the regression estimator \(\hat{\mu} = \bar{Y} - \hat{\beta}\bar{Z}\) achieves this bound.

References

Song, Kyunchul. 2021. Introduction to Econometrics.