These practice questions accompany the Probability lecture and will be discussed in the review session.
Question 1
For the rolling of a die experiment, define \(X(\omega)= \omega\) for \(\omega\in\Omega=\{1,2,3,4,5,6\}\). Construct the CDF \(F(x)\) of \(X\).
\[
F(x) = \begin{cases}
0 & x < 1 \\
1/6 & 1 \leq x < 2 \\
2/6 & 2 \leq x < 3 \\
3/6 & 3 \leq x < 4 \\
4/6 & 4 \leq x < 5 \\
5/6 & 5 \leq x < 6 \\
1 & x \geq 6
\end{cases}
\]
Question 2
Suppose \(X\) is uniformly distributed on \([a,b]\): the PDF is a strictly positive constant on \([a,b]\), and is zero otherwise. Construct the CDF and PDF of \(X\).
PDF and CDF of \(X \sim \text{Uniform}(a,b)\):
\[
f(x) = \begin{cases}
0 & x < a \\
\frac{1}{b-a} & a \leq x \leq b \\
0 & x > b
\end{cases}
\]
\[
F(x) = \begin{cases}
0 & x < a \\
\frac{x-a}{b-a} & a \leq x \leq b \\
1 & x > b
\end{cases}
\]
Question 3
The joint PDF of \(X\) and \(Y\) is \(f(x,y) = ax\cdot \mathbb{1}(0<x<1,x<y<1)\), where \(\mathbb{1}(\cdot)\) is the indicator function that takes the value 1 if the condition is true and 0 otherwise.
- Find \(a\) such that \(f(x,y)\) is a valid PDF
- Find the marginal PDF of \(X\)
- Find the marginal PDF of \(Y\)
- Are \(X\) and \(Y\) independent?
A valid PDF must integrate to 1: \(\int_{x=0}^1 \int_{y=x}^1 a x\, dy\, dx = 1\). The inner integral is \(\int_{y=x}^1 dy = 1 - x\), so \(\int_{x=0}^1 a x (1 - x)\, dx = a \int_{x=0}^1 x(1 - x)\, dx\). But \(\int_{x=0}^1 x(1 - x)\, dx = \frac{1}{2} - \frac{1}{3} = \frac{1}{6}\), so \(a \cdot \frac{1}{6} = 1 \implies a = 6\)
\(f_X(x) = \int_{y=x}^1 f(x, y)\, dy = \int_{y=x}^1 6x\, dy = 6x (1 - x)\) for \(0 < x < 1\). The PDF is zero for \(x \leq 0\) or \(x \geq 1\)
\(f_Y(y) = \int_{x=0}^y f(x, y)\, dx = \int_{x=0}^y 6x\, dx = 6 \cdot \frac{y^2}{2} = 3y^2\) for \(0 < y < 1\). The PDF is zero for \(y \leq 0\) or \(y \geq 1\)
\(X\) and \(Y\) are not independent: the indicator function \(\mathbb{1}(x < y < 1)\) cannot be written as a product of functions of \(x\) and \(y\) alone. Thus, the joint PDF cannot be factored into the product of the marginal PDFs
Question 4
Suppose \(X\) is a Bernoulli random variable: \(\mathbb{P}[X=1]=p\), \(\mathbb{P}[X=0]=1-p\) for \(p\in(0,1)\).
- Compute the first and second moments of \(X\)
- Compute the variance of \(X\)
\({\mathbb{E}}[X]=p\), \({\mathbb{E}}[X^2]=p\)
\(\operatorname{Var}(X)=p-p^2=p(1-p)\)
Question 5
Suppose \({\mathbb{E}}[X]={\mathbb{E}}[Y]\) for random variables \(X\) and \(Y\). Does this imply that \(X=Y\)? Prove if true or give a counterexample if false.
The claim “\({\mathbb{E}}[X]={\mathbb{E}}[Y]\) implies \(X=Y\)” is false. Counterexample: Let \(X\sim \text{Bernoulli}(1/2)\) and \(Y=1-X\). Then \({\mathbb{E}}[X]={\mathbb{E}}[Y]=1/2\), but \(\mathbb{P}[X=Y]=0\), so \(X\neq Y\) almost surely.
Question 6
For the joint PDF from Question 3, compute \({\mathbb{E}}[X]\), \({\mathbb{E}}[Y]\), \({\mathbb{E}}[XY]\), and \(\operatorname{Cov}(X,Y)\).
For \(f(x,y)=6x\cdot \mathbb{1}(0<x<1,\;x<y<1)\):
\[
\begin{aligned}
{\mathbb{E}}[X]&=\int_{0}^{1}\int_{x}^{1} x\cdot 6x\,dy\,dx = \int_{0}^{1} 6x^2(1-x)\,dx = 1/2 \\
{\mathbb{E}}[Y]&=\int_{0}^{1}\int_{0}^{y} y\cdot 6x\,dx\,dy = \int_{0}^{1} y\cdot 3y^2\,dy = 3/4 \\
{\mathbb{E}}[XY]&=\int_{0}^{1}\int_{x}^{1} xy\cdot 6x\,dy\,dx = \int_{0}^{1} 3x^2(1-x^2)\,dx = 2/5 \\
\operatorname{Cov}(X,Y)&={\mathbb{E}}[XY]-{\mathbb{E}}[X]{\mathbb{E}}[Y]=2/5-(1/2)(3/4)=1/40
\end{aligned}
\]
Question 7
Compute the quantiles of order \(\tau\) for the Uniform\((a,b)\) distribution.
For \(X \sim \text{Uniform}(a,b)\), the CDF is \(F_X(x) = 0\) for \(x < a\), \(F_X(x) = \frac{x-a}{b-a}\) for \(a \le x \le b\), and \(F_X(x) = 1\) for \(x > b\). By definition, \(q_\tau = a + \tau(b-a)\) for \(0 < \tau < 1\); \(q_0 = a\), \(q_1 = b\)
Question 8
Suppose \(Y=g(X)\), where \(g\) is a strictly increasing function. Show that \(q_\tau^Y = g(q_\tau^X)\), where \(q_\tau^X\) and \(q_\tau^Y\) are the quantiles of \(X\) and \(Y\), respectively.
\(\mathbb{P}[Y \leq g(q_\tau^X)] = \mathbb{P}[g(X) \leq g(q_\tau^X)] = \mathbb{P}[X \leq q_\tau^X] = \tau\), so \(q_\tau^Y = g(q_\tau^X)\), where the second equality holds because \(g\) is strictly increasing, preserving the order of inequalities
Question 9
Using the Markov inequality, show that \(\mathbb{P}[|X - {\mathbb{E}}[X]| \geq a] \leq \frac{\operatorname{Var}(X)}{a^2}\) for any \(a > 0\).
Let \(Z=(X-{\mathbb{E}}[X])^2\ge 0\). By Markov, \(\mathbb{P}[|X-{\mathbb{E}}[X]|\ge a]=\mathbb{P}[Z\ge a^2]\le {\mathbb{E}}[Z]/a^2=\operatorname{Var}(X)/a^2\) for any \(a>0\)
Question 10
The correlation coefficient \(\rho_{XY}\) between the random variables \(X\) and \(Y\) is defined as \(\rho_{XY} = \frac{\operatorname{Cov}(X,Y)}{\sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}}\). Show that \(|\rho_{XY}| \leq 1\) using the Cauchy-Schwarz inequality.
By Cauchy–Schwarz, \(|{\mathbb{E}}[(X-{\mathbb{E}}[X])(Y-{\mathbb{E}}[Y])]| \le \sqrt{{\mathbb{E}}[(X-{\mathbb{E}}[X])^2]\,{\mathbb{E}}[(Y-{\mathbb{E}}[Y])^2]}\), hence \(|\operatorname{Cov}(X,Y)|\le \sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}\) and \(|\rho_{XY}|\le 1\) (assuming positive, finite variances)
Question 11
Using Jensen’s inequality, show that if \({\mathbb{E}}[|X|^p]<\infty\) for some \(p>0\), then \({\mathbb{E}}[|X|^r]<\infty\) for all \(r\in[0,p]\). Hint: \(|x|^s\) is concave for \(0<s<1\).
\(|x|^r=(|x|^p)^{r/p}\), where \((\cdot)^{r/p}\) is concave for \(0<r<p\). Therefore, \({\mathbb{E}}[|X|^r]={\mathbb{E}}[(|X|^p)^{r/p}]\le ({\mathbb{E}}[|X|^p])^{r/p}<\infty\) for \(0<r<p\), where the first inequality follows by Jensen’s and the second by the assumption \({\mathbb{E}}[|X|^p]<\infty\).
Question 12
A disease affects 1% of the population: \(\mathbb{P}[D] = 0.01\). A diagnostic test has sensitivity \(\mathbb{P}[T^+ \mid D] = 0.95\) and specificity \(\mathbb{P}[T^- \mid D^c] = 0.90\). A randomly chosen person tests positive. Using the multiplication rule, the law of total probability, and Bayes’ rule, compute the posterior \(\mathbb{P}[D \mid T^+]\).
From specificity, the false-positive rate is \(\mathbb{P}[T^+ \mid D^c] = 1 - 0.90 = 0.10\), and \(\mathbb{P}[D^c] = 0.99\).
Law of total probability: \[
\mathbb{P}[T^+] = \mathbb{P}[T^+\mid D]\mathbb{P}[D] + \mathbb{P}[T^+\mid D^c]\mathbb{P}[D^c] = (0.95)(0.01) + (0.10)(0.99) = 0.0095 + 0.099 = 0.1085.
\]
Bayes’ rule: \[
\mathbb{P}[D\mid T^+] = \frac{\mathbb{P}[T^+\mid D]\mathbb{P}[D]}{\mathbb{P}[T^+]} = \frac{0.0095}{0.1085} = \frac{19}{217} \approx 0.088.
\]
Even with 95% sensitivity, only about 8.8% of positive testers have the disease: because the disease is rare, false positives from the healthy 99% far outnumber true positives. This base-rate effect is the main lesson.
Question 13
The joint PMF of discrete \(X_1 \in \{1,2\}\) and \(X_2 \in \{1,2,3\}\) is
| \(X_1=1\) |
0.10 |
0.15 |
0.05 |
| \(X_1=2\) |
0.20 |
0.30 |
0.20 |
Compute the marginal PMFs \(p_{X_1}(x_1)\) and \(p_{X_2}(x_2)\), and verify each sums to 1.
Marginal of \(X_1\) (sum across each row): \[
p_{X_1}(1) = 0.10 + 0.15 + 0.05 = 0.30, \qquad p_{X_1}(2) = 0.20 + 0.30 + 0.20 = 0.70,
\] and \(0.30 + 0.70 = 1\). \(\checkmark\)
Marginal of \(X_2\) (sum down each column): \[
p_{X_2}(1) = 0.30, \qquad p_{X_2}(2) = 0.45, \qquad p_{X_2}(3) = 0.25,
\] and \(0.30 + 0.45 + 0.25 = 1\). \(\checkmark\)
Question 14
Let \(X \in \{0,1\}\) and \(Y \in \{0,1,2\}\) have joint PMF
| \(X=0\) |
0.10 |
0.20 |
0.10 |
| \(X=1\) |
0.05 |
0.25 |
0.30 |
Compute \({\mathbb{E}}[Y \mid X=0]\) and \({\mathbb{E}}[Y \mid X=1]\), and express \({\mathbb{E}}[Y\mid X]\) as a function of \(X\).
Marginal of \(X\): \(p_X(0) = 0.40\), \(p_X(1) = 0.60\).
Conditional on \(X=0\): \(p_{Y\mid X}(\cdot\mid 0) = (0.25, 0.50, 0.25)\), so \[
{\mathbb{E}}[Y\mid X=0] = 0(0.25) + 1(0.50) + 2(0.25) = 1.
\]
Conditional on \(X=1\): \(p_{Y\mid X}(\cdot\mid 1) = \big(\tfrac{1}{12}, \tfrac{5}{12}, \tfrac{6}{12}\big)\), so \[
{\mathbb{E}}[Y\mid X=1] = 1\cdot\tfrac{5}{12} + 2\cdot\tfrac{6}{12} = \tfrac{17}{12} \approx 1.417.
\]
As a random variable, \({\mathbb{E}}[Y\mid X] = g(X)\) with \(g(0) = 1\) and \(g(1) = \tfrac{17}{12}\). Since \(X\) is binary, equivalently \({\mathbb{E}}[Y\mid X] = 1 + \tfrac{5}{12}\,X\).
Question 15
Prove the “taking out what is known” property for discrete random variables: for any function \(g\), \[
{\mathbb{E}}[g(X)\,Y \mid X] = g(X)\,{\mathbb{E}}[Y\mid X].
\]
Fix a value \(x\) with \(p_X(x) > 0\). Conditional on \(X = x\), the factor \(g(X)\) is the constant \(g(x)\), so \[
{\mathbb{E}}[g(X)Y \mid X = x] = \sum_y g(x)\,y\,p_{Y\mid X}(y\mid x) = g(x)\sum_y y\,p_{Y\mid X}(y\mid x) = g(x)\,{\mathbb{E}}[Y\mid X=x].
\] The constant \(g(x)\) pulls out of the sum because it does not depend on \(y\). Since this holds for every value \(x\) of \(X\), the random variables agree: \({\mathbb{E}}[g(X)Y\mid X] = g(X)\,{\mathbb{E}}[Y\mid X]\). Once we condition on \(X\), anything that is a function of \(X\) is known and factors out.
Question 16
A course has two sections: A has 30 students averaging 72, and B has 20 students averaging 80. With \(S\) the section and \(G\) the grade, \({\mathbb{E}}[G\mid S=A] = 72\) and \({\mathbb{E}}[G\mid S=B] = 80\). Use the law of iterated expectations to find \({\mathbb{E}}[G]\).
With \(30 + 20 = 50\) students, the section probabilities are \(\mathbb{P}[S=A] = 0.6\) and \(\mathbb{P}[S=B] = 0.4\). By the law of iterated expectations, \[
{\mathbb{E}}[G] = {\mathbb{E}}[G\mid S=A]\mathbb{P}[S=A] + {\mathbb{E}}[G\mid S=B]\mathbb{P}[S=B] = 72(0.6) + 80(0.4) = 43.2 + 32 = 75.2.
\] This is a weighted average of the section means, pulled toward \(72\) because section A is larger; the naive average \((72+80)/2 = 76\) would over-weight the smaller section.
Question 17
Using the joint PMF of \(X \in \{0,1\}\), \(Y \in \{0,1,2\}\) from Question 14,
| \(X=0\) |
0.10 |
0.20 |
0.10 |
| \(X=1\) |
0.05 |
0.25 |
0.30 |
compute \({\mathbb{E}}[Y]\) two ways and confirm they agree: (a) directly from the marginal \(p_Y\); (b) via the law of iterated expectations.
(a) Directly. Marginal of \(Y\): \(p_Y(0) = 0.15\), \(p_Y(1) = 0.45\), \(p_Y(2) = 0.40\). Then \[
{\mathbb{E}}[Y] = 0(0.15) + 1(0.45) + 2(0.40) = 1.25.
\]
(b) Via the LIE. From Question 14, \({\mathbb{E}}[Y\mid X=0] = 1\), \({\mathbb{E}}[Y\mid X=1] = \tfrac{17}{12}\), with \(p_X(0)=0.40\), \(p_X(1)=0.60\): \[
{\mathbb{E}}[Y] = 1(0.40) + \tfrac{17}{12}(0.60) = 0.40 + 0.85 = 1.25.
\]
Both give \({\mathbb{E}}[Y] = 1.25\), confirming \({\mathbb{E}}[Y] = {\mathbb{E}}\big[{\mathbb{E}}[Y\mid X]\big]\). \(\checkmark\)
Question 18
Prove the Cauchy–Schwarz inequality: for random variables \(X, Y\) with finite second moments, \[
\big({\mathbb{E}}[XY]\big)^2 \leq {\mathbb{E}}[X^2]\,{\mathbb{E}}[Y^2].
\] Hint: consider \({\mathbb{E}}[(X - tY)^2] \geq 0\) as a quadratic in the real variable \(t\).
For every real \(t\), \((X - tY)^2 \geq 0\), so \[
q(t) := {\mathbb{E}}[(X - tY)^2] = {\mathbb{E}}[X^2] - 2t\,{\mathbb{E}}[XY] + t^2\,{\mathbb{E}}[Y^2] \geq 0 \quad\text{for all } t.
\] If \({\mathbb{E}}[Y^2] = 0\) then \(Y = 0\) almost surely and both sides vanish. Otherwise \(q\) is a quadratic in \(t\) with positive leading coefficient that never goes negative, so its discriminant is nonpositive: \[
\big(2\,{\mathbb{E}}[XY]\big)^2 - 4\,{\mathbb{E}}[Y^2]\,{\mathbb{E}}[X^2] \leq 0 \;\Longrightarrow\; \big({\mathbb{E}}[XY]\big)^2 \leq {\mathbb{E}}[X^2]\,{\mathbb{E}}[Y^2].
\] Equality holds iff \(q(t) = 0\) for some \(t\), i.e. \(X = tY\) almost surely. Applying the inequality to the centered variables \(X - {\mathbb{E}}[X]\) and \(Y - {\mathbb{E}}[Y]\) gives \(|\operatorname{Cov}(X,Y)| \leq \sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}\), hence \(|\rho_{XY}| \leq 1\) (compare Question 10).