These practice questions accompany the Linear Algebra lecture and will be discussed in the review session.
Question 1
Compute the following matrix and vector products:
- \(\begin{bmatrix}3 & 1\\ 2 & 4 \end{bmatrix} \begin{bmatrix}2 & 1 & 0 \\ 1 & 5 & 2 \end{bmatrix}\)
- \(\begin{bmatrix}0 & 0 & 1 \\ 3 & 1 & 1 \\ -2 & 3 & 0 \end{bmatrix}\begin{bmatrix}2 & 0 & 5 \\ 1 & 3 & 1 \\ 4 & 1 & 2 \end{bmatrix}\)
- \(\begin{bmatrix}5 & 3 & 2 \end{bmatrix}\begin{bmatrix}2 & 7 & 1 \\ 0 & 4 & 5 \\ 2 & 3 & 4 \end{bmatrix}\)
- \(\begin{bmatrix}2 & 3 \\ 1 & -2 \end{bmatrix}\begin{bmatrix}2 \\ -1 \end{bmatrix}\)
Each entry of the product is the inner product of the corresponding row of the first matrix with the corresponding column of the second.
\[
\begin{bmatrix}3 & 1\\ 2 & 4 \end{bmatrix} \begin{bmatrix}2 & 1 & 0 \\ 1 & 5 & 2 \end{bmatrix}
= \begin{bmatrix}
3\cdot 2+1\cdot 1 & 3\cdot 1+1\cdot 5 & 3\cdot 0+1\cdot 2 \\
2\cdot 2+4\cdot 1 & 2\cdot 1+4\cdot 5 & 2\cdot 0+4\cdot 2
\end{bmatrix}
= \begin{bmatrix}7 & 8 & 2 \\ 8 & 22 & 8 \end{bmatrix}
\]
\[
\begin{aligned}
\begin{bmatrix}0 & 0 & 1 \\ 3 & 1 & 1 \\ -2 & 3 & 0 \end{bmatrix}\begin{bmatrix}2 & 0 & 5 \\ 1 & 3 & 1 \\ 4 & 1 & 2 \end{bmatrix}
&= \begin{bmatrix}
0\cdot 2+0\cdot 1+1\cdot 4 & 0\cdot 0+0\cdot 3+1\cdot 1 & 0\cdot 5+0\cdot 1+1\cdot 2 \\
3\cdot 2+1\cdot 1+1\cdot 4 & 3\cdot 0+1\cdot 3+1\cdot 1 & 3\cdot 5+1\cdot 1+1\cdot 2 \\
-2\cdot 2+3\cdot 1+0\cdot 4 & -2\cdot 0+3\cdot 3+0\cdot 1 & -2\cdot 5+3\cdot 1+0\cdot 2
\end{bmatrix} \\
&= \begin{bmatrix} 4 & 1 & 2 \\ 11 & 4 & 18 \\ -1 & 9 & -7 \end{bmatrix}
\end{aligned}
\]
\[
\begin{aligned}
\begin{bmatrix}5 & 3 & 2 \end{bmatrix}\begin{bmatrix}2 & 7 & 1 \\ 0 & 4 & 5 \\ 2 & 3 & 4 \end{bmatrix}
&= \begin{bmatrix} 5\cdot 2+3\cdot 0+2\cdot 2 & 5\cdot 7+3\cdot 4+2\cdot 3 & 5\cdot 1+3\cdot 5+2\cdot 4 \end{bmatrix} \\
&= \begin{bmatrix} 14 & 53 & 28 \end{bmatrix}
\end{aligned}
\]
\[
\begin{bmatrix}2 & 3 \\ 1 & -2 \end{bmatrix}\begin{bmatrix}2 \\ -1 \end{bmatrix}
= \begin{bmatrix} 2\cdot 2+3\cdot (-1) \\ 1\cdot 2+(-2)\cdot (-1) \end{bmatrix}
= \begin{bmatrix} 1 \\ 4 \end{bmatrix}
\]
Question 2
Being formal and explicit in the rules of matrix algebra (e.g. when things are commutative, distributive, when you require invertibility, etc.) solve the following equations for \(x \in \mathbb{R}^N\), with constants: vector \(b,c \in \mathbb{R}^N\), matrices \(A,D,Q,R\) all \(\mathbb{R}^{N\times N}\), and scalar \(m \in \mathbb{R}\),
- \(x A = b + c\)
- \(Ax + \left(x^{\top} D\right)^{\top} = b\)
- \(Q^{-1} \left( A\, x\, m + m b\right) = Rx\)
- Starting from \(x A = b + c\):
- Assume \(A\) is invertible
- \(x A A^{-1} = (b + c) A^{-1}\), right multiplying by \(A^{-1}\)
- \(x I = (b+c)A^{-1}\), using \(A A^{-1} = I\)
- \(x = (b+c)A^{-1}\), using \(I x = x\)
- Starting from \(Ax + \left(x^{\top} D\right)^{\top} = b\):
- \(Ax + D^{\top} (x^{\top})^{\top} = b\), using \((A B)^{\top} = B^{\top} A^{\top}\)
- \(Ax + D^{\top} x = b\), using \((x^{\top})^{\top} = x\)
- \((A + D^{\top}) x = b\), using distributive property of left-multiplication \((A + B)x = A x + B x\)
- If \(A + D^{\top}\) is invertible, then can do additional steps:
- \((A + D^{\top})^{-1} (A + D^{\top}) x = (A + D^{\top})^{-1} b\), using left multiplication
- \(x = (A + D^{\top})^{-1} b\), using \(A^{-1}A = I\) and \(Ix = x\)
- Starting from \(Q^{-1} \left( A\, x\, m + m b\right) = R x\):
- \(Q^{-1} \left(m A x + m b\right) = R x\), using commutativity of scalar multiplication
- \(Q^{-1} m \left(A x + b\right) = R x\), using distributivity of scalar multiplication
- \(Q^{-1} \left(A x + b\right) = \frac{1}{m} R x\), using commutativity of scalar multiplication
- \(Q Q^{-1} \left(A x + b\right) = Q \frac{1}{m} R x\), left multiply by \(Q\)
- \(A x + b = \frac{1}{m} Q R x\), using \(Q Q^{-1} = I\) and commutativity of scalar multiplication
- \(b = \frac{1}{m} Q R x - A x\), rearranging sides
- \(b = \left(\frac{1}{m} Q R - A\right) x\), using distributive property of left-multiplication \((A + B)x = A x + B x\)
- If \(\left(\frac{1}{m} Q R - A\right)\) is invertible, then can do additional steps:
- \(x = \left(\frac{1}{m} Q R - A\right)^{-1} b\), using left multiplication and then \(A^{-1} A = I\)
Question 3
Write the following as a system of equations in matrix form:
- \(\left\{\begin{matrix}2x + 3y = 2 \\ x - 2y = -1\end{matrix}\right.\) where \(\{x,y\}\) are variables
- \(\left\{\begin{matrix}2x - 3y = 1 \\ 3x + my = -2\end{matrix}\right.\) where \(\{x,y\}\) are variables
- \(\left\{\begin{matrix}2a + b - 1 = 0 \\ 3b + 4c = 2 \\ -2a + c = 0\end{matrix}\right.\) where \(\{a,b,c\}\) are variables
- \(\left\{\begin{matrix}a + b = -3 \\ c - y - 4 b = 0\end{matrix}\right.\) where \(\{a,b,c \}\) are variables
- \(\begin{bmatrix}2 & 3 \\ 1 & -2 \end{bmatrix} \begin{bmatrix}x \\ y \end{bmatrix} = \begin{bmatrix}2 \\ -1 \end{bmatrix}\)
- \(\begin{bmatrix}2 & -3 \\ 3 & m \end{bmatrix} \begin{bmatrix}x \\ y \end{bmatrix} = \begin{bmatrix}1 \\ -2 \end{bmatrix}\)
- \(\begin{bmatrix}2 & 1 & 0 \\ 0 & 3 & 4 \\ -2 & 0 & 1 \end{bmatrix} \begin{bmatrix}a \\ b \\ c \end{bmatrix} = \begin{bmatrix}1 \\ 2 \\ 0 \end{bmatrix}\), note moved the \(-1\) to the right hand side
- \(\begin{bmatrix}1 & 1 & 0\\ 0 & -4 & 1\end{bmatrix} \begin{bmatrix}a \\ b\\c \end{bmatrix} = \begin{bmatrix}-3 \\ y \end{bmatrix}\), moved the \(y\) to the right hand side since it is a constant given stated assumptions
Question 4
- Find \(G \in \mathbb{R}^2\) such that \(G \cdot \begin{bmatrix}a & b\end{bmatrix}\) returns the second element, \(b\).
- Find \(H \in \mathbb{R}^{1 \times 2}\) such that \(H \begin{bmatrix}x & y\end{bmatrix}^\top\) returns the sum \(x + y\).
- Find a \(2 \times 2\) matrix \(M\) such that \(M \begin{bmatrix}p & q\end{bmatrix}^\top = \begin{bmatrix}q & p\end{bmatrix}^\top\) (i.e., swaps the two elements).
- Let \(T\) be a linear transformation from \(\mathbb{R}^2\) to \(\mathbb{R}^2\) that doubles the first coordinate and leaves the second unchanged. Write a matrix representation of \(T\).
- \(G = \begin{bmatrix} 0 & 1 \end{bmatrix}^{\top}\), with or without the transpose since uses a dot-product
- \(H = \begin{bmatrix} 1 & 1\end{bmatrix}\)
- \(M = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\)
- \(T = \begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}\)
Question 5
- Find a vector \(x \in \mathbb{R}^2\) such that \(x \cdot \begin{bmatrix}1 & 2\end{bmatrix} = 0\)
- Find a vector \(x \in \mathbb{R}^2\) such that \(x \cdot \begin{bmatrix}0 & 1\end{bmatrix} = 1\)
- Find 3 vectors of unit length in \(\mathbb{R}^3\) which are mutually orthogonal
- Given vectors \(x^1\) and \(x^2\) which may have a different norm (i.e., length), how can I use the norm and inner product to test if they are orthogonal? collinear?
- \(x = \begin{bmatrix} 2 & -1 \end{bmatrix}\)
- \(x = \begin{bmatrix} 0 & 1 \end{bmatrix}\)
- The canonical orthonormal basis works: \(x^1 = \begin{bmatrix} 1 & 0 & 0 \end{bmatrix}, x^2 = \begin{bmatrix} 0 & 1 & 0 \end{bmatrix}, x^3 = \begin{bmatrix} 0 & 0 & 1 \end{bmatrix}\). Each has unit length and every pair has zero inner product. (Note: \(3\) is the maximum — there is no set of \(4\) mutually orthogonal nonzero vectors in \(\mathbb{R}^3\).)
- If \(x^1 \cdot x^2 = 0\) then they are orthogonal.
- To test if they are collinear, one approach might be to make them the same length with \(\frac{x^1}{\|x^1\|_2} \cdot \frac{x^2}{\|x^2\|_2}\) and check whether the inner product \(\frac{x^1}{\|x^1\|_2} \cdot \frac{x^2}{\|x^2\|_2} = \pm 1\)
Question 6
- What is the span of the vectors \(\begin{bmatrix}1 & 0\end{bmatrix}\) and \(\begin{bmatrix}0 & 1\end{bmatrix}\)?
- What is the span of the vectors \(\begin{bmatrix}1 & 0\end{bmatrix}\) and \(\begin{bmatrix}1 & 1\end{bmatrix}\)?
- What is the span of the vectors \(\begin{bmatrix}1 & 0 & 2\end{bmatrix}\), \(\begin{bmatrix}0 & 1 & 2\end{bmatrix}\)?
- For a set of vectors \(x^1, \ldots, x^k \in \mathbb{R}^n\), what is the highest dimension the span could be? The lowest?
- All of \(\mathbb{R}^2\)
- All of \(\mathbb{R}^2\). Note that \(x^3 \equiv x^2 - x^1 = \begin{bmatrix} 0 & 1 \end{bmatrix}\). Hence \(x^1\) and \(x^3\) can span anything as they are the canonical basis vectors.
- The set \(\left\{\alpha \begin{bmatrix} 1 & 0 & 2 \end{bmatrix}^\top + \beta \begin{bmatrix} 0 & 1 & 2 \end{bmatrix}^\top : \alpha, \beta \in \mathbb{R} \right\} = \left\{\begin{bmatrix} x_1 & x_2 & x_3 \end{bmatrix}^\top : x_3 = 2(x_1 + x_2)\right\}\), i.e. the plane \(2x_1 + 2x_2 - x_3 = 0\) through the origin.
- The highest dimension it could be is \(\min(k, n)\) (this requires the vectors to be linearly independent). The lowest it could be is \(0\), which happens if all \(x^i\) are the zero vector.
Question 7
Let \(A = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}\) and \(b = \begin{bmatrix} b_1 \\ b_2 \end{bmatrix}\).
Write the matrix-vector product \(A b\) as a linear combination of the columns of the matrix.
\[
A b = b_1 \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} + b_2 \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix}
\]
Question 8
Let \(A = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix} \equiv \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{bmatrix}\) and \(b = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}\).
- Write the matrix-vector product \(A b\) as a linear combination of the columns of \(A\).
- What is the span of the columns of \(A\)?
- Come up with 3 different variations on \(\begin{bmatrix} b_1 & b_2 & b_3 \end{bmatrix}^\top\) that solve \(A b = \begin{bmatrix} 2 & 3 & 0 \end{bmatrix}^\top\), or explain why it is not possible.
- Come up with 3 different variations on \(\begin{bmatrix} b_1 & b_2 & b_3 \end{bmatrix}^\top\) that solve \(A b = \begin{bmatrix} 2 & 3 & 1 \end{bmatrix}^\top\), or explain why it is not possible.
\(A b = b_1 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} + b_2 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} + b_3 \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}\)
The span of the columns of \(A\) is all vectors of the form \(\begin{bmatrix} x \\ y \\ 0 \end{bmatrix}\), i.e., the \(xy\)-plane in \(\mathbb{R}^3\). Note that the third column is a linear combination of the first two columns: \(\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} + \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}\), so the columns are not all linearly independent.
We need \(b_1 + b_3 = 2\) and \(b_2 + b_3 = 3\). Examples: \(b \equiv \begin{bmatrix} 2 & 3 & 0 \end{bmatrix}^\top\), \(b \equiv \begin{bmatrix} 1 & 2 & 1 \end{bmatrix}^\top\), \(b \equiv \begin{bmatrix} 0 & 1 & 2 \end{bmatrix}^\top\)
- Key here: there is more than one way to do it because of collinearity in the columns of \(A\)—will be important for solving systems of equations and understanding identification
- Note that the span of the columns of \(A\) is a plane in \(\mathbb{R}^3\)
None exist because that vector is not in the span of the columns of \(A\) (its third entry is nonzero, but every column of \(A\) has a zero third entry)
Question 9
Let \(A\) and \(B\) be invertible \(n \times n\) matrices. Show that \(B^{-1}A^{-1}\) is the inverse of \(AB\) — that is, \((AB)^{-1} = B^{-1}A^{-1}\). (Note the order reversal.)
Verify that \(B^{-1}A^{-1}\) undoes \(AB\) from both sides:
\[
(AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = A I A^{-1} = AA^{-1} = I,
\] \[
(B^{-1}A^{-1})(AB) = B^{-1}(A^{-1}A)B = B^{-1} I B = B^{-1}B = I.
\]
Since \(B^{-1}A^{-1}\) is both a left and a right inverse of the square matrix \(AB\), \(AB\) is invertible and \((AB)^{-1} = B^{-1}A^{-1}\). The factors reverse order: the matrix applied last is undone first. (We used associativity to regroup, and the assumed existence of \(A^{-1}\) and \(B^{-1}\).)
PhD Track
Building on the PhD-track sections of the Linear Algebra lecture: projection onto the column space, the structure of \(X'X\), and quadratic forms and definiteness.
Question 10
Let \(X\) be an \(n \times k\) matrix with full column rank, and define the projection matrix \(P = X(X'X)^{-1}X'\) and the annihilator \(M = I - P\).
- Show that \(P\) is idempotent (\(P^2 = P\)) and symmetric (\(P' = P\)).
- Show that \(MX = 0\), and interpret this geometrically.
- Show that \(PM = 0\), and conclude that the fitted vector \(Py\) and the residual vector \(My\) are orthogonal for every \(y\).
Full column rank makes \(X'X\) invertible, and \((X'X)^{-1}\) is symmetric because \(X'X\) is.
Idempotent: \[
P^2 = X(X'X)^{-1}\underbrace{X'X(X'X)^{-1}}_{I}X' = X(X'X)^{-1}X' = P.
\] Symmetric: using \((ABC)' = C'B'A'\) and \(\big((X'X)^{-1}\big)' = (X'X)^{-1}\), \[
P' = \big(X(X'X)^{-1}X'\big)' = X\big((X'X)^{-1}\big)'X' = X(X'X)^{-1}X' = P.
\]
\(MX = (I - P)X = X - X(X'X)^{-1}X'X = X - X = 0.\) Geometrically, \(M\) annihilates every column of \(X\), hence all of \(\text{col}(X)\): it strips off the part of a vector lying in the column space and keeps only the orthogonal (residual) component.
\(PM = P(I - P) = P - P^2 = P - P = 0.\) Therefore, for any \(y\), \[
(Py)'(My) = y'P'My = y'PMy = y'(PM)y = 0,
\] using \(P' = P\). The fitted values \(Py\) and residuals \(My\) are orthogonal — the geometric core of least squares.
Question 11
Let \(A\) be symmetric (\(A = A'\)) with eigenpairs \(Av_1 = \lambda_1 v_1\) and \(Av_2 = \lambda_2 v_2\), where \(\lambda_1 \neq \lambda_2\). Show that the eigenvectors are orthogonal: \(v_1' v_2 = 0\).
Evaluate the scalar \(v_1' A v_2\) two ways.
Directly, \(v_1' A v_2 = v_1'(\lambda_2 v_2) = \lambda_2\, v_1' v_2\).
Using symmetry, \(v_1' A = v_1' A' = (A v_1)' = (\lambda_1 v_1)' = \lambda_1 v_1'\), so \(v_1' A v_2 = \lambda_1\, v_1' v_2\).
Setting the two equal gives \((\lambda_1 - \lambda_2)\, v_1' v_2 = 0\). Since \(\lambda_1 \neq \lambda_2\), it must be that \(v_1' v_2 = 0\).
Question 12
Let \(P\) be a square matrix whose columns \(v_1, \ldots, v_n\) are orthonormal: \(v_i' v_j = 0\) for \(i \neq j\) and \(v_i' v_i = 1\). Show that \(P'P = I\), and conclude that \(P^{-1} = P'\).
The \((i,j)\) entry of \(P'P\) is the \(i\)-th row of \(P'\) (namely \(v_i'\)) times the \(j\)-th column of \(P\) (namely \(v_j\)): \[
(P'P)_{ij} = v_i' v_j = \begin{cases} 1 & i = j, \\ 0 & i \neq j, \end{cases}
\] which is exactly \(I\). So \(P'P = I\), i.e. \(P'\) is a left inverse of \(P\).
The orthonormal columns are linearly independent: if \(\sum_i \alpha_i v_i = 0\), taking the inner product with \(v_j\) gives \(\alpha_j = 0\). So the square matrix \(P\) has independent columns and is invertible. Right-multiplying \(P'P = I\) by \(P^{-1}\) yields \(P' = P^{-1}\).
Question 13
Let \(A\) be symmetric positive definite with spectral decomposition \(A = PDP'\), where \(P\) is orthogonal (\(P'P = I\)) and \(D = \mathrm{diag}(\lambda_1, \ldots, \lambda_n)\) with every \(\lambda_i > 0\). Define \(D^{1/2} = \mathrm{diag}(\sqrt{\lambda_1}, \ldots, \sqrt{\lambda_n})\) and \(K = D^{1/2}P'\). Show that \(K'K = A\), and explain why \(K\) is invertible — so every positive definite matrix can be written as \(K'K\) for an invertible \(K\).
Since \(D^{1/2}\) is diagonal, it is symmetric and \(D^{1/2}D^{1/2} = D\). Then \[
K'K = (D^{1/2}P')'(D^{1/2}P') = P\,(D^{1/2})'\,D^{1/2}\,P' = P\,D\,P' = A.
\]
\(K = D^{1/2}P'\) is a product of invertible matrices: \(D^{1/2}\) is invertible because each \(\sqrt{\lambda_i} > 0\) (inverse \(\mathrm{diag}(1/\sqrt{\lambda_i})\)), and \(P'\) is invertible because \(P\) is orthogonal (\((P')^{-1} = P\)). Hence \(K\) is invertible, and every PD matrix factors as \(A = K'K\). This is the matrix analog of writing a positive number as a square; the lower-triangular choice of such a factor is the Cholesky factor.
Question 14
Let \(A\) be symmetric with spectral decomposition \(A = PDP'\), \(P\) orthogonal, \(D = \mathrm{diag}(\lambda_1, \ldots, \lambda_n)\). Define new coordinates \(z = P'x\).
- Show that the quadratic form satisfies \(x'Ax = \sum_{i=1}^n \lambda_i z_i^2\).
- Use part (a) to prove that \(A\) is positive definite if and only if \(\lambda_i > 0\) for all \(i\).
Substitute \(A = PDP'\) and \(z = P'x\) (so \(z' = x'P\)): \[
x'Ax = x'PDP'x = (P'x)'D(P'x) = z'Dz = \sum_{i=1}^n \lambda_i z_i^2,
\] because \(D\) is diagonal.
The map \(x \mapsto z = P'x\) is a bijection (\(P\) invertible), with \(z = 0 \iff x = 0\); also \(\|z\|^2 = x'PP'x = x'x\), so \(x \neq 0 \iff z \neq 0\).
- (\(\Leftarrow\)) If every \(\lambda_i > 0\): for \(x \neq 0\) we have \(z \neq 0\), so some \(z_i \neq 0\) and \(x'Ax = \sum_i \lambda_i z_i^2 > 0\). Thus \(A\) is PD.
- (\(\Rightarrow\)) If some \(\lambda_j \leq 0\): take \(x = P e_j\) (the \(j\)-th eigenvector), giving \(z = e_j\), so \(x \neq 0\) but \(x'Ax = \lambda_j \leq 0\). Then \(A\) is not PD. Contrapositively, PD forces all \(\lambda_i > 0\).
Question 15
For any \(n \times k\) matrix \(X\), use the identity \(c'X'Xc = \|Xc\|^2\) to prove that \(X'X\) is positive definite if and only if \(X\) has full column rank.
- (\(\Rightarrow\)) Assuming \(X\) has full column rank, show \(c'X'Xc > 0\) for all \(c \neq 0\).
- (\(\Leftarrow\)) Assuming \(X\) is rank deficient, exhibit a specific \(c \neq 0\) with \(c'X'Xc = 0\).
- Combining with the eigenvalue characterization of definiteness, explain why \(X'X\) is invertible if and only if \(X\) has full column rank.
The identity \(c'X'Xc = (Xc)'(Xc) = \|Xc\|^2 \geq 0\) shows \(X'X\) is always positive semidefinite; strictness is the question.
Full column rank means the columns are linearly independent, i.e. \(Xc = 0 \Rightarrow c = 0\). So for \(c \neq 0\), \(Xc \neq 0\) and \(c'X'Xc = \|Xc\|^2 > 0\). Hence \(X'X\) is PD.
Rank deficiency means the columns are dependent: there is a \(c \neq 0\) with \(Xc = 0\). For it, \(c'X'Xc = \|Xc\|^2 = 0\) with \(c \neq 0\), so \(X'X\) is not PD (only PSD).
A symmetric matrix is PD iff all eigenvalues are positive, and \(\det(X'X) = \prod_i \lambda_i\), so PD \(\Rightarrow \det(X'X) \neq 0 \Rightarrow X'X\) invertible. Conversely, if \(X\) is rank deficient, the \(c \neq 0\) from (b) satisfies \(X'Xc = X'(Xc) = 0\), so \(X'X\) is singular. Therefore \(X'X\) invertible \(\iff X'X\) PD \(\iff X\) has full column rank.