Linear Algebra — Practice Problems

Practice problems for the Linear Algebra lecture
Author

Jesse Perla

These practice questions accompany the Linear Algebra lecture and will be discussed in the review session.

Question 1

Compute the following matrix and vector products:

  1. \(\begin{bmatrix}3 & 1\\ 2 & 4 \end{bmatrix} \begin{bmatrix}2 & 1 & 0 \\ 1 & 5 & 2 \end{bmatrix}\)
  2. \(\begin{bmatrix}0 & 0 & 1 \\ 3 & 1 & 1 \\ -2 & 3 & 0 \end{bmatrix}\begin{bmatrix}2 & 0 & 5 \\ 1 & 3 & 1 \\ 4 & 1 & 2 \end{bmatrix}\)
  3. \(\begin{bmatrix}5 & 3 & 2 \end{bmatrix}\begin{bmatrix}2 & 7 & 1 \\ 0 & 4 & 5 \\ 2 & 3 & 4 \end{bmatrix}\)
  4. \(\begin{bmatrix}2 & 3 \\ 1 & -2 \end{bmatrix}\begin{bmatrix}2 \\ -1 \end{bmatrix}\)

Question 2

Being formal and explicit in the rules of matrix algebra (e.g. when things are commutative, distributive, when you require invertibility, etc.) solve the following equations for \(x \in \mathbb{R}^N\), with constants: vector \(b,c \in \mathbb{R}^N\), matrices \(A,D,Q,R\) all \(\mathbb{R}^{N\times N}\), and scalar \(m \in \mathbb{R}\),

  1. \(x A = b + c\)
  2. \(Ax + \left(x^{\top} D\right)^{\top} = b\)
  3. \(Q^{-1} \left( A\, x\, m + m b\right) = Rx\)

Question 3

Write the following as a system of equations in matrix form:

  1. \(\left\{\begin{matrix}2x + 3y = 2 \\ x - 2y = -1\end{matrix}\right.\) where \(\{x,y\}\) are variables
  2. \(\left\{\begin{matrix}2x - 3y = 1 \\ 3x + my = -2\end{matrix}\right.\) where \(\{x,y\}\) are variables
  3. \(\left\{\begin{matrix}2a + b - 1 = 0 \\ 3b + 4c = 2 \\ -2a + c = 0\end{matrix}\right.\) where \(\{a,b,c\}\) are variables
  4. \(\left\{\begin{matrix}a + b = -3 \\ c - y - 4 b = 0\end{matrix}\right.\) where \(\{a,b,c \}\) are variables

Question 4

  1. Find \(G \in \mathbb{R}^2\) such that \(G \cdot \begin{bmatrix}a & b\end{bmatrix}\) returns the second element, \(b\).
  2. Find \(H \in \mathbb{R}^{1 \times 2}\) such that \(H \begin{bmatrix}x & y\end{bmatrix}^\top\) returns the sum \(x + y\).
  3. Find a \(2 \times 2\) matrix \(M\) such that \(M \begin{bmatrix}p & q\end{bmatrix}^\top = \begin{bmatrix}q & p\end{bmatrix}^\top\) (i.e., swaps the two elements).
  4. Let \(T\) be a linear transformation from \(\mathbb{R}^2\) to \(\mathbb{R}^2\) that doubles the first coordinate and leaves the second unchanged. Write a matrix representation of \(T\).

Question 5

  1. Find a vector \(x \in \mathbb{R}^2\) such that \(x \cdot \begin{bmatrix}1 & 2\end{bmatrix} = 0\)
  2. Find a vector \(x \in \mathbb{R}^2\) such that \(x \cdot \begin{bmatrix}0 & 1\end{bmatrix} = 1\)
  3. Find 3 vectors of unit length in \(\mathbb{R}^3\) which are mutually orthogonal
  4. Given vectors \(x^1\) and \(x^2\) which may have a different norm (i.e., length), how can I use the norm and inner product to test if they are orthogonal? collinear?

Question 6

  1. What is the span of the vectors \(\begin{bmatrix}1 & 0\end{bmatrix}\) and \(\begin{bmatrix}0 & 1\end{bmatrix}\)?
  2. What is the span of the vectors \(\begin{bmatrix}1 & 0\end{bmatrix}\) and \(\begin{bmatrix}1 & 1\end{bmatrix}\)?
  3. What is the span of the vectors \(\begin{bmatrix}1 & 0 & 2\end{bmatrix}\), \(\begin{bmatrix}0 & 1 & 2\end{bmatrix}\)?
  4. For a set of vectors \(x^1, \ldots, x^k \in \mathbb{R}^n\), what is the highest dimension the span could be? The lowest?

Question 7

Let \(A = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}\) and \(b = \begin{bmatrix} b_1 \\ b_2 \end{bmatrix}\).

Write the matrix-vector product \(A b\) as a linear combination of the columns of the matrix.

Question 8

Let \(A = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix} \equiv \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{bmatrix}\) and \(b = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}\).

  1. Write the matrix-vector product \(A b\) as a linear combination of the columns of \(A\).
  2. What is the span of the columns of \(A\)?
  3. Come up with 3 different variations on \(\begin{bmatrix} b_1 & b_2 & b_3 \end{bmatrix}^\top\) that solve \(A b = \begin{bmatrix} 2 & 3 & 0 \end{bmatrix}^\top\), or explain why it is not possible.
  4. Come up with 3 different variations on \(\begin{bmatrix} b_1 & b_2 & b_3 \end{bmatrix}^\top\) that solve \(A b = \begin{bmatrix} 2 & 3 & 1 \end{bmatrix}^\top\), or explain why it is not possible.

Question 9

Let \(A\) and \(B\) be invertible \(n \times n\) matrices. Show that \(B^{-1}A^{-1}\) is the inverse of \(AB\) — that is, \((AB)^{-1} = B^{-1}A^{-1}\). (Note the order reversal.)

PhD Track

Building on the PhD-track sections of the Linear Algebra lecture: projection onto the column space, the structure of \(X'X\), and quadratic forms and definiteness.

ImportantPhD Track

Question 10

Let \(X\) be an \(n \times k\) matrix with full column rank, and define the projection matrix \(P = X(X'X)^{-1}X'\) and the annihilator \(M = I - P\).

  1. Show that \(P\) is idempotent (\(P^2 = P\)) and symmetric (\(P' = P\)).
  2. Show that \(MX = 0\), and interpret this geometrically.
  3. Show that \(PM = 0\), and conclude that the fitted vector \(Py\) and the residual vector \(My\) are orthogonal for every \(y\).
ImportantPhD Track

Question 11

Let \(A\) be symmetric (\(A = A'\)) with eigenpairs \(Av_1 = \lambda_1 v_1\) and \(Av_2 = \lambda_2 v_2\), where \(\lambda_1 \neq \lambda_2\). Show that the eigenvectors are orthogonal: \(v_1' v_2 = 0\).

ImportantPhD Track

Question 12

Let \(P\) be a square matrix whose columns \(v_1, \ldots, v_n\) are orthonormal: \(v_i' v_j = 0\) for \(i \neq j\) and \(v_i' v_i = 1\). Show that \(P'P = I\), and conclude that \(P^{-1} = P'\).

ImportantPhD Track

Question 13

Let \(A\) be symmetric positive definite with spectral decomposition \(A = PDP'\), where \(P\) is orthogonal (\(P'P = I\)) and \(D = \mathrm{diag}(\lambda_1, \ldots, \lambda_n)\) with every \(\lambda_i > 0\). Define \(D^{1/2} = \mathrm{diag}(\sqrt{\lambda_1}, \ldots, \sqrt{\lambda_n})\) and \(K = D^{1/2}P'\). Show that \(K'K = A\), and explain why \(K\) is invertible — so every positive definite matrix can be written as \(K'K\) for an invertible \(K\).

ImportantPhD Track

Question 14

Let \(A\) be symmetric with spectral decomposition \(A = PDP'\), \(P\) orthogonal, \(D = \mathrm{diag}(\lambda_1, \ldots, \lambda_n)\). Define new coordinates \(z = P'x\).

  1. Show that the quadratic form satisfies \(x'Ax = \sum_{i=1}^n \lambda_i z_i^2\).
  2. Use part (a) to prove that \(A\) is positive definite if and only if \(\lambda_i > 0\) for all \(i\).
ImportantPhD Track

Question 15

For any \(n \times k\) matrix \(X\), use the identity \(c'X'Xc = \|Xc\|^2\) to prove that \(X'X\) is positive definite if and only if \(X\) has full column rank.

  1. (\(\Rightarrow\)) Assuming \(X\) has full column rank, show \(c'X'Xc > 0\) for all \(c \neq 0\).
  2. (\(\Leftarrow\)) Assuming \(X\) is rank deficient, exhibit a specific \(c \neq 0\) with \(c'X'Xc = 0\).
  3. Combining with the eigenvalue characterization of definiteness, explain why \(X'X\) is invertible if and only if \(X\) has full column rank.