MML - Practice for Exam 4

We will have our third exam this Friday! Here’s our in class practice sheet.

The problems

  1. Let \(A\) denote the matrix \[\begin{bmatrix} 2 & 3 & -5 \\ 0 & -7 & 11 \\ 0 & 0 & 13 \end{bmatrix}.\] Diagonalize \(A\).
    You may assume that the eigenvectors of \(A\) from most to least dominant are \[ \begin{bmatrix} -4 \\ 5 \\ 10 \end{bmatrix}, \: \: \begin{bmatrix} -2 \\ 5 \\ 0 \end{bmatrix}, \text{ and} \: \: \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}. \]

  2. Let \(A\) denote the matrix \[ A = \begin{bmatrix}5&-2\\6&-2\end{bmatrix}. \] Suppose that I diagonalize \(A\) and find that \[ A = \begin{bmatrix}1&2\\2&3\end{bmatrix} \begin{bmatrix}1&0\\0&2\end{bmatrix} \begin{bmatrix}-3&2\\2&-1\end{bmatrix}. \]

    1. Verify that this is, in fact, a diagonalization.
    2. Compute \(A^{10}\).

  3. Suppose that \(A\) is similar to \(B\). That is, there is a non-singular matrix \(S\) such that \[ A = SBS^{-1}. \] Find a matrix \(S\) that shows that \(B\) is similar to \(A\) and show that it works.

  1. Consider the data matrix \(X\) given by

    \(\mathbf{x}_1\) \(\mathbf{x}_2\)
    0 0
    1 1
    2 1
    1. Sketch the data together with a rough estimate of the first principal component.
    2. The principal components of \(X\) are the eigenvectors of what matrix?
    3. Which of the following vectors could potentially be the first principal component? \[ \begin{bmatrix}1\\1.6\end{bmatrix}, \: \begin{bmatrix}1.6\\1\end{bmatrix}, \: \text{ or } \: \begin{bmatrix}-0.6\\-0.2\end{bmatrix}. \]

  2. Let \(f(x,y,z) = xy^2 + y^2z.\)

    1. Draw an expression graph for \(f\). You should probably use operator nodes labeled “square”, “times”, and “plus”.
    2. Further label your expression graph to illustrate the computation \(f(1,2,3)\).

  3. The neural network shown in Figure 1 below consists of three layers:

    • the input layer,
    • one hidden layer, and
    • the output layer.

    Let’s also suppose that the input layer has a ReLU activation and the output layer has a sigmoid activation.

    Note that the inputs are given. Use those inputs together with forward propagation to compute the value produced by this neural network.

  4. Consider the neural network shown in Figure 2. Probabilities for the three possible outputs of red, yellow or blue from top to bottom are to be computed by applying the softmax to the values currently indicated.

    1. Which value red, yellow, or blue is most likely?
    2. What is the probability associated with that value?

Figures

A simple neural network image for computation
Figure 1: A neural network for forward propagation
A simple neural network image for softmax computation
Figure 2: A neural network for softmax computation