Matrix Inverses
Wed, Feb 05, 2025
Today we’re going to discuss matrix inverses and some related concepts, like determinants.
\[ \newcommand{\nsp}[1]{\mathcal{N}\!\left(#1\right)} \newcommand{\nullity}[1]{n\left(#1\right)} \newcommand{\detname}[1]{\det\left(#1\right)} \newcommand{\submatrix}[3]{#1\left(#2|#3\right)} \newcommand{\matrixentry}[2]{\left\lbrack#1\right\rbrack_{#2}} \newcommand{\detbars}[1]{\left\lvert#1\right\rvert} \]
Nonsingular matrices were introduced a few presentations ago. Honestly, I added that after the fact and, even then, didn’t give it enough importance.
Let’s take a good look at it now.
Def: Given a natural number \(n\), a square matrix of size \(n\) is simply an \(n\times n\) matrix.
Def: Suppose \(A\) is a square matrix. Suppose further that the solution set to the homogeneous linear system of equations \(A\vec{x}=\vec{0}\) is \(\vec{0}\), in other words, the system has only the trivial solution. Then we say that \(A\) is a nonsingular matrix. Otherwise we say \(A\) is a singular matrix.
There are quite a few more ways to characterize this, though:
Suppose that \(A\) is a square matrix of size \(n\). The following are equivalent.
We know or should know most of the terms on the previous slide. A couple of exceptions are invertible and determinant, which we’ll be talking about today.
So, let’s talk about matrix inversion, see how it’s related to what we’ve done so far, and how we’ll use it moving forward.
If \(A\) and \(B\) are both square matrices of the same size, then \(AB\) and \(BA\) are both defined, though not necessarily equal.
You can even multiply a square matrix by itself to get \(A^2\), then \(A^3\), etc…
The square matrix of size \(n\) with ones on the diagonal and zeros off the diagonal is called the identity matrix and denoted with an \(I\). It’s literally the multiplicative identity since
\[ \begin{bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{bmatrix} \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{bmatrix} = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{bmatrix}. \]
Or, more compactly \(IA = A = AI\).
Given a square matrix \(A\), it’s feasible that there could be another square matrix \(B\) of the same size and with the property that \(AB =I\).
Such a matrix, if it exists, is called the inverse of \(A\). We use the definite article here because the inverse is unique; we typically denote the inverse as \(A^{-1}\).
Note: Not every square matrix has an inverse!!
If, \(A\) has an inverse, then \(A\) is called invertible.
Given a square matrix \(A\) and a purported inverse \(A^{-1}\), it’s pretty easy to check if \(A^{-1}\) is, indeed, an inverse. Simply multiply!
\[ \left( \begin{array}{cc} 2 & -1 \\ -3 & 2 \\ \end{array} \right) \left( \begin{array}{cc} 2 & 1 \\ 3 & 2 \\ \end{array} \right) = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \\ \end{array} \right) =\left( \begin{array}{cc} 2 & 1 \\ 3 & 2 \\ \end{array} \right) \left( \begin{array}{cc} 2 & -1 \\ -3 & 2 \\ \end{array} \right) \]
Note that a matrix generally commutes with its inverse.
Suppose that \(A\) is an invertible, square matrix of size \(n\). We’d like a way to find the columns of \(A^{-1}\).
Let \(\vec{e}_j\) be the \(j^{\text{th}}\) standard coordinate vector of \(\mathbb R^n\). Thus \(\vec{e}_j\) is all zeros, except for a \(1\) in its \(i^{\text{th}}\) spot.
Now, suppose that \(\vec{x}\) solves the matrix equation \[A\vec{x}=\vec{e}_j.\] I guess that means that \[A^{-1}\vec{e}_j=\vec{x}.\] But, \(A^{-1}\vec{e}_j\) tells us exactly the \(j^{\text{th}}\) column of \(A^{-1}\).
Thus, we can find the columns of \(A^{-1}\) by solving \(A\vec{x}=\vec{e}_j\).
Now, recall that we can solve \[A\vec{x}=\vec{e}_j\] by row reducing the augmented matrix \([A|\vec{e_j}]\).
There will be a unique solution precisely when the \(A\) row reduces to \(I\), in which case the solution will be in the last column of the reduced, augmented matrix. Thus, we’ve got a way to find the individual columns of \(A^{-1}\) by row reduction.
For that matter, we could start with the multiply augmented matrix \[[A|I]\] and row reduce that.
Form the augmented matrix \[[A|I]\] and row reduce it. There are two possibilities.
We have, for example, the row reduction \[ \left( \begin{array}{ccc} 1 & 0 & 1 & 1 & 0 & 0 \\ 1 & -1 & 1 & 0 & 1 & 0 \\ 2 & 1 & 1 & 0 & 0 & 1\\ \end{array} \right) \longrightarrow \left( \begin{array}{ccc} 1 & 0 & 0 & -2 & 1 & 1 \\ 0 & 1 & 0 & 1 & -1 & 0 \\ 0 & 0 & 1 & 3 & -1 & -1 \\ \end{array} \right) \] and it’s pretty easy to see that \[ \left( \begin{array}{ccc} -2 & 1 & 1 \\ 1 & -1 & 0 \\ 3 & -1 & -1 \\ \end{array} \right) \left( \begin{array}{ccc} 1 & 0 & 1 \\ 1 & -1 & 1 \\ 2 & 1 & 1 \\ \end{array} \right) = \left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right). \]
For very small matrices, we can use symbolic expressions and, potentially, find formulae for the inverses. Here’s the row reduction to find the general formula for the inverse of a \(2\times2\) matrix, for example:
\[ \left( \begin{array}{cccc} a & b & 1 & 0 \\ c & d & 0 & 1 \\ \end{array} \right) \longrightarrow \left( \begin{array}{cccc} 1 & 0 & \frac{d}{a d-b c} & \frac{b}{b c-a d} \\ 0 & 1 & \frac{c}{b c-a d} & \frac{a}{a d-b c} \\ \end{array} \right). \]
It’s easy to check via matrix multiplication that the formula works. In the case where \(a d-b c = 0\), the matrix is not invertible.
You can also do this for a \(3\times 3\) matrix, or even more. It gets less useful pretty quick, though.
The expression \(ad-bc\) that appears in our formula for the inverse of \(2\times2\) matrix is called the determinant of the \(2\times2\) matrix. These can be defined in useful and consistent way for matrices of any dimension.
Determinants are defined in a recursive fashion. Once we’ve defined the notion of determinant for matrices of size \(n-1\), we extend it to matrices of size \(n\) using the concept of submatrices.
Def: Suppose that \(A\) is an \(n\times n\) matrix. Then the submatrix \(\submatrix{A}{i}{j}\) is the \((n-1)\times (n-1)\) matrix obtained from \(A\) by removing row \(i\) and column \(j\).
Given the following matrix
\[\begin{equation*} A= \begin{bmatrix} 1 & -2 & 3 & 9\\ 4 & -2 & 0 & 1\\ 3 & 5 & 2 & 1 \end{bmatrix}, \end{equation*}\]
we have the following submatrices:
\[\begin{align*} \submatrix{A}{2}{3} = \begin{bmatrix} 1 & -2 & 9\\ 3 & 5 & 1 \end{bmatrix}&& \submatrix{A}{3}{1} = \begin{bmatrix} -2 & 3 & 9\\ -2 & 0 & 1 \end{bmatrix}\text{.} \end{align*}\]
Suppose \(A\) is a square matrix. Then its determinant, \(\det(A)=|A|\), is the real number defined recursively by:
\[\begin{align*} \detname{A}&= \matrixentry{A}{11}\detname{\submatrix{A}{1}{1}} -\matrixentry{A}{12}\detname{\submatrix{A}{1}{2}} +\matrixentry{A}{13}\detname{\submatrix{A}{1}{3}}-\\ &\quad \matrixentry{A}{14}\detname{\submatrix{A}{1}{4}} +\cdots +(-1)^{n+1}\matrixentry{A}{1n}\detname{\submatrix{A}{1}{n}}\text{.} \end{align*}\]
\[ \begin{align*} \detname{A}=\detbars{A} &=\begin{vmatrix} 3 & 2 & -1\\ 4 & 1 & 6\\ -3 & -1 & 2 \end{vmatrix}\\ &= 3 \begin{vmatrix} 1 & 6\\ -1 & 2 \end{vmatrix} -2 \begin{vmatrix} 4 & 6\\ -3 & 2 \end{vmatrix} +(-1) \begin{vmatrix} 4 & 1\\ -3 & -1 \end{vmatrix}\\ &= 3\left( 1\begin{vmatrix} 2\\ \end{vmatrix} -6\begin{vmatrix} -1 \end{vmatrix}\right) -2\left( 4\begin{vmatrix} 2 \end{vmatrix} -6\begin{vmatrix} -3 \end{vmatrix}\right) -\left( 4\begin{vmatrix} -1 \end{vmatrix} -1\begin{vmatrix} -3 \end{vmatrix}\right)\\ &= 3\left(1(2)-6(-1)\right) -2\left(4(2)-6(-3)\right) -\left(4(-1)-1(-3)\right)\\ &=24-52+1\\ &=-27\text{.} \end{align*} \]
Ultimately, the primary utility of inverse matrices and determinants is their theoretical framework to help us understand linear transformations. They are simply not used in well-written numerical software.
Would love to talk about how determinants measure \(n\)-dimensional volume. Maybe next time?