Matrix start here
Matrix definitions|terms
Matrix determinant
Matrix inverse
Matrix operations
The inverse of a matrix
In any mathematical system that can be used to represent and solve real problems, it's a great advantage to have a multiplicative inverse. For the set of rational numbers, $\{\mathbb{Q}\},$ the multiplicative inverse is simply the reciprocal:
$$x \cdot \frac{1}{x} = \frac{1}{x} \cdot x = 1.$$
Multiplying an element of our set by its inverse yields the identity element, 1. We'd like to have such an inverse for square $(n \times n)$ matrices. For matrix $A,$, we'll call the inverse $A^{-1}.$ Then we have
$$A \cdot A^{-1} = A^{-1}A = I,$$
where $I$ is the identity matrix, the matrix with 1's along the diagonal, zeros elsewhere. It's also true that $A \cdot I = A$ and $A^{-1} \cdot I = A^{-1}.$
One use of the inverse
One important case where the inverse matrix can be very helpful is in solving systems of equations. If we represent a system as
$$A \cdot \vec{x} = \vec{s},$$
where A is the matrix of coefficients, the vector $\vec{x} = (x, y, z)$ or $(x_1, x_2, x_3, ... x_n)$ is the n-dimensional vector of variables, and $\vec{s}$ is the vector of solutions of the same dimension. When we solve such a system, it's $\vec{x}$ that we seek.
Now if we have the inverse of A, then we can also find $\vec{x}$ this way:
$$ \begin{align} A \cdot \vec{x} &= \vec{s} \\[5pt] A^{-1}\cdot A \vec{x} &= A^{-1}\cdot \vec{s} \\[5pt] \vec{x} &= A^{-1}\cdot \vec{s} \end{align}$$
Let's see how to calculate the inverse of a 3 × 3 matrix, and then apply one in this way to a 3-equation, 3-unknown problem.
Calculating the inverse of a matrix
We'll begin with a simple 3 × 3 example.
$$Let \; A = \left( \begin{matrix} 1 && 1 && -1 \\ 1 && -1 && 0 \\ 1 && 2 && 1 \end{matrix} \right)$$
To find the inverse of this matrix, we'll first construct an augmented matrix, in which we place our 3 × 3 matrix, $A,$ side-by-side with a 3 × 3 identity matrix, like this:
$$\left( \begin{array}{ccc|ccc} 1 & 1 & -1 & 1 & 0 & 0\\ 1 & -1 & 0 & 0 & 1 & 0\\ 1 & 2 & 1 & 0 & 0 & 1 \end{array} \right)$$
Now the idea is to perform elementary row operations (i.e. Gauss-Jordan elimination) on the left-side 3 × 3 in order to transform it into the identity matrix. The right side will "come along for the ride" through these operations, resulting in the identity matrix in the end. You'll see how it works. Let's begin. First we'll focus on zeroing the lower two 1's of the left-most column vector: $R2 - R1, \; R3 - R1$:
$$\left( \begin{array}{ccc|ccc} 1 & 1 & -1 & 1 & 0 & 0\\ 1 & -1 & 0 & 0 & 1 & 0\\ 1 & 2 & 1 & 0 & 0 & 1 \end{array} \right) \longrightarrow \left( \begin{array}{ccc|ccc} 1 & 1 & -1 & 1 & 0 & 0\\ 0 & -2 & 1 & -1 & 1 & 0\\ 0 & 2 & 1 & -1 & 0 & 1 \end{array} \right) $$
Now we'll keep moving forward. There are a number of choices you could make; I'll take this path: $R3 + \frac{1}{2}R2$ (left matrix) then $R2/(-2) \; \text{and} \; R3 \times \frac{2}{5}$ (right matrix):
$$\left( \begin{array}{ccc|ccc} 1 & 1 & -1 & 1 & 0 & 0\\ 1 & -2 & 1 & -1 & 1 & 0\\ 0 & 0 & \frac{5}{2} & -\frac{3}{2} & \frac{1}{2} & 1 \end{array} \right) \longrightarrow \left( \begin{array}{ccc|ccc} 1 & 1 & -1 & 1 & 0 & 0\\ 0 & 1 & -\frac{1}{2} & \frac{1}{2} & -\frac{1}{2} & 0\\ 0 & 0 & 1 & -\frac{3}{5} & \frac{1}{5} & \frac{2}{5} \end{array} \right) $$
Now do two more operations, $R2 + \frac{1}{2}R3$ and $R1 + R3$:
$$ \rightarrow \left( \begin{array}{ccc|ccc} 1 & 1 & -1 & 1 & 0 & 0\\ 0 & 1 & 0 & \frac{1}{5} & -\frac{2}{5} & \frac{1}{5}\\ 0 & 0 & 1 & -\frac{3}{5} & \frac{1}{5} & \frac{2}{5} \end{array} \right) \longrightarrow \left( \begin{array}{ccc|ccc} 1 & 1 & 0 & \frac{2}{5} & \frac{1}{5} & \frac{2}{5} \\ 0 & 1 & 0 & \frac{1}{5} & -\frac{2}{5} & \frac{1}{5} \\ 0 & 0 & 1 & -\frac{3}{5} & \frac{1}{5} & \frac{2}{5} \end{array} \right)$$
We can finish by simply subtracting row 2 from row 1:
$$ \rightarrow \left( \begin{array}{ccc|ccc} 1 & 0 & 0 & \frac{1}{5} & \frac{3}{5} & \frac{1}{5} \\ 0 & 1 & 0 & \frac{1}{5} & -\frac{2}{5} & \frac{1}{5}\\ 0 & 0 & 1 & -\frac{3}{5} & \frac{1}{5} & \frac{2}{5} \end{array} \right)$$
So our inverse matrix, $A^{-1},$ is
$$A^{-1} = \left( \begin{matrix} \frac{1}{5} && \frac{3}{5} && \frac{1}{5} \\ \frac{1}{5} && -\frac{2}{5} && \frac{1}{5} \\ -\frac{3}{5} && \frac{1}{5} && \frac{2}{5} \end{matrix} \right)$$
Now we can show that $A \cdot A^{-1} = \bf I$ by multiplying these matrices.
$$\left( \begin{array}{ccc} 1 & 1 & -1 \\ 1 & -1 & 0 \\ 1 & 2 & 1 \end{array} \right) \left( \begin{array}{ccc} \frac{1}{5} & \frac{3}{5} & \frac{1}{5} \\ \frac{1}{5} & -\frac{2}{5} & \frac{1}{5} \\ -\frac{3}{5} & \frac{1}{5} & \frac{2}{5} \end{array} \right) = \left( \begin{array}{ccc} \frac{1}{5} + \frac{1}{5} + \frac{3}{5} & \frac{3}{5} - \frac{2}{5} - \frac{1}{5} & \frac{1}{5} + \frac{1}{5} - \frac{2}{5} \\ \frac{1}{5} - \frac{1}{5} + 0 & \frac{3}{5} + \frac{2}{5} + 0 & \frac{1}{5} - \frac{1}{5} + 0 \\ \frac{1}{5} + \frac{2}{5} -\frac{3}{5} & \frac{3}{5} - \frac{4}{5} + \frac{1}{5} & \frac{1}{5} + \frac{2}{5} + \frac{2}{5} \end{array} \right) $$
The matrix on the right is the identity matrix, so we did indeed find the inverse of matrix $A.$ You can check for yourself that the result is commutative: $A^{-1} \cdot A = \bf I.$ (But remember that not all matrix multiplication is commutative).
Is every matrix invertible?
No. Not every n × n matrix can be inverted like we did above. Let's look at an example and try to figure out why.
$$Let \; A = \left( \begin{matrix} 1 && 1 && -1 \\ 1 && -1 && 1 \\ 2 && 2 && -2 \end{matrix} \right)$$
Our augmented matrix looks like this:
$$\left( \begin{array}{ccc|ccc} 1 & 1 & -1 & 1 & 0 & 0\\ 1 & -1 & 1 & 0 & 1 & 0\\ 2 & 2 & -2 & 0 & 0 & 1 \end{array} \right)$$
The first manipulations we might make are $R2 - R1$ and $R3 - R1$. Here's the resulting augmented matrix:
$$\left( \begin{array}{ccc|ccc} 1 & 1 & -1 & 1 & 0 & 0\\ 0 & -2 & 2 & -1 & 1 & 0\\ 0 & 0 & 0 & -2 & 0 & 1 \end{array} \right)$$
Notice that our quest for an inverse matrix completely breaks down here because we've zeroed out the last row of the original matrix. The matrix has no inverse. Why?
Linear independence
The rows of a matrix are said to be linearly independent if any one row cannot be obtained through elementary operations (multiplication by a constant, addition & subtraction) on the remaining row(s). Notice that the third row of our matrix in this example is just twice the first row. The third row is dependent on the first, thus this set of equations contains only two pieces of information, insufficient for determining three unknowns.
One way to tell if a matrix represents a solvable system and is invertible is to calculate the determinant. A zero determinant means that a matrix is non linearly independent, and that the system of equations it represents is not solvable. Here's the determinant of our example matrix:
$$det \left( \begin{matrix} 1 && 1 && -1 \\ 1 && -1 && 1 \\ 2 && 2 && -2 \end{matrix} \right)$$
$$ \begin{align} &= 2 + 2 - 2 - (2 + 2 - 2) \\ &= 2 - 2 = 0. \end{align}$$
A matrix of coefficients is linearly independent if no row of coefficients can be obtained by applying elementary operations on any other row(s) of the matrix.
A matrix that has a zero determinant
has no solution as a system of equations, and
has no inverse.
A. Determine (← see what I did there?) whether the following matrices have an inverse.
$$\left( \begin{matrix} 1 & -1 \\ -1 & 1 \end{matrix} \right)$$
$$\left( \begin{matrix} 1 & 1 \\ -1 & 1 \end{matrix} \right)$$
$$\left( \begin{matrix} 1 & -1 & 1 \\ -1 & 1 & 2 \\ -1 & -1 & 2 \end{matrix} \right)$$
$$\left( \begin{matrix} 1 & -1 & 2 \\ 2 & -2 & 1 \\ -2 & 2 & -4 \end{matrix} \right)$$
B. Calculate the inverses of these matrices, and show that $A \cdot A^{-1} = I.$
$$\left( \begin{matrix} 1 & -1 \\ 1 & 1 \end{matrix} \right)$$
$$\left( \begin{matrix} 1 & 2 \\ 3 & 5 \end{matrix} \right)$$
$$\left( \begin{matrix} -7 & 1 \\ 2 & 7 \end{matrix} \right)$$
$$\left( \begin{matrix} 1 & 1 & 4 \\ 2 & 1 & 1 \\ 1 & 2 & 1 \end{matrix} \right)$$
$$det \left( \begin{matrix} 1 & -1 \\ -1 & 1 \end{matrix} \right) = 1(1) - (-1)(-1) = 0$$
The matrix has no inverse because it has a zero determinant.
$$det \left( \begin{matrix} 1 & 1 \\ -1 & 1 \end{matrix} \right) = 1(1) - (-1)(1) = 2$$
The matrix has an inverse because it has a non-zero determinant.
$$det \left( \begin{matrix} 1 & -1 & 1 \\ -1 & 1 & 2 \\ -1 & -1 & 2 \end{matrix} \right)$$
$$ \begin{align} &= 1(1)(2) + -1(2)(-1) + 1(-1)(-1) \\[5pt] &\phantom{0000} - [-1(1)(1) + -1(2)(1) + 2(-1)(-1)] \\[5pt] &= 2 + 2 + 1 - [-1 - 2 + 2] \\[5pt] &= 5 - [-1] = 6 \ne 0 \end{align}$$
$$det \left( \begin{matrix} 1 & -1 & 2 \\ 2 & -2 & 1 \\ -2 & 2 & -4 \end{matrix} \right)$$
$$ \begin{align} &= 1(-2)(-4) + -1(1)(-2) + 2(2)(2) \\[5pt] &\phantom{0000} - [-2(-2)(2) + 2(1)(1) + -4(2)(-1)] \\[5pt] &= 8 + 2 + 8 - [8 + 2 + 8] \\[5pt] &= 18 - 18 = 0 \end{align}$$
$$\left( \begin{array}{cc|cc} 1 & -1 & 1 & 0 \\ 1 & 1 & 0 & 1 \end{array} \right)$$
R2 - R1 (subtrac the first row from the second, and leave the first alone):
$$\left( \begin{array}{cc|cc} 1 & -1 & 1 & 0 \\ 0 & 2 & -1 & 1 \end{array} \right)$$
Divide the second row by 2:
$$\left( \begin{array}{cc|cc} 1 & -1 & 1 & 0 \\ 0 & 1 & -\frac{1}{2} & \frac{1}{2} \end{array} \right)$$
R1 + R2 (add row 2 to row 1 and leave row 2 the same.
$$\left( \begin{array}{cc|cc} 1 & 0 & \frac{1}{2} & \frac{1}{2} \\ 0 & 1 & -\frac{1}{2} & \frac{1}{2} \end{array} \right)$$
Our inverse is
$$\left( \begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ -\frac{1}{2} & \frac{1}{2} \end{array} \right)$$
$$A^{-1} \cdot A = \left( \begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ -\frac{1}{2} & \frac{1}{2} \end{array} \right) \left( \begin{array}{cc} 1 & -1 \\ 1 & 1 \end{array} \right)$$
Check it:
$$ = \left( \begin{array}{cc} \frac{1}{2} + \frac{1}{2} & -\frac{1}{2} + \frac{1}{2} \\ -\frac{1}{2} + \frac{1}{2} & \frac{1}{2} + \frac{1}{2} \end{array} \right) = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right)$$
$$\left( \begin{array}{cc|cc} 1 & 2 & 1 & 0 \\ 3 & 5 & 0 & 1 \end{array} \right)$$
R2 - 3R1 (subtract three times the first row from the second, and leave the first alone):
$$\left( \begin{array}{cc|cc} 1 & 2 & 1 & 0 \\ 0 & -1 & -3 & 1 \end{array} \right)$$
R1 + 2R2, then R2 × -1
$$\left( \begin{array}{cc|cc} 1 & 0 & -5 & 2 \\ 0 & 1 & 3 & -1 \end{array} \right)$$
$$A^{-1} \cdot A = \left( \begin{array}{cc} -5 & 2 \\ 3 & -1 \end{array} \right) \left( \begin{array}{cc} 1 & 2 \\ 3 & 5 \end{array} \right)$$
$$ = \left( \begin{array}{cc} -5 + 6 & -10 + 10 \\ 3 - 3 & 6 - 6 \end{array} \right) = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right)$$
$$\left( \begin{array}{cc|cc} -7 & 1 & 1 & 0 \\ 2 & 7 & 0 & 1 \end{array} \right)$$
Divide rows 1 & 2 by 7:
$$\left( \begin{array}{cc|cc} -1 & \frac{1}{7} & \frac{1}{7} & 0 \\ \frac{2}{7} & 1 & 0 & \frac{1}{7} \end{array} \right)$$
$R2 + \frac{2}{7}R1,$
$$\left( \begin{array}{cc|cc} -1 & \frac{1}{7} & \frac{1}{7} & 0 \\ 0 & \frac{51}{49} & \frac{2}{49} & \frac{1}{7} \end{array} \right)$$
$R2 \times \frac{49}{51}$
$$\left( \begin{array}{cc|cc} -1 & \frac{1}{7} & \frac{1}{7} & 0 \\ 0 & 1 & \frac{2}{51} & \frac{7}{51} \end{array} \right)$$
$R1 - \frac{1}{7}R2$ then $-1 \times R1$
$$\left( \begin{array}{cc|cc} 1 & 0 & -\frac{7}{51} & \frac{1}{51} \\ 0 & 1 & \frac{2}{51} & \frac{7}{51} \end{array} \right)$$
Now check the inverse
$$\left( \begin{array}{cc} -\frac{7}{51} & \frac{1}{51} \\ \frac{2}{51} & \frac{7}{51} \end{array} \right) \left( \begin{array}{cc} -7 & 1 \\ 2 & 7 \end{array} \right)$$
$$ = \left( \begin{array}{cc} \frac{49}{51} + \frac{2}{51} & \frac{-7}{51} + \frac{7}{51} \\ \frac{-14}{51} + \frac{14}{51} & \frac{2}{51} + \frac{49}{51} \end{array} \right) = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right)$$
$$\left( \begin{array}{ccc|ccc} 1 & 1 & 4 & 1 & 0 & 0 \\ 2 & 1 & 1 & 0 & 1 & 0 \\ 1 & 2 & 1 & 0 & 0 & 1 \end{array} \right)$$
R3 - R1 and R1 - 2R1:
$$\left( \begin{array}{ccc|ccc} 1 & 1 & 4 & 1 & 0 & 0 \\ 0 & -1 & -7 & -2 & 1 & 0 \\ 0 & 1 & -3 & -1 & 0 & 1 \end{array} \right)$$
R3 + R2:
$$\left( \begin{array}{ccc|ccc} 1 & 1 & 4 & 1 & 0 & 0 \\ 0 & -1 & -7 & -2 & 1 & 0 \\ 0 & 0 & -10 & -3 & 1 & 1 \end{array} \right)$$
R1 + R2
$$\left( \begin{array}{ccc|ccc} 1 & 0 & -3 & -1 & 1 & 0 \\ 0 & -1 & -7 & -2 & 1 & 0 \\ 0 & 0 & -10 & -3 & 1 & 1 \end{array} \right)$$
R3/(-10)
$$\left( \begin{array}{ccc|ccc} 1 & 0 & -3 & -1 & 1 & 0 \\ 0 & -1 & -7 & -2 & 1 & 0 \\ 0 & 0 & 1 & \frac{3}{10} & \frac{-1}{10} & \frac{-1}{10} \end{array} \right)$$
R1 + 3R3 and R2 + 7R3, then -1×R2
$$\left( \begin{array}{ccc|ccc} 1 & 0 & 0 & \frac{-1}{10} & \frac{7}{10} & \frac{-3}{10} \\ 0 & 1 & 0 & \frac{-1}{10} & \frac{-3}{10} & \frac{7}{10} \\ 0 & 0 & 1 & \frac{3}{10} & \frac{-1}{10} & \frac{-1}{10} \end{array} \right)$$
$$\left( \begin{array}{ccc} \frac{-1}{10} & \frac{7}{10} & \frac{-3}{10} \\ \frac{-1}{10} & \frac{-3}{10} & \frac{7}{10} \\ \frac{3}{10} & \frac{-1}{10} & \frac{-1}{10} \end{array} \right) \left( \begin{array}{ccc} 1 & 1 & 4 \\ 2 & 1 & 1 \\ 1 & 2 & 1 \end{array} \right)$$
$$=\left( \begin{array}{ccc} \frac{-1}{10} + \frac{14}{10} + \frac{-3}{10} & \frac{-1}{10} + \frac{7}{10} + \frac{-6}{10} & \frac{-4}{10} + \frac{7}{10} + \frac{-3}{10} \\ \frac{-1}{10} + \frac{-6}{10} + \frac{7}{10} & \frac{-1}{10} + \frac{-3}{10} + \frac{14}{10} & \frac{-4}{10} + \frac{-3}{10} + \frac{7}{10} \\ \frac{3}{10} + \frac{-2}{10} + \frac{-1}{10} & \frac{3}{10} + \frac{-1}{10} + \frac{-2}{10} & \frac{12}{10} + \frac{-1}{10} + \frac{-1}{10} \\ \end{array}\right)$$
$$=\left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right)$$