Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

9.2. The Characteristic Polynomial

So far, we’ve found the eigenvalues and eigenvectors of a matrix by eyeballing them or reasoning about them geometrically, but this is not a sustainable strategy. Let’s develop a more systematic approach.

Deriving the Characteristic Polynomial

We’re looking for combinations of scalars, λ\mathbf{\lambda}, and vectors, v⃗\vec v, such that

Av⃗=λv⃗A \vec v = \lambda \vec v

In some ways, we’re trying to “solve” for λ\mathbf{\lambda} and v⃗\vec v given AA. Let’s experiment a little:

Av⃗=λv⃗Av⃗−λv⃗=0⃗(A−λI)v⃗=0⃗\begin{align*} A \vec v = \lambda \vec v \\ A \vec v - \lambda \vec v = \vec 0 \\ (A - \lambda I) \vec v = \vec 0 \end{align*}

What the above says is that if v⃗\vec v is an eigenvector of AA with eigenvalue λ\mathbf{\lambda}, then v⃗\vec v is in the null space of A−λIA - \lambda I. But, since eigenvectors can’t be the zero vector, this means that A−λIA - \lambda I is not invertible, since it has a non-trivial null space!

Thinking back to Chapter 6.2, we know that there are several equivalent ways to check if a matrix is not invertible. Perhaps the most computational approach is to compute its determinant; if a (square) matrix’s determinant is 0, then it is not invertible, otherwise it is.

So, since A−λIA - \lambda I is not invertible, its determinant must be 0!

Av⃗=λv⃗  ⟹  det⁡(A−λI)=0A \vec v = \lambda \vec v \implies \det(A - \lambda I) = 0

(We won’t always use the symbol p(λ)p(\lambda), I just introduced it above to make it clear that det(A−λI)\text{det}(A - \lambda I) is a polynomial function of λ\mathbf{\lambda}.)

Let’s revisit our example A=[1221]A = \begin{bmatrix} 1 & 2 \\ 2 & 1 \end{bmatrix}.

The matrix A−λIA - \lambda I is

A−λI=[1221]−λ[1001]=[1−λ221−λ]A - \lambda I = \begin{bmatrix} 1 & 2 \\ 2 & 1 \end{bmatrix} - \lambda \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 - \lambda & 2 \\ 2 & 1 - \lambda \end{bmatrix}

Note that A−λIA - \lambda I involves subtracting λ\mathbf{\lambda} from the diagonal elements of AA, and leaving all other elements unchanged.

The determinant of A−λIA - \lambda I is

det⁡(A−λI)=(1−λ)(1−λ)−2⋅2=λ2−2λ+1−4=λ2−2λ−3⏟p(λ)\det(A - \lambda I) = (1 - \lambda)(1 - \lambda) - 2 \cdot 2 = \lambda^2 - 2\lambda + 1 - 4 = \underbrace{\lambda^2 - 2\lambda - 3}_{p(\lambda)}

The eigenvalues of AA are the values of λ\mathbf{\lambda} where λ2−2λ−3=0\lambda^2 - 2\lambda - 3 = 0.

Image produced in Jupyter

(In general, we’re not going to plot the characteristic polynomial each time, but I think it’s useful to see once or twice to give the idea some context.)

By factoring, I can write this equation as

(λ+1)(λ−3)=0(\lambda + 1)(\lambda - 3) = 0

which tells me that the eigenvalues of AA are λ1=−1\lambda_1 = -1 and λ2=3\lambda_2 = 3. If this equation weren’t factorable, I’d need to use the quadratic formula to find the eigenvalues. For now, there’s no particular “ordering” to the eigenvalues, i.e. I could have said λ1=3\lambda_1 = 3 and λ2=−1\lambda_2 = -1; all that matters is that I stay consistent throughout a particular example.

Once we find the eigenvalues by solving p(λ)=0p(\lambda) = 0, we can find the eigenvectors by solving (A−λI)v⃗=0⃗(A - \lambda I) \vec v = \vec 0 for each eigenvalue.

  • For λ1=−1\lambda_1 = -1, we’re looking for vectors v⃗\vec v such that Av⃗=−1v⃗A \vec v = -1 \vec v, i.e. if v⃗=[ab]\vec v = \begin{bmatrix} a \\ b \end{bmatrix}, then

    [1221][ab]=[−a−b]\begin{bmatrix} 1 & 2 \\ 2 & 1 \end{bmatrix} \begin{bmatrix} a \\ b \end{bmatrix} = \begin{bmatrix} -a \\ -b \end{bmatrix}

    (I’m using aa and bb instead of v1v_1 and v2v_2 because I’ll refer to the vectors v⃗1\vec v_1 and v⃗2\vec v_2 in just a moment.) As a system of equations, this says

    a+2b=−a2a+b=−b\begin{align*} a + 2b &= -a \\ 2a + b &= -b \end{align*}

    The first and second equations both tell us that b=−ab = -a. Remember, we’d expect there to be infinitely many solutions to this system, since any scalar multiple of an eigenvector is still an eigenvector. So, the “simple” solution is v⃗1=[1−1]\vec v_1 = \begin{bmatrix} 1 \\ -1 \end{bmatrix}, but [2−2]\begin{bmatrix} 2 \\ -2 \end{bmatrix}, [3−3]\begin{bmatrix} 3 \\ -3 \end{bmatrix}, etc. are also solutions.

  • For λ2=3\lambda_2 = 3, let me introduce another way of finding the corresponding eigenvector. There’s nothing wrong with the system of equations approach, but it’s useful to have multiple techniques for solving problems in our toolkit. λ2=3\lambda_2 = 3 tells us that Av⃗=3v⃗A \vec v = 3 \vec v, or equivalently, (A−3I)v⃗=0⃗(A - 3I) \vec v = \vec 0. So, the eigenvector we’re looking for is in the null space of A−3IA - 3I.

    A−3I=[1−3221−3]=[−222−2]A - 3I = \begin{bmatrix} 1 - 3 & 2 \\ 2 & 1 - 3 \end{bmatrix} = \begin{bmatrix} -2 & 2 \\ 2 & -2 \end{bmatrix}

    Here, we notice that both rows of A−3IA - 3I sum to 0, meaning that

    (A−3I)[11]=[00](A - 3I) \begin{bmatrix} 1 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}

    So, [11]\begin{bmatrix} 1 \\ 1 \end{bmatrix} is an eigenvector for λ2=3\lambda_2 = 3.

Examples

Example: 2×22 \times 2 Matrices

Find the eigenvalues and eigenvectors of A=[3124]A = \begin{bmatrix} 3 & 1 \\ 2 & 4 \end{bmatrix}.

Solution
A−λI=[3−λ124−λ]det(A−λI)= (3−λ)(4−λ)−(1⋅2)=12−7λ+λ2−2=λ2−7λ+10=(λ−2)(λ−5)\begin{align*} A-\lambda I &= \begin{bmatrix} 3 - \lambda & 1 \\ 2 & 4 - \lambda \end{bmatrix} \\\text{det}(A-\lambda I) &= \ (3 - \lambda)(4 - \lambda)-(1 \cdot 2) \\&= 12 -7\lambda + \lambda^2 - 2 \\&= \lambda^2 - 7\lambda +10 \\&= (\lambda-2)(\lambda-5) \end{align*}

So our eigenvalues are λ1=2\lambda_1 = 2 and λ2=5\lambda_2=5. Next, let’s find eigenvectors for them. As I did in the first example, I’ll show two different techniques for finding eigenvectors.

  • For λ1=2\lambda_1 = 2, we’re looking for a vector v⃗=[ab]\vec v = \begin{bmatrix} a \\ b \end{bmatrix} such that Av⃗=2v⃗A \vec v = 2 \vec v:

    Av⃗=2v⃗[3124][ab]=[2a2b]3a+b=2a2a+4b=2b\begin{align*} A\vec v &= 2\vec v \\ \begin{bmatrix} 3 & 1 \\ 2 & 4 \end{bmatrix} \begin{bmatrix} a \\ b \end{bmatrix} &= \begin{bmatrix} 2a \\ 2b \end{bmatrix} \\ 3a + b &= 2a \\ 2a + 4b &= 2b \end{align*}

    As expected, there are infinitely many choices for aa and bb (since both equations tell us that b=−ab = -a), and the resulting eigenvectors v⃗\vec v all lie on the same line. So, if we let a=1a = 1, then we have that v⃗1=[1−1]\vec v_1 = \begin{bmatrix} 1 \\ -1 \end{bmatrix} is an eigenvector for λ1\lambda_1.

  • For λ2=5\lambda_2 = 5, I’ll try the null space approach, just to illustrate it once more. Again, I’m looking for a vector v⃗\vec v such that Av⃗=5v⃗A \vec v = 5 \vec v, or equivalently, (A−5I)v⃗=0⃗(A - 5I) \vec v = \vec 0.

    A−5I=[3−5124−5]=[−212−1]A - 5I = \begin{bmatrix} 3 - 5 & 1 \\ 2 & 4 - 5 \end{bmatrix} = \begin{bmatrix} -2 & 1 \\ 2 & -1 \end{bmatrix}

    The vector v⃗2=[12]\vec v_2 = \begin{bmatrix} 1 \\ 2 \end{bmatrix} is orthogonal to both rows of A−5IA - 5I, meaning (A−5I)[12]=[00](A - 5I) \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}. So, v⃗2=[12]\vec v_2 = \begin{bmatrix} 1 \\ 2 \end{bmatrix} is an eigenvector for λ2=5\lambda_2 = 5, though again any non-zero scalar multiple of [12]\begin{bmatrix} 1 \\ 2 \end{bmatrix} will also be an eigenvector for λ2=5\lambda_2 = 5.

So, AA has eigenvalues λ1=2\lambda_1 = 2 and λ2=5\lambda_2 = 5, and corresponding eigenvectors v⃗1=[1−1]\vec v_1 = \begin{bmatrix} 1 \\ -1 \end{bmatrix} and v⃗2=[12]\vec v_2 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}.

Example: Diagonal Matrices

Find the eigenvalues and eigenvectors of A=[300−4]A = \begin{bmatrix} 3 & 0 \\ 0 & -4 \end{bmatrix}. What do you notice about the eigenvalues? The eigenvectors?

Solution

The key is to notice that for diagonal matrices – that is, matrices where all the entries off the diagonal are 0 – the eigenvalues are simply the entries on the diagonal.

A−λI=[3−λ00−4−λ]det(A−λI)=(3−λ)(−4−λ)\begin{align*} A-\lambda I &= \begin{bmatrix} 3 - \lambda & 0 \\ 0 &-4 - \lambda \end{bmatrix} \\\text{det}(A-\lambda I) &= (3 - \lambda)(-4 - \lambda) \end{align*}

The eigenvalues λ1=3\lambda_1 = 3 and λ2=−4\lambda_2 = -4 are the same as the values on the diagonal! The corresponding eigenvectors are v⃗1=[10]\vec v_1 = \begin{bmatrix}1 \\ 0\end{bmatrix} and v⃗2=[01]\vec v_2 = \begin{bmatrix}0 \\ 1\end{bmatrix}. Just to illustrate one of those cases out,

Av⃗2=[300−4][01]=[0−4]=−4[01]A \vec v_2 = \begin{bmatrix} 3 & 0 \\ 0 & -4 \end{bmatrix} \begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ -4 \end{bmatrix} = -4 \begin{bmatrix} 0 \\ 1 \end{bmatrix}

The fact that eigenvalues and eigenvectors are so easy to find for diagonal matrices, coupled with the fact that diagonal matrices are really easy to compute powers of (e.g. A2A^2 is just the diagonal matrix with each entry squared), makes them very useful in practice.

Example: Sharing a Characteristic Polynomial

Find two different 2×22 \times 2 matrices who have the characteristic polynomial

p(λ)=λ2−4λ+3p(\lambda) = \lambda^2 - 4 \lambda + 3
Solution

Recall that for a 2×22 \times 2 matrix A=[abcd]A=\begin{bmatrix}a & b \\ c & d\end{bmatrix}:

p(λ)=det(A−λI)=∣a−λbcd−λ∣=(a−λ)(d−λ)−bc\begin{align*} p(\lambda)&=\text{det}(A - \lambda I) \\&=\begin{vmatrix}a - \lambda & b \\ c & d-\lambda \end{vmatrix} \\&=(a-\lambda)(d-\lambda)-bc \end{align*}

A simple approach to this problem is to find 2 diagonal matrices D=[a00d]D = \begin{bmatrix}a & 0 \\ 0 & d\end{bmatrix} and D′=[d00a]D'=\begin{bmatrix}d & 0 \\ 0 & a\end{bmatrix}. That way, all we have to do is factor p(λ)p(\lambda) into the form (a−λ)(d−λ)(a-\lambda)(d-\lambda):

λ2−4λ+3=(3−λ)(1−λ)\begin{align*} \lambda^2-4\lambda+3 &= (3-\lambda)(1-\lambda) \end{align*}

So, our two different matrices are D=[3001]D = \begin{bmatrix}3 & 0 \\ 0 & 1\end{bmatrix} and D′=[1003]D' = \begin{bmatrix}1 & 0 \\ 0 & 3\end{bmatrix}.

There exist non-diagonal matrices with the same characteristic polynomial, too. For instance,

[2112]\begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}

also has eigenvalues of 1 and 3, and its characteristic polynomial is also p(λ)=λ2−4λ+3p(\lambda) = \lambda^2 - 4\lambda + 3.

Example: 3×33 \times 3 Matrices

Recall, if AA is a 3×33 \times 3 matrix, like

A=[abcdefghi]A = \begin{bmatrix} {\color{3d81f6} a} & {\color{3d81f6} b} & {\color{3d81f6} c} \\ {\color{orange} d} & {\color{d81a60}e } & {\color{004d40}f } \\ {\color{orange} g} & {\color{d81a60}h } & {\color{004d40}i } \end{bmatrix}

then det(A)\text{det}(A) is

det(A)=a∣efhi∣−b∣dfgi∣+c∣degh∣\text{det}(A) = {\color{3d81f6} a} \begin{vmatrix} {\color{d81a60}e } & {\color{004d40}f } \\ {\color{d81a60}h } & {\color{004d40}i } \end{vmatrix} - {\color{3d81f6} b} \begin{vmatrix} {\color{orange} d} & {\color{004d40}f } \\ {\color{orange} g} & {\color{004d40}i } \end{vmatrix} + {\color{3d81f6} c} \begin{vmatrix} {\color{orange} d} & {\color{d81a60}e } \\ {\color{orange} g} & {\color{d81a60}h } \end{vmatrix}

where ∣⋅∣| \cdot | denotes the determinant of the matrix.

Find the eigenvalues and eigenvectors of the upper triangular matrix

A=[330015002]A = \begin{bmatrix} 3 & 3 & 0 \\ 0 & 1 & 5 \\ 0 & 0 & 2 \end{bmatrix}

What do you notice about the eigenvalues?

Solution

We’ll start by calculating det(A−λI)\text{det}(A-\lambda I) using the formula for the 3×33 \times 3 determinant provided for us:

A−λI=[3−λ3001−λ5002−λ]det(A−λI)= (3−λ)∣1−λ502−λ∣− 3∣0502−λ∣+ 0∣01−λ00∣=(3−λ)[(1−λ)(2−λ)−0]− 3[0−0]+ 0[0−0]=(3−λ)(1−λ)(2−λ)\begin{align*} A-\lambda I &= \begin{bmatrix} {\color{3d81f6}3 - \lambda} & {\color{3d81f6}3} & {\color{3d81f6}0} \\ {\color{orange}0} & {\color{d81a60}1-\lambda} & {\color{004d40}5} \\ {\color{orange}0} & {\color{d81a60}0} & {\color{004d40}2 - \lambda} \end{bmatrix} \\ \text{det}(A-\lambda I)= &~{\color{3d81f6}(3-\lambda)} \begin{vmatrix} {\color{d81a60}1-\lambda} & {\color{004d40}5} \\ {\color{d81a60}0} & {\color{004d40}2-\lambda} \end{vmatrix} -~ {\color{3d81f6}3} \begin{vmatrix} {\color{orange}0} & {\color{004d40}5} \\ {\color{orange}0} & {\color{004d40}2-\lambda} \end{vmatrix} +~ {\color{3d81f6}0} \begin{vmatrix} {\color{orange}0} & {\color{d81a60}1-\lambda} \\ {\color{orange}0} & {\color{d81a60}0} \end{vmatrix} \\ &= (3-\lambda)\big[(1-\lambda)(2-\lambda) - 0\big] -~ 3\big[0 - 0\big] +~ 0\big[0 - 0\big] \\ &= (3-\lambda)(1-\lambda)(2-\lambda) \end{align*}

So, the eigenvalues are the values along the diagonal: λ1=3\lambda_1=3, λ2=1\lambda_2=1, and λ3=2\lambda_3=2. This is a convenient property of triangular matrices (which is why I’ve planted this example here). Next, let’s find their eigenvectors.

  • For λ1=3\lambda_1 = 3, I’ll do it the null space way: I’m looking for a vector in the null space of A−3IA - 3I. (I say “the null space way” but it’s really just a different way of writing the system of equations approach, and one that can sometimes be easier to eyeball.)

    A−3I=[0300−2500−1]A - 3I = \begin{bmatrix} 0 & 3 & 0 \\ 0 & -2 & 5 \\ 0 & 0 & -1 \end{bmatrix}

    The zeros in the first column tell me that (A−3I)[100]=[000](A - 3I) \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}, so v⃗1=[100]\vec v_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} is an eigenvector for λ1=3\lambda_1 = 3. Because I have three unique eigenvalues and my matrix is 3×33 \times 3, I know that this is the only possible eigenvector direction for λ1=3\lambda_1 = 3; in Chapter 9.4, we’ll run into situations where the null space of A−3IA - 3I (for instance) is spanned by more than one vector, but that’s not the case here.

  • For λ2=1\lambda_2 = 1, I’ll do it the system of equations way: I’m looking for a vector v⃗=[abc]\vec v = \begin{bmatrix} a \\ b \\ c \end{bmatrix} such that Av⃗=1v⃗A \vec v = 1 \vec v.

    Av⃗=v⃗[330015002][abc]=[abc]3a+3b=ab+5c=b2c=c\begin{align} A\vec v &= \vec v \\ \begin{bmatrix} 3 & 3 & 0 \\ 0 & 1 & 5 \\ 0 & 0 & 2 \end{bmatrix} \begin{bmatrix} a \\ b \\ c \end{bmatrix} &= \begin{bmatrix} a \\ b \\ c \end{bmatrix} \\3a + 3b &= a \\b+5c &= b \\2c &= c \end{align}

    Once again, there are infinitely many solutions to this system, which we’d expect since there’s a whole line of eigenvectors.

    • The third equation above tells us that c=0c = 0.

    • Substituting that into the second equation, we have that b+5⋅0=bb + 5 \cdot 0 = b, i.e. b=bb = b, so let’s treat bb as a free variable for now.

    • The first equation above tells us that 3a+3b=a3a + 3b = a, i.e. a=−32ba = -\frac{3}{2}b.

    So, any vector of the form [−32bb0]\begin{bmatrix} -\frac{3}{2}b \\ b \\ 0 \end{bmatrix} is an eigenvector for λ2=1\lambda_2 = 1. But we just need to find one, so let’s take b=2b = 2, which gives us v⃗2=[−320]\vec v_2 = \begin{bmatrix} -3 \\ 2 \\ 0 \end{bmatrix}.

  • Finally, for λ3=2\lambda_3 = 2, I’ll do it the null space way.

    A−2I=[3−23001−25002−2]=[1300−15000]A - 2I = \begin{bmatrix} 3 - 2 & 3 & 0 \\ 0 & 1 - 2 & 5 \\ 0 & 0 & 2 - 2 \end{bmatrix} = \begin{bmatrix} 1 & 3 & 0 \\ 0 & -1 & 5 \\ 0 & 0 & 0 \end{bmatrix}

    I’m looking for a vector v⃗3=[abc]\vec v_3 = \begin{bmatrix} a \\ b \\ c \end{bmatrix} in the null space of A−2IA - 2I, i.e. a vector whose dot product with each row of A−2IA - 2I is 0. The last row of A−2IA - 2I is all zeros, so I can focus my attention on the first two rows. If the first two components of v⃗3\vec v_3 are -3 and 1, then the dot product of v⃗3\vec v_3 with the first row of A−2IA - 2I is (1)(−3)+(3)(1)=0(1)(-3) + (3)(1) = 0, which is what I want.

    Then, its dot product with the second row is (0)(−3)+(−1)(1)+(5)(c)=−1+5c(0)(-3) + (-1)(1) + (5)(c) = -1 + 5c, where cc is the last component of v⃗3\vec v_3. For −1+5c=0-1 + 5c = 0, we need c=15c = \frac{1}{5}. So, one vector in the null space of A−2IA - 2I – which is an eigenvector for λ3=2\lambda_3 = 2 – is v⃗3=[−3115]\vec v_3 = \begin{bmatrix} -3 \\ 1 \\ \frac{1}{5} \end{bmatrix}. To clean the numbers up a bit, I’ll scale this vector by 5, which gives us v⃗3=[−1551]\vec v_3 = \begin{bmatrix} -15 \\ 5 \\ 1 \end{bmatrix}.

So, our eigenvectors for λ1=3\lambda_1=3, λ2=1\lambda_2=1, and λ3=2\lambda_3=2 are v⃗1=[100]\vec v_1=\begin{bmatrix}1 \\ 0 \\ 0\end{bmatrix}, v⃗2=[−320]\vec v_2=\begin{bmatrix}-3 \\ 2 \\ 0\end{bmatrix}, and v⃗3=[−1551]\vec v_3 = \begin{bmatrix}-15 \\ 5 \\ 1 \end{bmatrix}.

# When in doubt, check with numpy!
np.linalg.eig(
    np.array([
        [2, 0, 0],
        [-1, 5, 0],
        [4, 3, -7]
    ])
)
EigResult(eigenvalues=array([-7., 5., 2.]), eigenvectors=array([[0. , 0. , 0.83925433], [0. , 0.9701425 , 0.27975144], [1. , 0.24253563, 0.4662524 ]]))

Number of Eigenvalues

The characteristic polynomial of an n×nn \times n matrix is a polynomial of degree nn. A fact from algebra is that a polynomial of degree nn has exactly nn roots. The intuitive way of thinking about this is that degree nn polynomials can have at most n−1n-1 “bends” (a line can’t bend, a quadratic can bend once, a cubic can bend twice, etc.), and each time it bends, it can change directions to cross the xx-axis again.

The issue is that some of these roots may be repeated, and some may be complex numbers, meaning that they don’t actually cross the xx-axis in the standard xyxy-plane (or in our case, the λ\mathbf{\lambda}-axis and λ,p(λ)\lambda, p(\lambda)-plane).

For example, the matrix A=[4000010000000004]A = \begin{bmatrix} 4 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 4 \end{bmatrix} is diagonal, meaning its easy to read off its characteristic polynomial:

p(λ)=det⁡(A−λI)=∣4−λ00001−λ0000−λ00004−λ∣=λ(λ−1)(λ−4)2p(\lambda) = \det(A - \lambda I) = \begin{vmatrix} 4 - \lambda & 0 & 0 & 0 \\ 0 & 1 - \lambda & 0 & 0 \\ 0 & 0 & -\lambda & 0 \\ 0 & 0 & 0 & 4 - \lambda \end{vmatrix} = \lambda (\lambda - 1)(\lambda - 4)^2

This characteristic polynomial has a double root at λ=4\lambda = 4, a single root at λ=1\lambda = 1, and a single root at λ=0\lambda = 0. This AA has 3 distinct eigenvalues, but one of them, λ=2\lambda = 2, is repeated, and has an algebraic multiplicity of 2.

Image produced in Jupyter

As another example, the matrix A=[3−443]A = \begin{bmatrix} 3 & -4 \\ 4 & 3 \end{bmatrix} has the characteristic polynomial

p(λ)=∣3−λ−443−λ∣=(λ−3)2+16p(\lambda) = \begin{vmatrix} 3 - \lambda & -4 \\ 4 & 3 - \lambda \end{vmatrix} = (\lambda - 3)^2 + 16

I can expand this out to get λ2−6λ+25\lambda^2 - 6\lambda + 25, but in a case like this I think the above form is more telling. Visually, p(λ)p(\lambda) here is a parabola that sits entirely above the λ\mathbf{\lambda}-axis, meaning it has no real roots, so AA has no real eigenvalues.

Image produced in Jupyter

It should not be all that surprising that A=[3−443]A = \begin{bmatrix} 3 & -4 \\ 4 & 3 \end{bmatrix} has no real eigenvalues. AA is a rotation matrix, scaled by a factor of 5:

A=[3−443]=5[35−454535]=5[cos⁡θ−sin⁡θsin⁡θcos⁡θ]A = \begin{bmatrix} 3 & -4 \\ 4 & 3 \end{bmatrix} = 5 \begin{bmatrix} \frac{3}{5} & -\frac{4}{5} \\ \frac{4}{5} & \frac{3}{5} \end{bmatrix} = 5 \begin{bmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{bmatrix}

where θ=cos⁡−1(35)\theta = \cos^{-1}\left(\frac{3}{5}\right). This matrix rotates vectors by θ\mathbf{\theta} radians counterclockwise, which means that no real-valued vector will remain in the same direction after being multiplied by AA. In the 2×22 \times 2 case, the only way to get a real-valued eigenvalue out of a rotation matrix is if the matrix rotates by an integer multiple of π\mathbf{\pi} radians (180 degrees), which either corresponds to reflecting/negating the vector (like in [−100−1]\begin{bmatrix} -1 & 0 \\ 0 & -1 \end{bmatrix}) or returning back the same vector itself (like in [1001]\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, the identity matrix, which can be thought of as a rotation by 2π2 \pi).

The solutions to p(λ)=(λ−3)2+16=0p(\lambda) = (\lambda - 3)^2 + 16 = 0 are the complex numbers λ=3+4i\lambda = 3 + 4i and λ=3−4i\lambda = 3 - 4i, again where ii is the imaginary unit, defined by i2=−1i^2 = -1. The corresponding eigenvectors are complex too, and we won’t worry about finding them. (If you’re familiar with complex numbers, you might recognize these eigenvalues as 5eiθ5e^{i\theta} and 5e−iθ5e^{-i\theta}, where θ=cos⁡−1(35)\theta = \cos^{-1}\left(\frac{3}{5}\right). This comes from Euler’s formula, eiθ=cos⁡θ+isin⁡θe^{i\theta} = \cos \theta + i \sin \theta.)

Trace and Determinant

As a final example, consider the arbitrary 2×22 \times 2 matrix

A=[abcd]A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}

Its characteristic polynomial is

p(λ)=∣a−λbcd−λ∣=(a−λ)(d−λ)−bc=λ2−(a+d)λ+ad−bcp(\lambda) = \begin{vmatrix} a - \lambda & b \\ c & d - \lambda \end{vmatrix} = (a - \lambda)(d-\lambda) - bc = \lambda^2 - (a + d)\lambda + ad-bc

The coefficient on λ\mathbf{\lambda} is −(a+d)-(a + d), which is the negative of the sum of the diagonal entries of AA. A term often used to refer to the sum of the diagonal entries of a matrix is its trace, trace(A)\text{trace}(A). And, the constant term of p(λ)p(\lambda) above is just det(A)\text{det}(A)! For 2×22 \times 2 matrices, this says that the characteristic polynomial is of the form

p(λ)=λ2−trace(A)λ+det(A)p(\lambda) = \lambda^2 - \text{trace}(A) \lambda + \text{det}(A)

For example, consider A=[3179]A = \begin{bmatrix} 3 & 1 \\ 7 & 9 \end{bmatrix}. The fact above says that AA’s characteristic polynomial is

p(λ)=λ2−(3+9)λ+(3⋅9−1⋅7)=λ2−12λ+20p(\lambda) = \lambda^2 - (3 + 9)\lambda + (3 \cdot 9 - 1 \cdot 7) = \lambda^2 - 12\lambda + 20

which, with enough practice, is a calculation you might be able to do in your head. But, this gives way to a more general fact, that is true for any n×nn \times n matrix.

Example 1: A=[3179]A = \begin{bmatrix} 3 & 1 \\ 7 & 9 \end{bmatrix}'s eigenvalues add up to 3+9=123 + 9 = 12 and multiply to det(A)=3⋅9−1⋅7=20\text{det}(A) = 3 \cdot 9 - 1 \cdot 7 = 20. This gives me a quick way of figuring out that AA’s eigenvalues are 10 and 2.

Example 2: A=[0.50.500.40.30.300.50.5]A = \begin{bmatrix} 0.5 & 0.5 & 0 \\ 0.4 & 0.3 & 0.3 \\ 0 & 0.5 & 0.5 \end{bmatrix} has an eigenvalue of 1, corresponding to the eigenvector [111]\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}, since the sum of each row of AA is 1. Let λ2\lambda_2 and λ3\lambda_3 be the other two eigenvalues. The trace fact tells me that

1+λ2+λ3=0.5+0.3+0.5=1.3  ⟹  λ2+λ3=0.31 + \lambda_2 + \lambda_3 = 0.5 + 0.3 + 0.5 = 1.3 \implies \lambda_2 + \lambda_3 = 0.3

and the determinant fact tells me that

1⋅λ2⋅λ3=∣0.50.500.40.30.300.50.5∣=0.5(0.3⋅0.5−0.3⋅0.5)−0.5(0.4⋅0.5−0.3⋅0)+0(0.4⋅0.5−0.3⋅0)=−0.5⋅0.4⋅0.5=−0.1\begin{align*} 1 \cdot \lambda_2 \cdot \lambda_3 &= \begin{vmatrix} 0.5 & 0.5 & 0 \\ 0.4 & 0.3 & 0.3 \\ 0 & 0.5 & 0.5 \end{vmatrix} \\ &= 0.5 (0.3 \cdot 0.5 - 0.3 \cdot 0.5) - 0.5 (0.4 \cdot 0.5 - 0.3 \cdot 0) + 0 (0.4 \cdot 0.5 - 0.3 \cdot 0) \\ &= -0.5 \cdot 0.4 \cdot 0.5 &= -0.1 \end{align*}

In other words, the other two eigenvalues sum to 0.3 and multiply to -0.1. This sounds like a job for 0.5 and -0.2, which are indeed the other two eigenvalues of AA (other than 1).

Example 3: A=[1224]A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix} has a determinant of 1⋅4−2⋅2=01 \cdot 4 - 2 \cdot 2 = 0, which means that its eigenvalues multiply to 0, which is another reminder that AA has an eigenvalue of 0. The trace fact tells me that the other eigenvalue λ2\lambda_2 must satisfy 1+4=0+λ21 + 4 = 0 + \lambda_2, so λ2=5\lambda_2 = 5.

Key Takeaways

Let’s summarize the key ideas from both this section and the last.

Suppose AA is an n×nn \times n matrix.

  1. λ\mathbf{\lambda} is an eigenvalue of AA, corresponding to the eigenvector v⃗\vec v, if

    Av⃗=λv⃗A \vec v = \lambda \vec v

    The English interpretation of this is that v⃗\vec v is an eigenvector of AA if, when multiplied by AA, v⃗\vec v still points in the same direction, and is just stretched by a factor of λ\mathbf{\lambda}.

  2. An n×nn \times n matrix AA has exactly nn eigenvalues. Some of these may be complex, and some of these may be repeated, but all of them are roots to the characteristic polynomial of AA,

p(λ)=det⁡(A−λI)p(\lambda) = \det(A - \lambda I)

  1. A quick way to check if you’ve found the right eigenvalues is that

  • The product of the eigenvalues of AA is equal to det⁡(A)\det(A), i.e. λ1λ2⋯λn=det⁡(A)\lambda_1 \lambda_2 \cdots \lambda_n = \det(A).

  • The sum of the eigenvalues of AA is equal to the trace of AA, which is the sum of the diagonal elements of AA.

  1. 0 is an eigenvalue of AA if and only if AA is not invertible.

  2. If λ\mathbf{\lambda} is an eigenvalue of AA with eigenvector v⃗\vec v, then λk\mathbf{\lambda}^k is an eigenvalue of AkA^k with the same eigenvector v⃗\vec v.

Chapter 9: Eigenvalues and Eigenvectors
9.1. Eigenvalues and Eigenvectors
Chapter 9: Eigenvalues and Eigenvectors
9.3. Markov Chains and Adjacency Matrices