Eigenvalues and Eigenvectors

Consider a $2 \times 2$ homogeneous constant-coefficient linear system:


\begin{displaymath}\begin{array}{rl}
x_1' &= a x_1 + b x_2 \cr
x_2' &= c x_1 + d x_2\cr
\end{array}\end{displaymath}

where we assume $b \ne 0$ in order to solve this by reducing it to a second-order linear equation:


\begin{displaymath}
\displaylines{
x_2 = (x_1' - a x_1)/b \cr
x_2' = (x_1'' - a ...
...1' - a x_1)/b \cr
x_1'' - (a + d) x_1' + (ad - bc) x_1 = 0\cr}
\end{displaymath}

The characteristic polynomial is $P(r) = r^2 - (a+d) r + ad - bc$. If $r$ is a root of this, we get a solution $x_1 = {\rm e}^{rt}$ with $x_2 = (x_1' - a x_1)/b = (r-a) {\rm e}^{rt}/b$, i.e.

\begin{displaymath}
\pmatrix{x_1\cr x_2\cr} = \pmatrix{1\cr (r-a)/b\cr} {\rm e}^{rt}
\end{displaymath}

Look at it from another point of view. This solution is of the form ${\bf x}= {\bf u}{\rm e}^{rt}$ where ${\bf u}$ is a constant vector and $r$ is a constant number. Suppose we look for solutions of that form. Recall that we write the system in matrix and vector language as ${\bf x}' = A {\bf x}$ where $\displaystyle A =
\pmatrix{a & b\cr c & d\cr}$ is the matrix of coefficients.

\begin{displaymath}
\displaylines{
{\bf x}= {\bf u}{\rm e}^{rt} \cr
{\bf x}' = r {\bf u}{\rm e}^{rt} \cr
A {\bf x}= A {\bf u}{\rm e}^{rt}\cr}\end{displaymath}

To make these equal, we need $A {\bf u}= r {\bf u}$. Of course, we could take $ {\bf u}= 0$ (i.e. the zero vector $\displaystyle \pmatrix{0\cr 0\cr}$), but that would be quite useless: 0 is always a solution of a homogeneous system. We obtain a general result (true for $n \times n$ as well as $2 \times 2$ systems):

If $A {\bf u}= r {\bf u}$ for some nonzero vector ${\bf u}$ and some number $r$, then we say that $r$ is an eigenvalue of the matrix $A$ and ${\bf u}$ is an eigenvector of $A$ for that eigenvalue.

If ${\bf u}$ is an eigenvector of $A$ for an eigenvalue $r$, then ${\bf x}= {\bf u}{\rm e}^{rt}$ is a solution of the homogeneous system ${\bf x}' = A {\bf x}$.


Now, how do we find the eigenvalues and their corresponding eigenvectors? For $2 \times 2$ systems we get the eigenvalues from the characteristic polynomial of the corresponding second-order differential equation: $P(r) = r^2 - (a+d) r + ad - bc
= (a-r)(d-r) - b c$. This can be written in terms of determinants.


The determinant of the $2 \times 2$ matrix $\displaystyle A =
\pmatrix{a & b\cr c & d\cr}$ is $a d - b c$.

The characteristic polynomial of $A$ is $P(r) = (a-r)(d-r) - bc = \mbox{det} \pmatrix{ a-r & b\cr c & d-r\cr}$.

The eigenvectors of $A$ are the roots of the characteristic polynomial.


Once you have the eigenvalue $r$, you can find an eigenvector ${\bf u}$ by looking at one of the equations in the system $A {\bf u}= r {\bf u}$. This will only determine the ratio of the entries $u_1$ and $u_2$: since a multiple of an eigenvector is also an eigenvector, there is nothing else to be determined.

Example: $\displaystyle
A = \pmatrix{ 1 & 2\cr 4 & 3\cr}$. The characteristic polynomial of $A$ is $\displaystyle \mbox{det} \pmatrix {1-r & 2\cr
4 & 3-r\cr} = r^2 - 4 r - 5$. Its roots, which are the eigenvalues, are $-1$ and $5$.

For the eigenvalue $-1$, the system $A {\bf u}= r {\bf u}$ says

\begin{displaymath}
\begin{array}{rl}
u_1 + 2 u_2 &= - u_1 \\
4 u_1 + 3 u_2 &= - u_2
\end{array}\end{displaymath}

or

\begin{displaymath}
\begin{array}{rl}
2 u_1 + 2 u_2 &= 0\\
4 u_1 + 4 u_2 &= 0\\
\end{array}\end{displaymath}

Note that the two equations say the same thing (one is a multiple of the other). That must be the case if $r$ is an eigenvalue: otherwise the two equations together would require $u_1 = u_2 = 0$, which we don't allow for an eigenvector.

We can take any convenient value for one of the variables, and determine the other from one of the equations: e.g. take $u_1 = 1$ and then $u_2 = -1$. Thus $\displaystyle \pmatrix{1\cr -1\cr}$ is an eigenvector for eigenvalue $-1$, and one solution of our system is $\displaystyle {\bf x}= \pmatrix{1\cr-1\cr} {\rm e}^{-t}$. Similarly, an eigenvector for the other eigenvalue, 5, is $\displaystyle \pmatrix{1\cr 2\cr}$, corresponding to a solution $\displaystyle {\bf x}= \pmatrix{1\cr 2\cr} {\rm e}^{5t}$. Thus the general solution of our homogeneous system is

\begin{displaymath}
{\bf x}= c_1 \pmatrix{1\cr-1\cr} {\rm e}^{-t} + c_2 \pmatrix{1\cr 2\cr} {\rm e}^{5t}\end{displaymath}

In some cases the equation may say that one of the variables is 0, so we take any convenient value (say 1) for the other variable. Also, one of the two equations may just say $0 = 0$, so we look at the other equation. Both of these occur (for different eigenvalues) with the matrix $\displaystyle A = \pmatrix{2 & 1\cr 0 & 3\cr}$. The eigenvalues are 2 and 3. For eigenvalue 2 we have

\begin{displaymath}
\begin{array}{rl}
2 u_1 + u_2 &= 2 u_1\\
3 u_2 &= 2 u_2
\end{array}\end{displaymath}

so $u_2 = 0$ and we can take $u_1 = 1$. For eigenvalue 3 we have

\begin{displaymath}
\begin{array}{rl}
2 u_1 + u_2 &= 3 u_1\\
3 u_2 &= 3 u_2
\end{array}\end{displaymath}

The second equation is equivalent to $0 = 0$, but the first says $u_2 = u_1$, so we can take $u_1 = 1$ and $u_2 = 1$.

It is also possible for both of the equations to say $0 = 0$, in which case all nonzero vectors are eigenvectors. This occurs with the matrix $\displaystyle A = \pmatrix{2 & 0\cr 0 & 2\cr}$, whose only eigenvalue is 2.

We need two solutions of a $2 \times 2$ homogeneous system to form a fundamental set. If there are two different eigenvalues, the solutions obtained using an eigenvector for each do form a fundamental set. If there is only one eigenvalue and all nonzero vectors are eigenvectors (as in the last paragraph), we can use any two eigenvectors that are not multiples of each other (e.g. $\displaystyle \pmatrix{1\cr 0\cr}$ and $\displaystyle \pmatrix{0\cr 1\cr}$) and obtain a fundamental set. However, if there is only one eigenvalue from which we get only one solution, we are missing a solution. In that case we can fall back on the reduction to a second-order equation, which will supply a second solution involving $t {\rm e}^{rt}$ as well as ${\rm e}^{rt}$.

Example: Let $\displaystyle A = \pmatrix{4 & -1\cr 1 & 2\cr}$. The characteristic polynomial is $(4-r)(2-r) + 1 = r^2 - 6 r + 9 = (r-3)^2$, so there is only one eigenvalue $r=3$. The eigenvalue equations say

\begin{displaymath}
\begin{array}{rl}
4 u_1 - u_2 &= 3 u_1\cr
u_1 + 2 u_2 &= 3 u_2\cr
\end{array}\end{displaymath}

so $u_1 = u_2$, and we have one solution $\displaystyle {\bf x}= \pmatrix{1\cr 1\cr} {\rm e}^{3t}$. But the system corresponds to the second-order equation $x_1'' - 6 x_1' + 9 x_1 = 0$ with $x_2 = - x_1' + 4 x_1$. The solution $x_1 = {\rm e}^{3t}$ of the second-order equation (with $\displaystyle x_2 = - x_1' + 4 x_1 = {\rm e}^{3t}$) is the same as the solution from the eigenvector. The second solution $x_1 = t {\rm e}^{3t}$ (with $\displaystyle x_2 = - x_1' + 4 x_1 = (t - 1) {\rm e}^{3t}$) gives us the second solution for our fundamental set: $\displaystyle {\bf x}= \pmatrix{t\cr t-1\cr} {\rm e}^{3t}$.


The next example has complex eigenvalues.

Example: Let $\displaystyle A = \pmatrix{3 & -5\cr 10 & 5\cr}$.

The characteristic polynomial is $(3-r)(5-r)+50 = r^2 - 8 r + 65$, which has complex roots $4 \pm 7 i$. The eigenvector equations for the eigenvalue $4 + 7i$ are

\begin{displaymath}
\begin{array}{rl}
3 u_1 - 5 u_2 &= (4 + 7 i) u_1\\
10 u_1 + 5 u_2 &= (4 + 7 i) u_2\\
\end{array}\end{displaymath}

or

\begin{displaymath}
\begin{array}{rl}
(-1-7i) u_1 - 5 u_2 &= 0 \\
10 u_1 + (1 - 7 i) u_2 &= 0\\
\end{array}\end{displaymath}

It may not be obvious that these equations say the same thing, but they must if we haven't made a mistake (in fact, the second is $(-1+7i)/5$ times the first). To avoid fractions, take $u_1 = 5$ and then $u_2 = -1-7i$, i.e. one solution is $\displaystyle {\bf x}= \pmatrix{5\cr -1-7i\cr} {\rm e}^{(4+7i)t}$.

Now we could repeat this procedure for the second eigenvalue $4-7i$, but it's not really necessary: the result (if we again choose $u_1 = 5$) would be the complex conjugate of this one. If we're interested in real solutions (which we often are), we can use an extension of the Principle of Equating Real Parts:


If the matrix $A$ has real entries, the real and imaginary parts of a solution of ${\bf x}' = A {\bf x}$ are solutions.


Writing $\exp((4+7i)t) = \exp(4 t)(\cos(7 t) + i \sin(7 t))$, the real and imaginary parts of the solution above are $\displaystyle {\rm e}^{4 t} \pmatrix{5 \cos(7 t) \cr -\cos(7 t) + 7 \sin(7 t)\cr}$ and $\displaystyle {\rm e}^{4 t} \pmatrix{5 \sin(7 t)\cr -7 \cos(7 t) - \sin(7 t)\cr}$ respectively. These form a fundamental set. Thus when $A$ has real entries we need only look at one eigenvalue from a pair of complex conjugate eigenvalues, and take real and imaginary parts of the solution obtained from that eigenvalue.




Robert Israel
2002-03-19