Eigenvalues and Differential Equations

Click Start Over menu at the left bottom to start Back to Contents

1. Eigenvalues and Eigenvectors

Let $\mathbf{A}$ be a $n\times n$ square matrix. If a complex number $\lambda$ and a nonzero ${n\times 1}$ vector $\mathbf{x}$ satisfy

$\begin{equation} \mathbf{A} \mathbf{x} = \lambda \mathbf{x}, \end{equation} \label{eigenequation}$

then we call $\lambda$ an eigenvalue of $\mathbf{A}$ and $\mathbf{x}$ an eigenvector of $\mathbf{A}$ .

An eigenvalue $\lambda$ is the solution of $det(\mathbf{A}-\lambda\mathbf{I}) = 0$ , where $det$ stands for determinant. If $\lambda$ is an $k^{th}$ order of solution, then we say that $\lambda$ has algebraic multiplicity $\alpha_{\mathbf{A}} (\lambda) = k$ . The dimension of the eigenspace $\mathcal{E}_{\mathbf{A}}(\lambda)\equiv \{\mathbf{x} \mid \mathbf{A} \mathbf{x} = \lambda \mathbf{x} \}$ , or the number of independent eigenvectors associated with $\lambda$ are called the geometric multiplicity $\gamma_{\mathbf{A}} (\lambda)$ of $\lambda$ . If $\alpha_{\mathbf{A}} (\lambda)=1$ , $\lambda$ is called a simple eigenvalue of $\mathbf{A}$ .

Example 1. $\mathbf{I}_{2\times 2}$ has one eigenvalue 1. Its algebraic multiplicity is 2. It has two independent eigenvectors $[0, 1]^T$ and $[1, 0]^T$ . Its geometric multiplicity is 2.

A1 <- matrix(c(1, 0, 0, 1), nrow = 2)
eigen(A1)

## eigen() decomposition
## $values
## [1] 1 1
## 
## $vectors
##      [,1] [,2]
## [1,]    0   -1
## [2,]    1    0

Example 2. $\mathbf{A}_2$ has one eigenvalue 1. Its algebraic multiplicity is 2. It has one independent eigenvector $[1, 0]^T$ . So its geometric multiplicity is 1.

A2 <- matrix(c(1, 0, 1, 1), nrow = 2)
eigen(A2)

## eigen() decomposition
## $values
## [1] 1 1
## 
## $vectors
##      [,1]          [,2]
## [1,]    1 -1.000000e+00
## [2,]    0  2.220446e-16

Example 3. $\mathbf{A}_3$ has two simple eigenvalues 2 and 1. Their corresponding eigenvectors are $[0, 1]^T$ and $[1, -1]^T$ .

A3 <- matrix(c(1, 1, 0, 2), nrow = 2)
eigen(A3)

## eigen() decomposition
## $values
## [1] 2 1
## 
## $vectors
##      [,1]       [,2]
## [1,]    0  0.7071068
## [2,]    1 -0.7071068

Theorem 1. Geometric multiplicity and algebraic multiplicity can be different. But geometric multiplicity can never exceed algebraic multiplicity.

Proof. Let $\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_r$ be all the linearly independent eigenvectors associated with eigenvalue $e$ . So the geometric multiplicity of $\mathbf{A}$ is $r$ . Adding $n-r$ independent vectors $\mathbf{x}_{r+1}, \mathbf{x}_{r+2}, \cdots, \mathbf{x}_n$ to form a basis of $\mathbb{R}^n$ , we can create an inversible matrix $\mathbf{S} = [\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_r, \mathbf{x}_{r+1}, \mathbf{x}_{r+2}, \cdots, \mathbf{x}_n]$ such that

$\mathbf{S}^{-1}\mathbf{A}\mathbf{S} = \left[ \begin{array}{cc} e\mathbf{I}_{r\times r} & ? \\ \mathbf{0} & ? \end{array} \right]$

Since $det(\mathbf{S}^{-1}(\mathbf{A}-\lambda\mathbf{I})\mathbf{S})=det(\mathbf{A}-\lambda\mathbf{I})$ , the characteristic polynomial $det(\mathbf{A}-\lambda\mathbf{I})$ contains factor $(\lambda - e)^r$ . Therefore its algebraic multiplicity is not less than $r$ .

2. Generalized Eigenspaces

Let $\mathbf{A}$ be a $n\times n$ square matrix and $\lambda$ an eigenvalue of $\mathbf{A}$ . Let $\alpha_{\mathbf{A}} (\lambda)$ be the algebraic multiplicity of $\lambda$ . A nonzero ${n\times 1}$ vector $\mathbf{x}$ satisfying

$\begin{equation} (\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)} \mathbf{x} = 0 \end{equation} \label{geigenequation}$

is called a generalized eigenvector of $\mathbf{A}$ associated with eigenvalue $\lambda$ . The space $\mathcal{G}_{\mathbf{A}}(\lambda)=\{\mathbf{x} \mid (\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)} \mathbf{x} = \mathbf{0} \}$ is called the generalized eigenspace.

Theorem 2. Generalized Eigenspace Decomposition. If an $n\times n$ matrix $\mathbf{A}$ has $m$ ( $m \leq n$ ) distinct eigenvalues $\lambda_j$ , $1\leq j \leq m$ , then

$\begin{equation} \mathbb{C}^n = \mathcal{G}_{\mathbf{A}}(\lambda_1) \bigoplus \mathcal{G}_{\mathbf{A}}(\lambda_2) \bigoplus \cdots \bigoplus \mathcal{G}_{\mathbf{A}}(\lambda_m) \end{equation} \label{span}$

Corollary 1. Dimension of Generalized Eigenspaces. For any eigenvalue $\lambda$ of a matrix $\mathbf{A}$ ,

$dim(\mathcal{G}_{\mathbf{A}}(\lambda)) = \alpha_{\mathbf{A}}(\lambda)$

Proofs of Theorem 2 and Corollary 1 are not given here. But we will explain the construction of a basis in a generalized eigenspace using the Jordan chain. Suppose that $\mathbf{A}$ has an eigenvalue $\lambda$ whose geometricity is 1 and algebraic multiplicity 3. Let us compose three vectors:

$(\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_1 = \mathbf{0} \\ (\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_2 = \mathbf{x}_1 \\ (\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_3 = \mathbf{x}_2$

It is obvious that $\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3$ are in $\mathcal{G}_{\mathbf{A}}(\lambda)$ . If there is a constant vector $[c_1, c_2, c_3]^T$ such that $c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2 + c_3 \mathbf{x}_3 = \mathbf{0}$ .

$(\mathbf{A} - \lambda \mathbf{I})^2 (c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2 + c_3 \mathbf{x}_3) = c_3 \mathbf{x}_1 = \mathbf{0}$ yields $c_3 = 0$ ;

$(\mathbf{A} - \lambda \mathbf{I}) (c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2) = c_2 \mathbf{x}_1 = \mathbf{0}$ yields $c_2 = 0$ ;

$c_1 \mathbf{x}_1 = \mathbf{0}$ yields $c_1 = 0$ .

So $\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3$ are independent.

In R, we can first find a vector in the generalized eigenspace using nullspace( $(\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)}$ ) and then backward generate the Jordan chain (see Example 6).

3. Systems of Differential Equations

Consider the following first order, linear systems of ordinary differential equations with constant coefficients

$\begin{equation} \frac{d\mathbf{x}(t)}{dt}=\mathbf{A}\mathbf{x}(t), \; t > 0 \end{equation} \label{diffequation}$

Given initial value , we want to solve Eq. ( $\ref{diffequation}$ ).

Case 1. $\mathbf{A}$ has exactly $n$ independent eigenvectors.

In this case, $\mathbf{A}$ has $n$ independent eigenvectors $\mathbf{e}_1, \mathbf{e}_2, \cdots, \mathbf{e}_n$ corresponding to $n$ eigenvalues $\lambda_1, \lambda_2, \cdots, \lambda_n$ (possibly repeated). The initial value vector can be uniquely expressed as , where is the coeficient vector. Therefore the solution to Eq. ( $\ref{diffequation}$ ) can be written as

$\begin{equation} \begin{split} \mathbf{x}(t) & = e^{\mathbf{A}t}\mathbf{x}_0 \\ & = \sum_{j=1}^{n}c_j e^{\mathbf{A}t}\mathbf{e}_j \\ & = \sum_{j=1}^{n}c_j e^{\lambda_j t}\mathbf{e}_j \end{split} \end{equation} \label{case1}$

Example 4. Find the solution to the following system

$\begin{aligned} & x_1' = x_1 + 2 x_2 \;\;\; x_1 (0) = 0 \\ & x_2' = 3 x_1 + 2 x_2 \;\;\; x_2(0) = -4 \end{aligned}$

We first convert the system into matrix form:

$\frac{d}{dt}\left[\begin{array}{c} x_1(t)\\ x_2(t) \end{array} \right] = \left[ \begin{array}{cc} 1 & 2 \\ 3 & 2 \end{array} \right] \left[\begin{array}{c} x_1(t)\\ x_2(t) \end{array} \right],$

A <- matrix(c(1, 3, 2, 2), nrow = 2)
Aeig <- eigen(A)
c0 <- solve(Aeig$vectors, c(0, -4))
c0

## [1]  2.884441 -2.262742

Aeig$values

## [1]  4 -1

Aeig$vectors

##            [,1]       [,2]
## [1,] -0.5547002 -0.7071068
## [2,] -0.8320503  0.7071068

Solution

$\mathbf{x}(t)=2.884441 e^{4t} \left[\begin{array}{c} -0.5547002 \\ -0.8320503 \end{array}\right] - 2.262742 e^{-t} \left[\begin{array}{c} -0.7071068 \\ 0.7071068 \end{array}\right]$

Example 5. Solve the following initial value problem.

$\mathbf{x}' = \left[ \begin{array}{cc} 3 & 9 \\ -4 & -3 \end{array} \right]\mathbf{x} \;\;\; \mathbf{x}(0) = \left[ \begin{array}{c} 2 \\ -4 \end{array} \right]$

A <- matrix(c(3, -4, 9, -3), nrow = 2)
Aeig <- eigen(A)
c0 <- solve(Aeig$vectors, c(2, -4))
c0

## [1] 1.20185+3.469443i 1.20185-3.469443i

Aeig$values

## [1] 0+5.196152i 0-5.196152i

Aeig$vectors

##                       [,1]                  [,2]
## [1,]  0.8320503+0.0000000i  0.8320503+0.0000000i
## [2,] -0.2773501+0.4803845i -0.2773501-0.4803845i

Solution

$\mathbf{x}(t)=2.884441 e^{+5.196152it} \left[\begin{array}{c} 0.8320503+0.0000000i \\ -0.2773501+0.4803845i \end{array}\right] - 2.262742 e^{-5.196152it} \left[\begin{array}{c} 0.8320503+0.0000000i \\ -0.2773501-0.4803845i \end{array}\right]$

Case 2. $\mathbf{A}$ has less than $n$ independent eigenvectors.

Lemma 1. If $\lambda$ is an eigenvalue of $\mathbf{A}$ with algebraic multiplicity $\alpha$ , then for any generalized eigenvector $\mathbf{x}$ associated with $\lambda$

$\begin{equation} \begin{split} e^{\mathbf{A}t} \mathbf{x}=\sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\left[\sum_{k=0}^{\infty}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} \right](\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}. \end{split} \end{equation} \label{Taylor1}$

Proof. Since

$\begin{equation} \begin{split} \frac{(\mathbf{A}t)^k}{k!} \mathbf{x}& = \frac{t^k}{k!}(\lambda \mathbf{I} + \mathbf{A} - \lambda \mathbf{I})^k \mathbf{x}\\ & = \frac{t^k}{k!}\sum_{p=0}^{\alpha - 1}\frac{k!}{(k-p)!p!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}\\ & = \sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}, \end{split} \end{equation}$

$\begin{equation} \begin{split} e^{\mathbf{A}t}\mathbf{x}=\sum_{k=0}^{\infty}\frac{(\mathbf{A}t)^k}{k!} \mathbf{x} & = \sum_{k=0}^{\infty}\sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}\\ & = \sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\left[\sum_{k=0}^{\infty}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} \right](\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}. \end{split} \end{equation} \label{Taylor2}$

In this case, $\mathbf{A}$ has at least one eigenvalue whose geometric multiplicity is less than its algebraic multiplicity. Let us consider a special case where the eigenvalue $\lambda_1$ has algebraic multiplicity 3 and geometric multiplicity 1. Three independent vectors in the generalized eigenspace of $\lambda_1$ are chosen as described in Section 2, i.e.

$(\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_1 = \mathbf{0} \\ (\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_2 = \mathbf{x}_1 \\ (\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_3 = \mathbf{x}_2$

So a basis can be formed as $\{\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3, \mathbf{e}_4, \cdots, \mathbf{e}_n$ }$ corresponding to $n$ eigenvalues $\lambda_1, \lambda_1, \lambda_1, \lambda_4, \cdots, \lambda_n$ (possibly repeated). The initial value vector $\mathbf{x}_0$ can be uniquely expressed as $\mathbf{x}_0=c_1\mathbf{x}_1 + c_2\mathbf{x}_2 + c_3\mathbf{x}_3 + \sum_{j=4}^{n}c_j \mathbf{e}_j$ , where $[c_1, c_2, \cdots, c_n]^T$ is the coeficient vector.

For , using Eq. ( $\ref{Taylor1}$ ), we obtain $e^{\mathbf{A}t} \mathbf{x}_1 = e^{\lambda_1 t} \mathbf{x}_1$ .

For , using Eq. ( $\ref{Taylor1}$ ), we obtain $e^{\mathbf{A}t} \mathbf{x}_2 = e^{\lambda_1 t} \mathbf{x}_2 + t e^{\lambda_1 t} \mathbf{x}_1$ .

For , using Eq. ( $\ref{Taylor1}$ ), we obtain $e^{\mathbf{A}t} \mathbf{x}_3 = e^{\lambda_1 t} \mathbf{x}_3 + t e^{\lambda_1 t} \mathbf{x}_2 + \frac{t^2}{2!} e^{\lambda_1 t} \mathbf{x}_1$ .

Therefore the solution to Eq. ( $\ref{diffequation}$ ) can be written as

$\begin{equation} \begin{split} \mathbf{x}(t) & = e^{\mathbf{A}t}\mathbf{x}_0 \\ & = c_1 e^{\mathbf{A}t}\mathbf{x}_1 + c_2 e^{\mathbf{A}t}\mathbf{x}_2 + c_3 e^{\mathbf{A}t}\mathbf{x}_3 + \sum_{j=4}^{n}c_j e^{\mathbf{A}t}\mathbf{e}_j \\ & = (c_1 + c_2 t + c_3 \frac{t^2}{2!} )e^{\lambda_1 t}\mathbf{x}_1 + (c_2 + c_3 t) e^{\lambda_1 t}\mathbf{x}_2 + c_3 e^{\lambda_1 t}\mathbf{x}_3 + \sum_{j=4}^{n}c_j e^{\lambda_j t}\mathbf{e}_j \end{split} \end{equation} \label{case2}$

Example 6. Solve the following initial value problem.

$\mathbf{x}' = \left[ \begin{array}{cc} 7 & 1 \\ -4 & 3 \end{array} \right]\mathbf{x} \;\;\; \mathbf{x}(0) = \left[ \begin{array}{c} 2 \\ -5 \end{array} \right]$

Step 1. Find eigenvalues and eigenvectors

A <- matrix(c(7, -4, 1, 3), nrow = 2)
Aeig <- eigen(A)
Aeig$values

## [1] 5 5

Aeig$vectors

##            [,1]       [,2]
## [1,] -0.4472136  0.4472136
## [2,]  0.8944272 -0.8944272

Step 2. Find generalized eigenvectors

library(pracma)
M <- (A - Aeig$values[1] * diag(2)) %*% (A - Aeig$values[1] * diag(2))
K <- nullspace(M)
# Jordan chain
(A - Aeig$values[1] * diag(2)) %*% K

##      [,1] [,2]
## [1,]    2    1
## [2,]   -4   -2

(A - Aeig$values[1] * diag(2)) %*% (A - Aeig$values[1] * diag(2)) %*% K

##      [,1] [,2]
## [1,]    0    0
## [2,]    0    0

Step 3. Construct a basis

B <- cbind(c(1, -2), K[,2])
B

##      [,1] [,2]
## [1,]    1    0
## [2,]   -2    1

# decompose initial value
C <- solve(B, c(2, -5))
C

## [1]  2 -1

Solution

$(2 - t) e^{5t} \left[\begin{array}{c} 1 \\ -2 \end{array} \right] - e^{5t} \left[\begin{array}{c} 0 \\ 1 \end{array} \right]$

References

Kuang Z. and Yang M., On the index of the eigenvalue of a transport operator. Transport Theory and Statistical Physics 1995, 24 (9), 1411 - 1418.

https://www.statmethods.net/advstats/matrix.html

http://linear.ups.edu/scla/html/frontmatter.html

https://tutorial.math.lamar.edu/Classes/DE/DE.aspx