Eigenvalues and Differential Equations

Click Start Over menu at the left bottom to start Back to Contents


1. Eigenvalues and Eigenvectors

Let \(\mathbf{A}\) be a \(n\times n\) square matrix. If a complex number \(\lambda\) and a nonzero \({n\times 1}\) vector \(\mathbf{x}\) satisfy

\[ \begin{equation} \mathbf{A} \mathbf{x} = \lambda \mathbf{x}, \end{equation} \label{eigenequation} \]

then we call \(\lambda\) an eigenvalue of \(\mathbf{A}\) and \(\mathbf{x}\) an eigenvector of \(\mathbf{A}\).

An eigenvalue \(\lambda\) is the solution of \(det(\mathbf{A}-\lambda\mathbf{I}) = 0\), where \(det\) stands for determinant. If \(\lambda\) is an \(k^{th}\) order of solution, then we say that \(\lambda\) has algebraic multiplicity \(\alpha_{\mathbf{A}} (\lambda) = k\). The dimension of the eigenspace \(\mathcal{E}_{\mathbf{A}}(\lambda)\equiv \{\mathbf{x} \mid \mathbf{A} \mathbf{x} = \lambda \mathbf{x} \}\), or the number of independent eigenvectors associated with \(\lambda\) are called the geometric multiplicity \(\gamma_{\mathbf{A}} (\lambda)\) of \(\lambda\). If \(\alpha_{\mathbf{A}} (\lambda)=1\), \(\lambda\) is called a simple eigenvalue of \(\mathbf{A}\).

Example 1. \(\mathbf{I}_{2\times 2}\) has one eigenvalue 1. Its algebraic multiplicity is 2. It has two independent eigenvectors \([0, 1]^T\) and \([1, 0]^T\). Its geometric multiplicity is 2.

A1 <- matrix(c(1, 0, 0, 1), nrow = 2)
eigen(A1)
## eigen() decomposition
## $values
## [1] 1 1
## 
## $vectors
##      [,1] [,2]
## [1,]    0   -1
## [2,]    1    0

Example 2. \(\mathbf{A}_2\) has one eigenvalue 1. Its algebraic multiplicity is 2. It has one independent eigenvector \([1, 0]^T\). So its geometric multiplicity is 1.

A2 <- matrix(c(1, 0, 1, 1), nrow = 2)
eigen(A2)
## eigen() decomposition
## $values
## [1] 1 1
## 
## $vectors
##      [,1]          [,2]
## [1,]    1 -1.000000e+00
## [2,]    0  2.220446e-16

Example 3. \(\mathbf{A}_3\) has two simple eigenvalues 2 and 1. Their corresponding eigenvectors are \([0, 1]^T\) and \([1, -1]^T\).

A3 <- matrix(c(1, 1, 0, 2), nrow = 2)
eigen(A3)
## eigen() decomposition
## $values
## [1] 2 1
## 
## $vectors
##      [,1]       [,2]
## [1,]    0  0.7071068
## [2,]    1 -0.7071068

Theorem 1. Geometric multiplicity and algebraic multiplicity can be different. But geometric multiplicity can never exceed algebraic multiplicity.

Proof. Let \(\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_r\) be all the linearly independent eigenvectors associated with eigenvalue \(e\). So the geometric multiplicity of \(\mathbf{A}\) is \(r\). Adding \(n-r\) independent vectors \(\mathbf{x}_{r+1}, \mathbf{x}_{r+2}, \cdots, \mathbf{x}_n\) to form a basis of \(\mathbb{R}^n\), we can create an inversible matrix \(\mathbf{S} = [\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_r, \mathbf{x}_{r+1}, \mathbf{x}_{r+2}, \cdots, \mathbf{x}_n]\) such that

\[ \mathbf{S}^{-1}\mathbf{A}\mathbf{S} = \left[ \begin{array}{cc} e\mathbf{I}_{r\times r} & ? \\ \mathbf{0} & ? \end{array} \right] \]

Since \(det(\mathbf{S}^{-1}(\mathbf{A}-\lambda\mathbf{I})\mathbf{S})=det(\mathbf{A}-\lambda\mathbf{I})\), the characteristic polynomial \(det(\mathbf{A}-\lambda\mathbf{I})\) contains factor \((\lambda - e)^r\). Therefore its algebraic multiplicity is not less than \(r\).

2. Generalized Eigenspaces

Let \(\mathbf{A}\) be a \(n\times n\) square matrix and \(\lambda\) an eigenvalue of \(\mathbf{A}\). Let \(\alpha_{\mathbf{A}} (\lambda)\) be the algebraic multiplicity of \(\lambda\). A nonzero \({n\times 1}\) vector \(\mathbf{x}\) satisfying

\[ \begin{equation} (\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)} \mathbf{x} = 0 \end{equation} \label{geigenequation} \]

is called a generalized eigenvector of \(\mathbf{A}\) associated with eigenvalue \(\lambda\). The space \(\mathcal{G}_{\mathbf{A}}(\lambda)=\{\mathbf{x} \mid (\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)} \mathbf{x} = \mathbf{0} \}\) is called the generalized eigenspace.

Theorem 2. Generalized Eigenspace Decomposition. If an \(n\times n\) matrix \(\mathbf{A}\) has \(m\) (\(m \leq n\)) distinct eigenvalues \(\lambda_j\), \(1\leq j \leq m\), then

\[ \begin{equation} \mathbb{C}^n = \mathcal{G}_{\mathbf{A}}(\lambda_1) \bigoplus \mathcal{G}_{\mathbf{A}}(\lambda_2) \bigoplus \cdots \bigoplus \mathcal{G}_{\mathbf{A}}(\lambda_m) \end{equation} \label{span} \]

Corollary 1. Dimension of Generalized Eigenspaces. For any eigenvalue \(\lambda\) of a matrix \(\mathbf{A}\),

\[ dim(\mathcal{G}_{\mathbf{A}}(\lambda)) = \alpha_{\mathbf{A}}(\lambda) \]

Proofs of Theorem 2 and Corollary 1 are not given here. But we will explain the construction of a basis in a generalized eigenspace using the Jordan chain. Suppose that \(\mathbf{A}\) has an eigenvalue \(\lambda\) whose geometricity is 1 and algebraic multiplicity 3. Let us compose three vectors:

\[ (\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_1 = \mathbf{0} \\ (\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_2 = \mathbf{x}_1 \\ (\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_3 = \mathbf{x}_2 \]

It is obvious that \(\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3\) are in \(\mathcal{G}_{\mathbf{A}}(\lambda)\). If there is a constant vector \([c_1, c_2, c_3]^T\) such that \(c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2 + c_3 \mathbf{x}_3 = \mathbf{0}\).

\((\mathbf{A} - \lambda \mathbf{I})^2 (c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2 + c_3 \mathbf{x}_3) = c_3 \mathbf{x}_1 = \mathbf{0}\) yields \(c_3 = 0\);

\((\mathbf{A} - \lambda \mathbf{I}) (c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2) = c_2 \mathbf{x}_1 = \mathbf{0}\) yields \(c_2 = 0\);

\(c_1 \mathbf{x}_1 = \mathbf{0}\) yields \(c_1 = 0\).

So \(\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3\) are independent.

In R, we can first find a vector in the generalized eigenspace using nullspace(\((\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)}\)) and then backward generate the Jordan chain (see Example 6).

3. Systems of Differential Equations

Consider the following first order, linear systems of ordinary differential equations with constant coefficients

\[ \begin{equation} \frac{d\mathbf{x}(t)}{dt}=\mathbf{A}\mathbf{x}(t), \; t > 0 \end{equation} \label{diffequation} \]

Given initial value \(\mathbf{x}(0)=\mathbf{x}_0\), we want to solve Eq. (\ref{diffequation}).

Case 1. \(\mathbf{A}\) has exactly \(n\) independent eigenvectors.

In this case, \(\mathbf{A}\) has \(n\) independent eigenvectors \(\mathbf{e}_1, \mathbf{e}_2, \cdots, \mathbf{e}_n\) corresponding to \(n\) eigenvalues \(\lambda_1, \lambda_2, \cdots, \lambda_n\) (possibly repeated). The initial value vector \(\mathbf{x}_0\) can be uniquely expressed as \(\mathbf{x}_0=\sum_{j=1}^{n}c_j \mathbf{e}_j\), where \([c_1, c_2, \cdots, c_n]^T\) is the coeficient vector. Therefore the solution to Eq. (\ref{diffequation}) can be written as

\[ \begin{equation} \begin{split} \mathbf{x}(t) & = e^{\mathbf{A}t}\mathbf{x}_0 \\ & = \sum_{j=1}^{n}c_j e^{\mathbf{A}t}\mathbf{e}_j \\ & = \sum_{j=1}^{n}c_j e^{\lambda_j t}\mathbf{e}_j \end{split} \end{equation} \label{case1} \]

Example 4. Find the solution to the following system

\[ \begin{aligned} & x_1' = x_1 + 2 x_2 \;\;\; x_1 (0) = 0 \\ & x_2' = 3 x_1 + 2 x_2 \;\;\; x_2(0) = -4 \end{aligned} \]

We first convert the system into matrix form:

\[ \frac{d}{dt}\left[\begin{array}{c} x_1(t)\\ x_2(t) \end{array} \right] = \left[ \begin{array}{cc} 1 & 2 \\ 3 & 2 \end{array} \right] \left[\begin{array}{c} x_1(t)\\ x_2(t) \end{array} \right], \]

A <- matrix(c(1, 3, 2, 2), nrow = 2)
Aeig <- eigen(A)
c0 <- solve(Aeig$vectors, c(0, -4))
c0
## [1]  2.884441 -2.262742
Aeig$values
## [1]  4 -1
Aeig$vectors
##            [,1]       [,2]
## [1,] -0.5547002 -0.7071068
## [2,] -0.8320503  0.7071068

Solution

\[ \mathbf{x}(t)=2.884441 e^{4t} \left[\begin{array}{c} -0.5547002 \\ -0.8320503 \end{array}\right] - 2.262742 e^{-t} \left[\begin{array}{c} -0.7071068 \\ 0.7071068 \end{array}\right] \]

Example 5. Solve the following initial value problem.

\[ \mathbf{x}' = \left[ \begin{array}{cc} 3 & 9 \\ -4 & -3 \end{array} \right]\mathbf{x} \;\;\; \mathbf{x}(0) = \left[ \begin{array}{c} 2 \\ -4 \end{array} \right] \]

A <- matrix(c(3, -4, 9, -3), nrow = 2)
Aeig <- eigen(A)
c0 <- solve(Aeig$vectors, c(2, -4))
c0
## [1] 1.20185+3.469443i 1.20185-3.469443i
Aeig$values
## [1] 0+5.196152i 0-5.196152i
Aeig$vectors
##                       [,1]                  [,2]
## [1,]  0.8320503+0.0000000i  0.8320503+0.0000000i
## [2,] -0.2773501+0.4803845i -0.2773501-0.4803845i

Solution

\[ \mathbf{x}(t)=2.884441 e^{+5.196152it} \left[\begin{array}{c} 0.8320503+0.0000000i \\ -0.2773501+0.4803845i \end{array}\right] - 2.262742 e^{-5.196152it} \left[\begin{array}{c} 0.8320503+0.0000000i \\ -0.2773501-0.4803845i \end{array}\right] \]



Case 2. \(\mathbf{A}\) has less than \(n\) independent eigenvectors.

Lemma 1. If \(\lambda\) is an eigenvalue of \(\mathbf{A}\) with algebraic multiplicity \(\alpha\), then for any generalized eigenvector \(\mathbf{x}\) associated with \(\lambda\)

\[ \begin{equation} \begin{split} e^{\mathbf{A}t} \mathbf{x}=\sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\left[\sum_{k=0}^{\infty}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} \right](\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}. \end{split} \end{equation} \label{Taylor1} \]

Proof. Since

\[ \begin{equation} \begin{split} \frac{(\mathbf{A}t)^k}{k!} \mathbf{x}& = \frac{t^k}{k!}(\lambda \mathbf{I} + \mathbf{A} - \lambda \mathbf{I})^k \mathbf{x}\\ & = \frac{t^k}{k!}\sum_{p=0}^{\alpha - 1}\frac{k!}{(k-p)!p!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}\\ & = \sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}, \end{split} \end{equation} \]

\[ \begin{equation} \begin{split} e^{\mathbf{A}t}\mathbf{x}=\sum_{k=0}^{\infty}\frac{(\mathbf{A}t)^k}{k!} \mathbf{x} & = \sum_{k=0}^{\infty}\sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}\\ & = \sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\left[\sum_{k=0}^{\infty}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} \right](\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}. \end{split} \end{equation} \label{Taylor2} \]

In this case, \(\mathbf{A}\) has at least one eigenvalue whose geometric multiplicity is less than its algebraic multiplicity. Let us consider a special case where the eigenvalue \(\lambda_1\) has algebraic multiplicity 3 and geometric multiplicity 1. Three independent vectors in the generalized eigenspace of \(\lambda_1\) are chosen as described in Section 2, i.e.

\[ (\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_1 = \mathbf{0} \\ (\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_2 = \mathbf{x}_1 \\ (\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_3 = \mathbf{x}_2 \]

So a basis can be formed as \(\{\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3, \mathbf{e}_4, \cdots, \mathbf{e}_n\)}$ corresponding to \(n\) eigenvalues \(\lambda_1, \lambda_1, \lambda_1, \lambda_4, \cdots, \lambda_n\) (possibly repeated). The initial value vector \(\mathbf{x}_0\) can be uniquely expressed as \(\mathbf{x}_0=c_1\mathbf{x}_1 + c_2\mathbf{x}_2 + c_3\mathbf{x}_3 + \sum_{j=4}^{n}c_j \mathbf{e}_j\), where \([c_1, c_2, \cdots, c_n]^T\) is the coeficient vector.

For \(\mathbf{x}_1\), using Eq. (\ref{Taylor1}), we obtain \(e^{\mathbf{A}t} \mathbf{x}_1 = e^{\lambda_1 t} \mathbf{x}_1\).

For \(\mathbf{x}_2\), using Eq. (\ref{Taylor1}), we obtain \(e^{\mathbf{A}t} \mathbf{x}_2 = e^{\lambda_1 t} \mathbf{x}_2 + t e^{\lambda_1 t} \mathbf{x}_1\).

For \(\mathbf{x}_3\), using Eq. (\ref{Taylor1}), we obtain \(e^{\mathbf{A}t} \mathbf{x}_3 = e^{\lambda_1 t} \mathbf{x}_3 + t e^{\lambda_1 t} \mathbf{x}_2 + \frac{t^2}{2!} e^{\lambda_1 t} \mathbf{x}_1\).

Therefore the solution to Eq. (\ref{diffequation}) can be written as

\[ \begin{equation} \begin{split} \mathbf{x}(t) & = e^{\mathbf{A}t}\mathbf{x}_0 \\ & = c_1 e^{\mathbf{A}t}\mathbf{x}_1 + c_2 e^{\mathbf{A}t}\mathbf{x}_2 + c_3 e^{\mathbf{A}t}\mathbf{x}_3 + \sum_{j=4}^{n}c_j e^{\mathbf{A}t}\mathbf{e}_j \\ & = (c_1 + c_2 t + c_3 \frac{t^2}{2!} )e^{\lambda_1 t}\mathbf{x}_1 + (c_2 + c_3 t) e^{\lambda_1 t}\mathbf{x}_2 + c_3 e^{\lambda_1 t}\mathbf{x}_3 + \sum_{j=4}^{n}c_j e^{\lambda_j t}\mathbf{e}_j \end{split} \end{equation} \label{case2} \]

Example 6. Solve the following initial value problem.

\[ \mathbf{x}' = \left[ \begin{array}{cc} 7 & 1 \\ -4 & 3 \end{array} \right]\mathbf{x} \;\;\; \mathbf{x}(0) = \left[ \begin{array}{c} 2 \\ -5 \end{array} \right] \]

Step 1. Find eigenvalues and eigenvectors

A <- matrix(c(7, -4, 1, 3), nrow = 2)
Aeig <- eigen(A)
Aeig$values
## [1] 5 5
Aeig$vectors
##            [,1]       [,2]
## [1,] -0.4472136  0.4472136
## [2,]  0.8944272 -0.8944272

Step 2. Find generalized eigenvectors

library(pracma)
M <- (A - Aeig$values[1] * diag(2)) %*% (A - Aeig$values[1] * diag(2))
K <- nullspace(M)
# Jordan chain
(A - Aeig$values[1] * diag(2)) %*% K
##      [,1] [,2]
## [1,]    2    1
## [2,]   -4   -2
(A - Aeig$values[1] * diag(2)) %*% (A - Aeig$values[1] * diag(2)) %*% K
##      [,1] [,2]
## [1,]    0    0
## [2,]    0    0

Step 3. Construct a basis

B <- cbind(c(1, -2), K[,2])
B
##      [,1] [,2]
## [1,]    1    0
## [2,]   -2    1
# decompose initial value
C <- solve(B, c(2, -5))
C
## [1]  2 -1

Solution

\[ (2 - t) e^{5t} \left[\begin{array}{c} 1 \\ -2 \end{array} \right] - e^{5t} \left[\begin{array}{c} 0 \\ 1 \end{array} \right] \]

References

Kuang Z. and Yang M., On the index of the eigenvalue of a transport operator. Transport Theory and Statistical Physics 1995, 24 (9), 1411 - 1418.

https://www.statmethods.net/advstats/matrix.html

http://linear.ups.edu/scla/html/frontmatter.html

https://tutorial.math.lamar.edu/Classes/DE/DE.aspx