Click Start Over menu at the left bottom to start Back to Contents
1. Eigenvalues and Eigenvectors
Let \(\mathbf{A}\) be a \(n\times n\) square matrix. If a complex number \(\lambda\) and a nonzero \({n\times 1}\) vector \(\mathbf{x}\) satisfy
\[ \begin{equation} \mathbf{A} \mathbf{x} = \lambda \mathbf{x}, \end{equation} \label{eigenequation} \]
then we call \(\lambda\) an eigenvalue of \(\mathbf{A}\) and \(\mathbf{x}\) an eigenvector of \(\mathbf{A}\).
An eigenvalue \(\lambda\) is the solution of \(det(\mathbf{A}-\lambda\mathbf{I}) = 0\), where \(det\) stands for determinant. If \(\lambda\) is an \(k^{th}\) order of solution, then we say that \(\lambda\) has algebraic multiplicity \(\alpha_{\mathbf{A}} (\lambda) = k\). The dimension of the eigenspace \(\mathcal{E}_{\mathbf{A}}(\lambda)\equiv \{\mathbf{x} \mid \mathbf{A} \mathbf{x} = \lambda \mathbf{x} \}\), or the number of independent eigenvectors associated with \(\lambda\) are called the geometric multiplicity \(\gamma_{\mathbf{A}} (\lambda)\) of \(\lambda\). If \(\alpha_{\mathbf{A}} (\lambda)=1\), \(\lambda\) is called a simple eigenvalue of \(\mathbf{A}\).
Example 1. \(\mathbf{I}_{2\times 2}\) has one eigenvalue 1. Its algebraic multiplicity is 2. It has two independent eigenvectors \([0, 1]^T\) and \([1, 0]^T\). Its geometric multiplicity is 2.
A1 <- matrix(c(1, 0, 0, 1), nrow = 2)
eigen(A1)## eigen() decomposition
## $values
## [1] 1 1
##
## $vectors
## [,1] [,2]
## [1,] 0 -1
## [2,] 1 0Example 2. \(\mathbf{A}_2\) has one eigenvalue 1. Its algebraic multiplicity is 2. It has one independent eigenvector \([1, 0]^T\). So its geometric multiplicity is 1.
A2 <- matrix(c(1, 0, 1, 1), nrow = 2)
eigen(A2)## eigen() decomposition
## $values
## [1] 1 1
##
## $vectors
## [,1] [,2]
## [1,] 1 -1.000000e+00
## [2,] 0 2.220446e-16Example 3. \(\mathbf{A}_3\) has two simple eigenvalues 2 and 1. Their corresponding eigenvectors are \([0, 1]^T\) and \([1, -1]^T\).
A3 <- matrix(c(1, 1, 0, 2), nrow = 2)
eigen(A3)## eigen() decomposition
## $values
## [1] 2 1
##
## $vectors
## [,1] [,2]
## [1,] 0 0.7071068
## [2,] 1 -0.7071068Theorem 1. Geometric multiplicity and algebraic multiplicity can be different. But geometric multiplicity can never exceed algebraic multiplicity.
Proof. Let \(\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_r\) be all the linearly independent eigenvectors associated with eigenvalue \(e\). So the geometric multiplicity of \(\mathbf{A}\) is \(r\). Adding \(n-r\) independent vectors \(\mathbf{x}_{r+1}, \mathbf{x}_{r+2}, \cdots, \mathbf{x}_n\) to form a basis of \(\mathbb{R}^n\), we can create an inversible matrix \(\mathbf{S} = [\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_r, \mathbf{x}_{r+1}, \mathbf{x}_{r+2}, \cdots, \mathbf{x}_n]\) such that
\[ \mathbf{S}^{-1}\mathbf{A}\mathbf{S} = \left[ \begin{array}{cc} e\mathbf{I}_{r\times r} & ? \\ \mathbf{0} & ? \end{array} \right] \]
Since \(det(\mathbf{S}^{-1}(\mathbf{A}-\lambda\mathbf{I})\mathbf{S})=det(\mathbf{A}-\lambda\mathbf{I})\), the characteristic polynomial \(det(\mathbf{A}-\lambda\mathbf{I})\) contains factor \((\lambda - e)^r\). Therefore its algebraic multiplicity is not less than \(r\).
2. Generalized Eigenspaces
Let \(\mathbf{A}\) be a \(n\times n\) square matrix and \(\lambda\) an eigenvalue of \(\mathbf{A}\). Let \(\alpha_{\mathbf{A}} (\lambda)\) be the algebraic multiplicity of \(\lambda\). A nonzero \({n\times 1}\) vector \(\mathbf{x}\) satisfying
\[ \begin{equation} (\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)} \mathbf{x} = 0 \end{equation} \label{geigenequation} \]
is called a generalized eigenvector of \(\mathbf{A}\) associated with eigenvalue \(\lambda\). The space \(\mathcal{G}_{\mathbf{A}}(\lambda)=\{\mathbf{x} \mid (\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)} \mathbf{x} = \mathbf{0} \}\) is called the generalized eigenspace.
Theorem 2. Generalized Eigenspace Decomposition. If an \(n\times n\) matrix \(\mathbf{A}\) has \(m\) (\(m \leq n\)) distinct eigenvalues \(\lambda_j\), \(1\leq j \leq m\), then
\[ \begin{equation} \mathbb{C}^n = \mathcal{G}_{\mathbf{A}}(\lambda_1) \bigoplus \mathcal{G}_{\mathbf{A}}(\lambda_2) \bigoplus \cdots \bigoplus \mathcal{G}_{\mathbf{A}}(\lambda_m) \end{equation} \label{span} \]
Corollary 1. Dimension of Generalized Eigenspaces. For any eigenvalue \(\lambda\) of a matrix \(\mathbf{A}\),
\[ dim(\mathcal{G}_{\mathbf{A}}(\lambda)) = \alpha_{\mathbf{A}}(\lambda) \]
Proofs of Theorem 2 and Corollary 1 are not given here. But we will explain the construction of a basis in a generalized eigenspace using the Jordan chain. Suppose that \(\mathbf{A}\) has an eigenvalue \(\lambda\) whose geometricity is 1 and algebraic multiplicity 3. Let us compose three vectors:
\[ (\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_1 = \mathbf{0} \\ (\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_2 = \mathbf{x}_1 \\ (\mathbf{A} - \lambda \mathbf{I}) \mathbf{x}_3 = \mathbf{x}_2 \]
It is obvious that \(\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3\) are in \(\mathcal{G}_{\mathbf{A}}(\lambda)\). If there is a constant vector \([c_1, c_2, c_3]^T\) such that \(c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2 + c_3 \mathbf{x}_3 = \mathbf{0}\).
\((\mathbf{A} - \lambda \mathbf{I})^2 (c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2 + c_3 \mathbf{x}_3) = c_3 \mathbf{x}_1 = \mathbf{0}\) yields \(c_3 = 0\);
\((\mathbf{A} - \lambda \mathbf{I}) (c_1 \mathbf{x}_1 + c_2 \mathbf{x}_2) = c_2 \mathbf{x}_1 = \mathbf{0}\) yields \(c_2 = 0\);
\(c_1 \mathbf{x}_1 = \mathbf{0}\) yields \(c_1 = 0\).
So \(\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3\) are independent.
In R, we can first find a vector in the generalized eigenspace using nullspace(\((\mathbf{A} - \lambda \mathbf{I})^{\alpha_{\mathbf{A}} (\lambda)}\)) and then backward generate the Jordan chain (see Example 6).
3. Systems of Differential Equations
Consider the following first order, linear systems of ordinary differential equations with constant coefficients
\[ \begin{equation} \frac{d\mathbf{x}(t)}{dt}=\mathbf{A}\mathbf{x}(t), \; t > 0 \end{equation} \label{diffequation} \]
Given initial value \(\mathbf{x}(0)=\mathbf{x}_0\), we want to solve Eq. (\ref{diffequation}).
Case 1. \(\mathbf{A}\) has exactly \(n\) independent eigenvectors.
In this case, \(\mathbf{A}\) has \(n\) independent eigenvectors \(\mathbf{e}_1, \mathbf{e}_2, \cdots, \mathbf{e}_n\) corresponding to \(n\) eigenvalues \(\lambda_1, \lambda_2, \cdots, \lambda_n\) (possibly repeated). The initial value vector \(\mathbf{x}_0\) can be uniquely expressed as \(\mathbf{x}_0=\sum_{j=1}^{n}c_j \mathbf{e}_j\), where \([c_1, c_2, \cdots, c_n]^T\) is the coeficient vector. Therefore the solution to Eq. (\ref{diffequation}) can be written as
\[ \begin{equation} \begin{split} \mathbf{x}(t) & = e^{\mathbf{A}t}\mathbf{x}_0 \\ & = \sum_{j=1}^{n}c_j e^{\mathbf{A}t}\mathbf{e}_j \\ & = \sum_{j=1}^{n}c_j e^{\lambda_j t}\mathbf{e}_j \end{split} \end{equation} \label{case1} \]
Example 4. Find the solution to the following system
\[ \begin{aligned} & x_1' = x_1 + 2 x_2 \;\;\; x_1 (0) = 0 \\ & x_2' = 3 x_1 + 2 x_2 \;\;\; x_2(0) = -4 \end{aligned} \]
We first convert the system into matrix form:
\[ \frac{d}{dt}\left[\begin{array}{c} x_1(t)\\ x_2(t) \end{array} \right] = \left[ \begin{array}{cc} 1 & 2 \\ 3 & 2 \end{array} \right] \left[\begin{array}{c} x_1(t)\\ x_2(t) \end{array} \right], \]
A <- matrix(c(1, 3, 2, 2), nrow = 2)
Aeig <- eigen(A)
c0 <- solve(Aeig$vectors, c(0, -4))
c0## [1] 2.884441 -2.262742Aeig$values## [1] 4 -1Aeig$vectors## [,1] [,2]
## [1,] -0.5547002 -0.7071068
## [2,] -0.8320503 0.7071068Solution
\[ \mathbf{x}(t)=2.884441 e^{4t} \left[\begin{array}{c} -0.5547002 \\ -0.8320503 \end{array}\right] - 2.262742 e^{-t} \left[\begin{array}{c} -0.7071068 \\ 0.7071068 \end{array}\right] \]
Example 5. Solve the following initial value problem.
\[ \mathbf{x}' = \left[ \begin{array}{cc} 3 & 9 \\ -4 & -3 \end{array} \right]\mathbf{x} \;\;\; \mathbf{x}(0) = \left[ \begin{array}{c} 2 \\ -4 \end{array} \right] \]
A <- matrix(c(3, -4, 9, -3), nrow = 2)
Aeig <- eigen(A)
c0 <- solve(Aeig$vectors, c(2, -4))
c0## [1] 1.20185+3.469443i 1.20185-3.469443iAeig$values## [1] 0+5.196152i 0-5.196152iAeig$vectors## [,1] [,2]
## [1,] 0.8320503+0.0000000i 0.8320503+0.0000000i
## [2,] -0.2773501+0.4803845i -0.2773501-0.4803845iSolution
\[ \mathbf{x}(t)=2.884441 e^{+5.196152it} \left[\begin{array}{c} 0.8320503+0.0000000i \\ -0.2773501+0.4803845i \end{array}\right] - 2.262742 e^{-5.196152it} \left[\begin{array}{c} 0.8320503+0.0000000i \\ -0.2773501-0.4803845i \end{array}\right] \]
Case 2. \(\mathbf{A}\) has less than \(n\) independent eigenvectors.
Lemma 1. If \(\lambda\) is an eigenvalue of \(\mathbf{A}\) with algebraic multiplicity \(\alpha\), then for any generalized eigenvector \(\mathbf{x}\) associated with \(\lambda\)
\[ \begin{equation} \begin{split} e^{\mathbf{A}t} \mathbf{x}=\sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\left[\sum_{k=0}^{\infty}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} \right](\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}. \end{split} \end{equation} \label{Taylor1} \]
Proof. Since
\[ \begin{equation} \begin{split} \frac{(\mathbf{A}t)^k}{k!} \mathbf{x}& = \frac{t^k}{k!}(\lambda \mathbf{I} + \mathbf{A} - \lambda \mathbf{I})^k \mathbf{x}\\ & = \frac{t^k}{k!}\sum_{p=0}^{\alpha - 1}\frac{k!}{(k-p)!p!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}\\ & = \sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}, \end{split} \end{equation} \]
\[ \begin{equation} \begin{split} e^{\mathbf{A}t}\mathbf{x}=\sum_{k=0}^{\infty}\frac{(\mathbf{A}t)^k}{k!} \mathbf{x} & = \sum_{k=0}^{\infty}\sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} (\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}\\ & = \sum_{p=0}^{\alpha - 1}\frac{t^p}{p!}\left[\sum_{k=0}^{\infty}\frac{t^{k-p}}{(k-p)!}(\lambda \mathbf{I})^{k-p} \right](\mathbf{A} - \lambda \mathbf{I})^p \mathbf{x}. \end{split} \end{equation} \label{Taylor2} \]
In this case, \(\mathbf{A}\) has at least one eigenvalue whose geometric multiplicity is less than its algebraic multiplicity. Let us consider a special case where the eigenvalue \(\lambda_1\) has algebraic multiplicity 3 and geometric multiplicity 1. Three independent vectors in the generalized eigenspace of \(\lambda_1\) are chosen as described in Section 2, i.e.
\[ (\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_1 = \mathbf{0} \\ (\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_2 = \mathbf{x}_1 \\ (\mathbf{A} - \lambda_1 \mathbf{I}) \mathbf{x}_3 = \mathbf{x}_2 \]
So a basis can be formed as \(\{\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3, \mathbf{e}_4, \cdots, \mathbf{e}_n\)}$ corresponding to \(n\) eigenvalues \(\lambda_1, \lambda_1, \lambda_1, \lambda_4, \cdots, \lambda_n\) (possibly repeated). The initial value vector \(\mathbf{x}_0\) can be uniquely expressed as \(\mathbf{x}_0=c_1\mathbf{x}_1 + c_2\mathbf{x}_2 + c_3\mathbf{x}_3 + \sum_{j=4}^{n}c_j \mathbf{e}_j\), where \([c_1, c_2, \cdots, c_n]^T\) is the coeficient vector.
For \(\mathbf{x}_1\), using Eq. (\ref{Taylor1}), we obtain \(e^{\mathbf{A}t} \mathbf{x}_1 = e^{\lambda_1 t} \mathbf{x}_1\).
For \(\mathbf{x}_2\), using Eq. (\ref{Taylor1}), we obtain \(e^{\mathbf{A}t} \mathbf{x}_2 = e^{\lambda_1 t} \mathbf{x}_2 + t e^{\lambda_1 t} \mathbf{x}_1\).
For \(\mathbf{x}_3\), using Eq. (\ref{Taylor1}), we obtain \(e^{\mathbf{A}t} \mathbf{x}_3 = e^{\lambda_1 t} \mathbf{x}_3 + t e^{\lambda_1 t} \mathbf{x}_2 + \frac{t^2}{2!} e^{\lambda_1 t} \mathbf{x}_1\).
Therefore the solution to Eq. (\ref{diffequation}) can be written as
\[ \begin{equation} \begin{split} \mathbf{x}(t) & = e^{\mathbf{A}t}\mathbf{x}_0 \\ & = c_1 e^{\mathbf{A}t}\mathbf{x}_1 + c_2 e^{\mathbf{A}t}\mathbf{x}_2 + c_3 e^{\mathbf{A}t}\mathbf{x}_3 + \sum_{j=4}^{n}c_j e^{\mathbf{A}t}\mathbf{e}_j \\ & = (c_1 + c_2 t + c_3 \frac{t^2}{2!} )e^{\lambda_1 t}\mathbf{x}_1 + (c_2 + c_3 t) e^{\lambda_1 t}\mathbf{x}_2 + c_3 e^{\lambda_1 t}\mathbf{x}_3 + \sum_{j=4}^{n}c_j e^{\lambda_j t}\mathbf{e}_j \end{split} \end{equation} \label{case2} \]
Example 6. Solve the following initial value problem.
\[ \mathbf{x}' = \left[ \begin{array}{cc} 7 & 1 \\ -4 & 3 \end{array} \right]\mathbf{x} \;\;\; \mathbf{x}(0) = \left[ \begin{array}{c} 2 \\ -5 \end{array} \right] \]
Step 1. Find eigenvalues and eigenvectors
A <- matrix(c(7, -4, 1, 3), nrow = 2)
Aeig <- eigen(A)
Aeig$values## [1] 5 5Aeig$vectors## [,1] [,2]
## [1,] -0.4472136 0.4472136
## [2,] 0.8944272 -0.8944272Step 2. Find generalized eigenvectors
library(pracma)
M <- (A - Aeig$values[1] * diag(2)) %*% (A - Aeig$values[1] * diag(2))
K <- nullspace(M)
# Jordan chain
(A - Aeig$values[1] * diag(2)) %*% K## [,1] [,2]
## [1,] 2 1
## [2,] -4 -2(A - Aeig$values[1] * diag(2)) %*% (A - Aeig$values[1] * diag(2)) %*% K## [,1] [,2]
## [1,] 0 0
## [2,] 0 0Step 3. Construct a basis
B <- cbind(c(1, -2), K[,2])
B## [,1] [,2]
## [1,] 1 0
## [2,] -2 1# decompose initial value
C <- solve(B, c(2, -5))
C## [1] 2 -1Solution
\[ (2 - t) e^{5t} \left[\begin{array}{c} 1 \\ -2 \end{array} \right] - e^{5t} \left[\begin{array}{c} 0 \\ 1 \end{array} \right] \]
References
Kuang Z. and Yang M., On the index of the eigenvalue of a transport operator. Transport Theory and Statistical Physics 1995, 24 (9), 1411 - 1418.
https://www.statmethods.net/advstats/matrix.html