---
# Section 5.2: Basic Facts About Eigenvalues and Eigenvectors
---

## Complex Numbers

To discuss eigenvalues and eigenvectors, we will need to consider matrices and vectors that have complex entries,

$$a + i b,$$

where $a, b \in \mathbb{R}$ and $i^2 = -1$. The set of all complex numbers is denoted $\mathbb{C}$.

---

## Example

An example of a $2 \times 2$ matrix $A$ with complex entries is

$$
A =
\begin{bmatrix}
1 + i & -i \\
2 - i &  1
\end{bmatrix}
\in \mathbb{C}^{2 \times 2}.
$$

---

## Eigenvalues and Eigenvectors

Let $A \in \mathbb{C}^{n \times n}$. If $v$ is a _nonzero_ vector in $\mathbb{C}^n$ and $\lambda \in \mathbb{C}$ that satisfy

$$
A v = \lambda v,
$$

then $v$ is an **eigenvector** of $A$ with associated **eigenvalue** $\lambda$. In this case, we call $(\lambda, v)$ an **eigenpair**.

The **spectrum** of $A$ is the set of all eigenvalues of $A$.

---

## The Characteristic Polynomial

The **characteristic polynomial** of $A \in \mathbb{C}^{n \times n}$ is 

$$
p(\lambda) = \det(\lambda I - A).
$$

A complex number $\lambda$ is an eigenvalue of $A$ if and only if $\lambda$ is a root of the characteristic polynomial of $A$ (that is, $\det(\lambda I - A) = 0$). This is because

$$
\begin{align}
\exists v \ne 0 : A v = \lambda v
&\iff \exists v \ne 0 : (\lambda I - A) v = 0 \\
&\iff \lambda I - A \ \text{is singular} \\
&\iff \det(\lambda I - A) = 0.
\end{align}
$$

Since $p(\lambda) = \det(\lambda I - A)$ is a polynomial of degree $n$, it has $n$ roots (counting possible repeated roots),

$$ \lambda_1, \lambda_2, \ldots, \lambda_n, $$

which are the eigenvalues of the $n \times n$ matrix $A$.

---

## Eigenvalues and the Determinant

Note that

$$ \det(\lambda I - A) = (\lambda - \lambda_1)(\lambda - \lambda_2)\cdots(\lambda - \lambda_n), $$

so substituting $\lambda = 0$ we have

$$ \det(-A) = (-1)^n \lambda_1 \lambda_2 \cdots \lambda_n. $$

Since $\det(-A) = (-1)^n \det(A)$, we find that

$$ \det(A) = \lambda_1 \lambda_2 \cdots \lambda_n. $$

From this, we have

$$
\begin{align}
\text{$A$ is nonsingular}
&\iff \det(A) \ne 0 \\
&\iff \lambda_i \ne 0, \ i=1,\ldots,n.
\end{align}
$$

---

## Exercise

Compute the eigenvalues of the following matrices and use the Julia function `eigen` to check your answers.

1. $A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$

2. $A = \begin{bmatrix} 1 & 2 & 3 \\ 0 & 2 & 1 \\ 0 & 0 & 1 \end{bmatrix}$

---

> ### Theorem: (Linearly Independent Eigenvectors)
>
> Let $v_1,\ldots,v_k$ be eigenvectors of $A \in \mathbb{C}^{n \times n}$ with associated eigenvalues $\lambda_1,\ldots,\lambda_k$.
>
> If $\lambda_1,\ldots,\lambda_k$ are distinct, then $v_1,\ldots,v_n$ are linearly independent.

---

## Eigenvalue Decomposition

Let $A \in \mathbb{C}^{n \times n}$ and let $\lambda_1,\ldots,\lambda_n$ be its eigenvalues with associated eigenvectors $v_1,\ldots,v_n$.

Let

$$
V = \begin{bmatrix} v_1 & v_2 & \cdots & v_n \end{bmatrix}
\qquad \text{and} \qquad
D = \begin{bmatrix}
 \lambda_1 \\
 & \lambda_2 \\
 & & \ddots \\
 & & & \lambda_n
 \end{bmatrix}.
$$

Then $A v_i = \lambda_i v_i$, for $i=1,\ldots,n$, implies that

$$ A V = V D. $$

If $v_1,\ldots,v_n$ are linearly independent, then $V$ is a nonsingular matrix, so

$$ A = V D V^{-1}. $$

This factorization of $A$ is known as the **eigenvalue decomposition** of $A$.

---

## Similar Matrices and Diagonalization

When $A = S B S^{-1}$ for some nonsingular $S$, we say that $A$ and $B$ are **similar matrices**. If $A$ and $B$ are similar, then

$$
\det(\lambda I - A) = \det(\lambda I - B),
$$

which implies that $A$ and $B$ have exactly the same eigenvalues. We will use this fact in numerical methods for computing eigenvalues.

When $A$ is similar to a diagonal matrix $D$, we say that $A$ is **diagonalizable**. An $n \times n$ matrix $A$ is diagonalizable if and only if $A$ has $n$ linearly independent eigenvectors. By the above theorem, if $A$ has $n$ distinct eigenvalues, then $A$ is diagonalizable.

A matrix that is not diagonalizable (i.e., does not have $n$ linearly independent eigenvectors) is called **defective**. The $3 \times 3$ matrix in the previous exercise is defective.

---

## Iterative Methods

There is no closed formula for computing eigenvalues due to the fact that the problem of finding the roots of a polynomial can be posed as an eigenvalue problem.

Let

$$ p(x) = x^n + a_{n-1} x^{n-1} + \cdots + a_1 x + a_0. $$

The **companion matrix** of the polynomial $p(x)$ is

$$
A = 
\begin{bmatrix}
-a_{n-1} & -a_{n-2} & \cdots & -a_1 & -a_0 \\
 1 \\
 & 1 \\
 & & \ddots \\
 & & & 1 \\
 & & & & 1
\end{bmatrix}
$$

and has the property that $\det(\lambda I - A) = p(\lambda)$.

[Neils Henrik Abel](https://en.wikipedia.org/wiki/Niels_Henrik_Abel) showed that there is no closed formula in radicals for the roots of a polynomial with degree five and higher. So, there can be no closed formula for computing eigenvalues.

This means that we need to use an **iterative method** to compute the eigenvalues of a matrix. For example, the **power method** that we will study in the next section, is an iterative method that generates a sequence of vectors in $\mathbb{C}^n$,

$$q_1, q_2, q_3, \ldots,$$

that converges to an eigenvector of a matrix $A \in \mathbb{C}^{n \times n}$.

We say that

$$ q_j \to v \quad \text{as} \quad j \to \infty $$

if

$$ \|q_j - v\| \to 0 \quad \text{as} \quad j \to \infty.$$

Any vector norm will do here since all norms are equivalent on $\mathbb{C}^n$.

---

## Rate of Convergence

We say that the sequence $q_j$ ($j = 1, 2, \ldots$) converges to $v$ with a **linear rate of convergence** if

$$
\lim_{j \to \infty} \frac{\|q_{j+1} - v\|}{\|q_j - v\|} = r, \qquad 0 < r < 1.
$$

That is, $\|q_{j+1} - v\| \approx r \|q_j - v\|$ for all $j$ sufficiently large. The number $r$ is called the **convergence ratio**. The closer $r$ is to zero, the faster the convergence.

We say that $q_j \to v$ with a **quadratic rate of convergence** if

$$
\lim_{j \to \infty} \frac{\|q_{j+1} - v\|}{\|q_j - v\|^2} = C, \qquad C > 0.
$$

That is, $\|q_{j+1} - v\| \approx C \|q_j - v\|^2$ for all $j$ sufficiently large. Once $\|q_j - v\|$ is small enough, we expect to double the number of correct digits with each iteration.

---

ec3d0dd973b23e0db9e3f429b326a17f4827a6ec
ipynb
Jupyter Notebook
Section 5.2 - Basic Facts About Eigenvalues and Eigenvectors.ipynb
jesus-lua-8/fall2021math434
e6c6931676922193a6718b51c92232a3de7fa650
MIT
Section 5.2 - Basic Facts About Eigenvalues and Eigenvectors.ipynb
jesus-lua-8/fall2021math434
e6c6931676922193a6718b51c92232a3de7fa650
MIT
Section 5.2 - Basic Facts About Eigenvalues and Eigenvectors.ipynb
jesus-lua-8/fall2021math434
e6c6931676922193a6718b51c92232a3de7fa650
MIT
Qwen/Qwen-72B
1. YES
2. YES