Added eigenspaces and thereby finished linear algebra.
This commit is contained in:
parent
2d4cc90f36
commit
8198f285b2
4 changed files with 284 additions and 4 deletions
|
|
@ -90,7 +90,7 @@ nav:
|
|||
- 'Linear transformations': mathematics/linear-algebra/linear-transformations.md
|
||||
- 'Inner product spaces': mathematics/linear-algebra/inner-product-spaces.md
|
||||
- 'Orthogonality': mathematics/linear-algebra/orthogonality.md
|
||||
- 'Diagonalization': mathematics/linear-algebra/diagonalization.md
|
||||
- 'Eigenspaces': mathematics/linear-algebra/eigenspaces.md
|
||||
- 'Calculus':
|
||||
- 'Limits': mathematics/calculus/limits.md
|
||||
- 'Continuity': mathematics/calculus/continuity.md
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
# Diagonalization
|
||||
267
docs/en/mathematics/linear-algebra/eigenspaces.md
Normal file
267
docs/en/mathematics/linear-algebra/eigenspaces.md
Normal file
|
|
@ -0,0 +1,267 @@
|
|||
# Eigenspaces
|
||||
|
||||
## Eigenvalues and eigenvectors
|
||||
|
||||
If a linear transformation is represented by an $n \times n$ matrix $A$ and there exists a nonzero vector $\mathbf{x} \in V$ such that $A \mathbf{x} = \lambda \mathbf{x}$ for some $\lambda \in \mathbb{K}$, then for this transformation $\mathbf{x}$ is a natural choice to use as a basis vector for $V$.
|
||||
|
||||
> *Definition 1*: let $A$ be a $n \times n$ matrix, a scalar $\lambda \in \mathbb{K}$ is defined as an **eigenvalue** of $A$ if and only if there exists a vector $\mathbf{x} \in V \backslash \{\mathbf{0}\}$ such that
|
||||
>
|
||||
> $$
|
||||
> A \mathbf{x} = \lambda \mathbf{x},
|
||||
> $$
|
||||
>
|
||||
> with $\mathbf{x}$ defined as an **eigenvector** belonging to $\lambda$.
|
||||
|
||||
This notion can be further generalized to a linear operator $L: V \to V$ such that
|
||||
|
||||
$$
|
||||
L(\mathbf{x}) = \lambda \mathbf{x},
|
||||
$$
|
||||
|
||||
note that $L(\mathbf{x}) = A \mathbf{x}$, which implies the similarity.
|
||||
|
||||
Furthermore it follows from the definition that any linear combination of eigenvectors is also a eigenvector of $A$.
|
||||
|
||||
> *Theorem 1*: let $A$ be a $n \times n$ matrix, a scalar $\lambda \in \mathbb{K}$ is an eigenvalue of $A$ if and only if
|
||||
>
|
||||
> $$
|
||||
> \det (A - \lambda I) = 0.
|
||||
> $$
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
A scalar $\lambda \in \mathbb{K}$ is an eigenvalue of $A$ if and only if there exists a vector $\mathbf{x} \in V \backslash \{\mathbf{0}\}$ such that
|
||||
|
||||
$$
|
||||
A \mathbf{x} = \lambda \mathbf{x},
|
||||
$$
|
||||
|
||||
obtains
|
||||
|
||||
$$
|
||||
A \mathbf{x} - \lambda \mathbf{x} = (A - \lambda I) \mathbf{x} = \mathbf{0},
|
||||
$$
|
||||
|
||||
which implies that $(A - \lambda I)$ is singular and $\det(A - \lambda I) = 0$ by [definition](..//determinants/#properties-of-determinants).
|
||||
|
||||
The eigenvalues $\lambda$ may thus be determined from the **characteristic polynomial** of degree $n$ that is obtained from $\det (A - \lambda I) = 0$. In particular, the eigenvalues are the roots of this polynomial.
|
||||
|
||||
> *Theorem 2*: let $A$ be a $n \times n$ matrix and let $\lambda \in \mathbb{K}$ be an eigenvalue of $A$. A vector $\mathbf{x} \in V$ is an eigenvector of $A$ corresponding to $\lambda$ if and only if
|
||||
>
|
||||
> $$
|
||||
> \mathbf{x} \in N(A - \lambda I) \backslash \{\mathbf{0}\}.
|
||||
> $$
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Let $A$ be a $n \times n$ matrix, $\mathbf{x} \in V$ is an eigenvector of $A$ if and only if
|
||||
|
||||
$$
|
||||
A \mathbf{x} = \lambda \mathbf{x},
|
||||
$$
|
||||
|
||||
for an eigenvalue $\lambda \in \mathbb{K}$. Therefore
|
||||
|
||||
$$
|
||||
A \mathbf{x} - \lambda \mathbf{x} = (A - \lambda I) \mathbf{x} = \mathbf{0},
|
||||
$$
|
||||
|
||||
which implies that $\mathbf{x} \in N(A - \lambda I)$.
|
||||
|
||||
Which implies that the eigenvectors can be obtained by determining the corresponding null space of $A - \lambda I$.
|
||||
|
||||
> *Definition 2*: let $L: V \to V$ be a linear operator and let $\lambda \in \mathbb{K}$ be an eigenvalue of $L$. Let the **eigenspace** $E_\lambda$ of the corresponding eigenvalue $\lambda$ be defined as
|
||||
>
|
||||
> $$
|
||||
> E_\lambda = \{\mathbf{x} \in V \;|\; L(\mathbf{x}) = \lambda \mathbf{x}\} = N(A - \lambda I),
|
||||
> $$
|
||||
>
|
||||
> with $L(\mathbf{x}) = A \mathbf{x}$.
|
||||
|
||||
It may be observed that $E_\lambda$ is a subspace of $V$ consisting of the zero vector and the eigenvectors of $L$ or $A.$
|
||||
|
||||
### Properties
|
||||
|
||||
> *Theorem 3*: if $\lambda_1, \dots, \lambda_n \in \mathbb{K}$ are distinct eigenvalues of an $n \times n$ matrix $A$ with corresponding eigenvectors $\mathbf{x}_1, \dots \mathbf{x}_k \in V\backslash \{\mathbf{0}\}$, then $\mathbf{x}_1, \dots \mathbf{x}_k$ are linearly independent.
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Will be added later.
|
||||
|
||||
If $A \in \mathbb{R}^{n \times n}$ and $A \mathbf{x} = \lambda \mathbf{x}$ for some $\mathbf{x} \in V$ and $\lambda \in \mathbb{K}$. Then
|
||||
|
||||
$$
|
||||
A \mathbf{\bar x} = \overline{A \mathbf{x}} = \overline{\lambda \mathbf{x}} = \bar \lambda \mathbf{\bar x}.
|
||||
$$
|
||||
|
||||
The complex conjugate of an eigenvector of $A$ is also an eigenvector of $A$ with an eigenvalue $\bar \lambda$.
|
||||
|
||||
> *Theorem 4*: let $A$ be a $n \times n$ matrix and let $\lambda_1, \dots, \lambda_n \in \mathbb{K}$ be the eigenvalues of $A$. It follows that
|
||||
>
|
||||
> $$
|
||||
> \det (A - \lambda I) = (\lambda_1 - \lambda)(\lambda_2 - \lambda) \cdots (\lambda_n - \lambda),
|
||||
> $$
|
||||
>
|
||||
> and
|
||||
>
|
||||
> $$
|
||||
> \det (A) = \lambda_1 \lambda_2 \cdots \lambda_n.
|
||||
> $$
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Let $A$ be a $n \times n$ matrix and let $\lambda_1, \dots, \lambda_n \in \mathbb{K}$ be the eigenvalues of $A$. It follows from the [fundamental theorem of algebra](../../number-theory/complex-numbers/#roots-of-polynomials) that
|
||||
|
||||
$$
|
||||
\det (A - \lambda I) = (\lambda_1 - \lambda)(\lambda_2 - \lambda) \cdots (\lambda_n - \lambda),
|
||||
$$
|
||||
|
||||
by taking $\lambda = 0$ it follows that
|
||||
|
||||
$$
|
||||
\det (A) = \lambda_1 \lambda_2 \cdots \lambda_n.
|
||||
$$
|
||||
|
||||
From $\det (A) = \lambda_1 \lambda_2 \cdots \lambda_n$ it must follow that
|
||||
|
||||
$$
|
||||
\mathrm{trace}(A) = \sum_{i=1}^n \lambda_i.
|
||||
$$
|
||||
|
||||
> *Theorem 5*: let $A$ and $B$ be $n \times n$ matrices. If $B$ is similar to $A$, then $A$ and $B$ have the same eigenvalues.
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Let $A$ and $B$ be similar $n \times n$ matrices, then there exists a nonsingular matrix $S$ such that
|
||||
|
||||
$$
|
||||
B = S^{-1} A S.
|
||||
$$
|
||||
|
||||
Let $\lambda \in \mathbb{K}$ be an eigenvalue of $B$ then
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
0 &= \det(B - \lambda I), \\
|
||||
&= \det(S^{-1} A S - \lambda I), \\
|
||||
&= \det(S^{-1}(A - \lambda I) S), \\
|
||||
&= \det(S^{-1}) \det(A - \lambda I) \det(S), \\
|
||||
&= \det(A - \lambda I).
|
||||
\end{align*}
|
||||
$$
|
||||
|
||||
## Diagonalization
|
||||
|
||||
> *Definition 3*: an $n \times n$ matrix $A$ is **diagonalizable** if there exists a nonsingular diagonalizing matrix $X$ and a diagonal matrix $D$ such that
|
||||
>
|
||||
> $$
|
||||
> A X = X D.
|
||||
> $$
|
||||
|
||||
We may now pose the following theorem.
|
||||
|
||||
> *Theorem 6*: an $n \times n$ matrix $A$ is diagonalizable if and only if $A$ has $n \in \mathbb{N}$ linearly independent eigenvectors.
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Will be added later.
|
||||
|
||||
It follows from the proof that the column vectors of the diagonalizing matrix $X$ are eigenvectors of $A$ and the diagonal elements of $D$ are the corresponding eigenvalues of $A$. If $A$ is diagonalizable, then
|
||||
|
||||
$$
|
||||
A = X D X^{-1},
|
||||
$$
|
||||
|
||||
it follows then that
|
||||
|
||||
$$
|
||||
A^k = X D^k X^{-1},
|
||||
$$
|
||||
|
||||
for $k \in \mathbb{K}$.
|
||||
|
||||
### Hermitian case
|
||||
|
||||
The following section is for the special case that a matrix is [Hermitian](../matrices/matrix-arithmatic/#hermitian-matrix).
|
||||
|
||||
> *Theorem 7*: the eigenvalues of a Hermitian matrix are real.
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Let $A$ be a Hermitian matrix and let $\mathbf{x} \in V \backslash \{\mathbf{0}\}$ be an eigenvector of $A$ with corresponding eigenvalue $\lambda \in \mathbb{C}$. We have
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
\lambda \mathbf{x}^H \mathbf{x} &= \mathbf{x}^H (\lambda \mathbf{x}), \\
|
||||
&= \mathbf{x}^H (A \mathbf{x}), \\
|
||||
&= (\mathbf{x}^H A) \mathbf{x}, \\
|
||||
&= (A^H \mathbf{x})^H \mathbf{x} , \\
|
||||
&= (A \mathbf{x})^H \mathbf{x}, \\
|
||||
&= (\lambda \mathbf{x})^H \mathbf{x}, \\
|
||||
&= \bar \lambda \mathbf{x}^H \mathbf{x},
|
||||
\end{align*}
|
||||
$$
|
||||
|
||||
since $\bar \lambda = \lambda$ we must have that $\lambda \in \mathbb{R}$.
|
||||
|
||||
> *Theorem 8*: the eigenvectors of a Hermitian matrix corresponding to distinct eigenvalues are orthogonal.
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Let $A$ be a Hermitian matrix and let $\mathbf{x}_1, \mathbf{x}_2 \in V \backslash \{\mathbf{0}\}$ be two eigenvectors of $A$ with corresponding eigenvalues $\lambda_1, \lambda_2 \in \mathbb{C}[\lambda_1 \neq \lambda_2]$. We have
|
||||
|
||||
$$
|
||||
\begin{align*}
|
||||
\lambda_1 \mathbf{x}_1^H \mathbf{x}_2 &= (\lambda_1 \mathbf{x}_1)^H \mathbf{x}_2, \\
|
||||
&= (A \mathbf{x}_1)^H \mathbf{x}_2, \\
|
||||
&= \mathbf{x}_1^H A^H \mathbf{x}_2, \\
|
||||
&= \mathbf{x}_1^H A \mathbf{x}_2, \\
|
||||
&= \mathbf{x}_1^H (\lambda_2 \mathbf{x}_2), \\
|
||||
&= \lambda_2 \mathbf{x}_1^H \mathbf{x}_2,
|
||||
\end{align*}
|
||||
$$
|
||||
|
||||
since $\lambda_1 \neq \lambda_2$ this must imply that $\mathbf{x}_1^H \mathbf{x}_2 = 0$, implying orthogonality in terms of the Hermite scalar product.
|
||||
|
||||
Theorem 7 and 8 impose that the following definition can be used.
|
||||
|
||||
> *Definition 4*: an $n \times n$ matrix $U$ is **unitary** if the column vectors of $U$ form an orthonormal set in $V$.
|
||||
|
||||
Thus, $U$ is unitary if and only if $U^H U = I$. Then it also follows that $U^{-1} = U^H$. A real unitary matrix is an orthogonal matrix.
|
||||
|
||||
One may observe that theorem 8 implies that the diagonalizing matrix of a Hermitian matrix $A$ is unitary when $A$ has distinct eigenvalues.
|
||||
|
||||
> *Lemma 1*: if the eigenvalues of a Hermitian matrix $A$ are distinct, then there exists a unitary matrix $U$ and a diagonal matrix $D$ such that
|
||||
>
|
||||
> $$
|
||||
> A U = U D.
|
||||
> $$
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Will be added later.
|
||||
|
||||
|
||||
With the column vectors of $U$ the eigenvectors of $A$ and the diagonal elements of $D$ the corresponding eigenvalues of $A$.
|
||||
|
||||
> *Theorem 9*: let $A$ be an $n \times n$ matrix, there exists a unitary matrix $U$ and a upper triangular matrix $T$ such that
|
||||
>
|
||||
> $$
|
||||
> A U = U T.
|
||||
> $$
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Will be added later.
|
||||
|
||||
The factorization $A = U T U^H$ is often referred to as the *Schur decomposition* of $A$.
|
||||
|
||||
> *Theorem 10*: if $A$ is Hermitian, then there exists a unitary matrix $U$ and a diagonal matrix $D$ such that
|
||||
>
|
||||
> $$
|
||||
> A U = U D.
|
||||
> $$
|
||||
|
||||
??? note "*Proof*:"
|
||||
|
||||
Will be added later.
|
||||
|
|
@ -70,10 +70,10 @@ with $A = (\mathbf{a_1}, \mathbf{a_2}, \dots, \mathbf{a_n})$.
|
|||
|
||||
## Transpose matrix
|
||||
|
||||
> *Definition*: the transpose of an $m \times n$ matrix A is the $n \times m$ matrix $B$ defined by
|
||||
> *Definition*: the **transpose** of an $m \times n$ matrix A is the $n \times m$ matrix $B$ defined by
|
||||
>
|
||||
> $$
|
||||
> b_{ji} = a_{ij}
|
||||
> b_{ji} = a_{ij},
|
||||
> $$
|
||||
>
|
||||
> for $j \in \{1, \dots, n\}$ and $i \in \{1, \dots m\}$. The transpose of $A$ is denoted by $A^T$.
|
||||
|
|
@ -82,6 +82,20 @@ with $A = (\mathbf{a_1}, \mathbf{a_2}, \dots, \mathbf{a_n})$.
|
|||
|
||||
> *Definition*: an $n \times n$ matrix $A$ is said to be **symmetric** if $A^T = A$.
|
||||
|
||||
## Hermitian matrix
|
||||
|
||||
> *Definition*: the **conjugate transpose** of an $m \times n$ matrix A is the $n \times m$ matrix $B$ defined by
|
||||
>
|
||||
> $$
|
||||
> b_{ji} = \bar a_{ij},
|
||||
> $$
|
||||
>
|
||||
> for $j \in \{1, \dots, n\}$ and $i \in \{1, \dots m\}$. The **conjugate transpose** of $A$ is denoted by $A^H$.
|
||||
|
||||
<br>
|
||||
|
||||
> *Definition*: an $n \times n$ matrix $A$ is said to be **Hermitian** if $A^H = A$.
|
||||
|
||||
## Matrix multiplication
|
||||
|
||||
> *Definition*: if $A = (a_{ij})$ is an $m \times n$ matrix and $B = (b_{ij})$ is an $n \times r$ matrix, then the product $A B = C = (c_{ij})$ is the $m \times r$ matrix whose entries are defined by
|
||||
|
|
|
|||
Loading…
Reference in a new issue