336 lines
14 KiB
Markdown
336 lines
14 KiB
Markdown
|
# Vector spaces
|
||
|
|
||
|
## Definition
|
||
|
|
||
|
> *Definition*: a **vector space** $V$ is a set on which the operations of addition and scalar multiplication are defined. Such that for all vectors $\mathbf{u}$ and $\mathbf{v}$ in $V$ the vectors $\mathbf{u} + \mathbf{v}$ are in $V$ and for each scalar $a$ the vector $a\mathbf{v}$ is in $V$. With the following axioms satisfied.
|
||
|
>
|
||
|
> 1. Associativity of vector addition: $\mathbf{u} + (\mathbf{v} + \mathbf{w}) = (\mathbf{u} + \mathbf{v}) + \mathbf{w}$ for any $\mathbf{u},\mathbf{v}, \mathbf{w} \in V$.
|
||
|
> 2. Commutativity of vector addition: $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$ for any $\mathbf{u},\mathbf{v} \in V$.
|
||
|
> 3. Identity element of vector addition: $\exists \mathbf{0} \in V$ such that $\mathbf{v} + \mathbf{0}$ for all $\mathbf{v} \in V$.
|
||
|
> 4. Inverse element of vector addition: $\forall \mathbf{v} \in V \exists (-\mathbf{v}) \in V$ such that $\mathbf{v} + (-\mathbf{v}) = \mathbf{0}$.
|
||
|
> 5. Distributivity of scalar multiplication with respect to vector addition: $a(\mathbf{u} + \mathbf{v}) = a\mathbf{u} + a\mathbf{v}$ for any scalar $a$ and any $\mathbf{u}, \mathbf{v} \in V$.
|
||
|
> 6. Distributivity of scalar multiplication with respect to field addition: $(a + b) \mathbf{v} = a \mathbf{v} + b \mathbf{v}$ for any scalars $a$ and $b$ and any $\mathbf{v} \in V$.
|
||
|
> 7. Compatibility of scalar multiplication with field multiplication: $a(b\mathbf{v}) = (ab) \mathbf{v}$ for any scalars $a$ and $b$ and any $\mathbf{v} \in V$.
|
||
|
> 8. Identity element of scalar multiplication: $1 \mathbf{v} = \mathbf{v}$ for all $\mathbf{v} \in V$.
|
||
|
|
||
|
Some important properties of a vector space can be derived from this definition in the following proposition a few have been listed.
|
||
|
|
||
|
> *Proposition*: if $V$ is a vector space and $\mathbf{u}$, $\mathbf{v}$ is in $V$, then
|
||
|
>
|
||
|
> 1. $0 \mathbf{v} = \mathbf{0}$.
|
||
|
> 2. $\mathbf{u} + \mathbf{v} = \mathbf{0} \implies \mathbf{u} = - \mathbf{v}$.
|
||
|
> 3. $(-1)\mathbf{v} = - \mathbf{v}$.
|
||
|
|
||
|
??? note "*Proof*:"
|
||
|
|
||
|
For 1, suppose $\mathbf{v} \in V$ then it follows from axioms 3, 6 and 8
|
||
|
|
||
|
$$
|
||
|
\mathbf{v} = 1 \mathbf{v} = (1 + 0)\mathbf{v} = 1 \mathbf{v} + 0 \mathbf{v} = \mathbf{v} + 0\mathbf{v},
|
||
|
$$
|
||
|
|
||
|
therefore
|
||
|
|
||
|
$$
|
||
|
\begin{align*}
|
||
|
-\mathbf{v} + \mathbf{v} &= - \mathbf{v} + (\mathbf{v} + 0\mathbf{v}) = (-\mathbf{v} + \mathbf{v}) + 0\mathbf{v}, \\
|
||
|
\mathbf{0} &= \mathbf{0} + 0\mathbf{v} = 0\mathbf{v}.
|
||
|
\end{align*}
|
||
|
$$
|
||
|
|
||
|
For 2, suppose for $\mathbf{u}, \mathbf{v} \in V$ that $\mathbf{u} + \mathbf{v} = \mathbf{0}$ then it follows from axioms 1, 3 and 4
|
||
|
|
||
|
$$
|
||
|
- \mathbf{v} = - \mathbf{v} + \mathbf{0} = - \mathbf{v} + (\mathbf{v} + \mathbf{u}),
|
||
|
$$
|
||
|
|
||
|
therefore
|
||
|
|
||
|
$$
|
||
|
-\mathbf{v} = (-\mathbf{v} + \mathbf{v}) + \mathbf{u} = \mathbf{0} + \mathbf{u} = \mathbf{u}.
|
||
|
$$
|
||
|
|
||
|
For 3, suppose $\mathbf{v} \in V$ then it follows from 1 and axioms 4 and 6
|
||
|
|
||
|
$$
|
||
|
\mathbf{0} = 0 \mathbf{v} = (1 + (-1))\mathbf{v} = 1\mathbf{v} + (-1)\mathbf{v},
|
||
|
$$
|
||
|
|
||
|
therefore
|
||
|
|
||
|
$$
|
||
|
\mathbf{v} + (-1)\mathbf{v} = \mathbf{0},
|
||
|
$$
|
||
|
|
||
|
from 2 it follows then that
|
||
|
|
||
|
$$
|
||
|
(-1)\mathbf{v} = -\mathbf{v}.
|
||
|
$$
|
||
|
|
||
|
### Euclidean spaces
|
||
|
|
||
|
Perhaps the most elementary vector spaces are the Euclidean vector spaces $V = \mathbb{R}^n$ with $n \in \mathbb{N}$. Given a nonzero vector $\mathbf{u} \in \mathbb{R}^n$ defined by
|
||
|
|
||
|
$$
|
||
|
\mathbf{u} = \begin{pmatrix}u_1 \\ \vdots \\ u_n\end{pmatrix},
|
||
|
$$
|
||
|
|
||
|
it may be associated with the directed line segment from $(0, \dots, 0)$ to $(u_1, \dots, u_n)$. Or more generally line segments that have the same length and direction can be represented by any line segment from $(a_1, \dots, a_n)$ to $(a_1 + u_1, \dots, a_n + u_n)$. Vector addition and scalar multiplication in $\mathbb{R}^n$ are respectively defined by
|
||
|
|
||
|
$$
|
||
|
\mathbf{u} + \mathbf{v} = \begin{pmatrix} u_1 + v_1 \\ \vdots \\ u_n + v_n \end{pmatrix} \quad \text{ and } \quad a \mathbf{u} = \begin{pmatrix} a u_1 \\ \vdots \\ a u_n \end{pmatrix},
|
||
|
$$
|
||
|
|
||
|
for any $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$ and any scalar $a$.
|
||
|
|
||
|
This can be extended to matrices with $V = \mathbb{R}^{m \times n}$ with $m,n \in \mathbb{N}$, the set of all matrices. Given a nonzero matrix $A \in \mathbb{R}^{m \times n}$ defined by $A = (a_{ij})$. Matrix addition and scalar multiplication in $\mathbb{R}^{m \times n}$ are respectively defined by
|
||
|
|
||
|
$$
|
||
|
A + B = C \iff a_{ij} + b_{ij} = c_{ij} \quad \text{ and } \quad \alpha A = C \iff \alpha a_{ij} = c_{ij},
|
||
|
$$
|
||
|
|
||
|
for any $A, B, C \in \mathbb{R}^{m \times n}$ and any scalar $\alpha$.
|
||
|
|
||
|
### Function spaces
|
||
|
|
||
|
Let $V$ be a vector space over a field $F$ and let $X$ be any set. The functions $X \to F$ can be given the structure of a vector space over $F$ where the operations are defined by
|
||
|
|
||
|
$$
|
||
|
\begin{align*}
|
||
|
(f + g)(x) = f(x) + g(x), \\
|
||
|
(af)(x) = af(x),
|
||
|
\end{align*}
|
||
|
$$
|
||
|
|
||
|
for any $f,g: X \to F$, any $x \in X$ and any $a \in F$.
|
||
|
|
||
|
### Polynomial spaces
|
||
|
|
||
|
Let $P_n$ denote the set of all polynomials of degree less than $n \in \mathbb{N}$ where the operations are defined by
|
||
|
|
||
|
$$
|
||
|
\begin{align*}
|
||
|
(p+q)(x) = p(x) + q(x), \\
|
||
|
(ap)(x) = ap(x),
|
||
|
\end{align*}
|
||
|
$$
|
||
|
|
||
|
for any $p,q: X \to P_n$, any $x \in X$ and any $a \in P_n$.
|
||
|
|
||
|
## Vector subspaces
|
||
|
|
||
|
> *Definition*: if $S$ is a nonempty subset of a vector space $V$ and $S$ satisfies the conditions
|
||
|
>
|
||
|
> 1. $a \mathbf{u} \in S$ whenever $\mathbf{u} \in S$ for any scalar $a$.
|
||
|
> 2. $\mathbf{u} + \mathbf{v} \in S$ whenever $\mathbf{u}, \mathbf{v} \in S$.
|
||
|
>
|
||
|
> then $S$ is said to be a **subspace** of $V$.
|
||
|
|
||
|
In a vector space $V$ it can be readily verified that $\{\mathbf{0}\}$ and $V$ are subspaces of $V$. All other subspaces are referred to as *proper subspaces* and $\{\mathbf{0}\}$ is referred to as the *zero subspace*.
|
||
|
|
||
|
> *Theorem*: Every subspace of a vector space is a vector space.
|
||
|
|
||
|
??? note "*Proof*:"
|
||
|
|
||
|
May be proved by testing if all axioms remain valid for the definition of a subspace.
|
||
|
|
||
|
|
||
|
### The null space of a matrix
|
||
|
|
||
|
> *Definition*: let $A \in \mathbb{R}^{m \times n}$, $\mathbf{x} \in \mathbb{R}^n$ and let $N(A)$ denote the set of all solutions of the homogeneous system $A\mathbf{x} = \mathbf{0}$. Therefore
|
||
|
>
|
||
|
> $$
|
||
|
> N(A) = \{\mathbf{x} \in \mathbb{R}^n \;|\; A \mathbf{x} = \mathbf{0}\},
|
||
|
> $$
|
||
|
>
|
||
|
> referred to as the null space of $A$.
|
||
|
|
||
|
Claiming that $N(A)$ is a subspace of $\mathbb{R}^n$. Clearly $\mathbf{0} \in N(A)$ so $N(A)$ is nonempty. If $\mathbf{x} \in N(A)$ and $\alpha$ is a scalar then
|
||
|
|
||
|
$$
|
||
|
A(\alpha \mathbf{x}) = \alpha A\mathbf{x} = \alpha \mathbf{0} = \mathbf{0}
|
||
|
$$
|
||
|
|
||
|
and hence $\alpha \mathbf{x} \in N(A)$. If $\mathbf{x}, \mathbf{y} \in N(A)$ then
|
||
|
|
||
|
$$
|
||
|
A(\mathbf{x} + \mathbf{y}) = A\mathbf{x} + A\mathbf{y} = \mathbf{0} + \mathbf{0} = \mathbf{0}
|
||
|
$$
|
||
|
|
||
|
therefore $\mathbf{x} + \mathbf{y} \in N(A)$ and it follows that $N(A)$ is a subspace of $\mathbb{R}^n$.
|
||
|
|
||
|
|
||
|
### The span of a set of vectors
|
||
|
|
||
|
> *Definition*: let $\mathbf{v}_1, \dots, \mathbf{v}_n$ be vectors in a vector space $V$ with $n \in \mathbb{N}$. A sum of the form
|
||
|
>
|
||
|
> $$
|
||
|
> a_1 \mathbf{v}_1 + \dots + a_n \mathbf{v}_n,
|
||
|
> $$
|
||
|
>
|
||
|
> with scalars $a_1, \dots, a_n$ is called a **linear combination** of $\mathbf{v}_1, \dots, \mathbf{v}_n$.
|
||
|
>
|
||
|
> The set of all linear combinations of $\mathbf{v}_1, \dots, \mathbf{v}_n$ is called the **span** of $\mathbf{v}_1, \dots, \mathbf{v}_n$ which is denoted by $\text{Span}(\mathbf{v}_1, \dots, \mathbf{v}_n)$.
|
||
|
|
||
|
The nullspace can be for example defined by a span of vectors.
|
||
|
|
||
|
> *Theorem*: if $\mathbf{v}_1, \dots, \mathbf{v}_n$ are vectors in a vector space $V$ with $n \in \mathbb{N}$ then $\text{Span}(\mathbf{v}_1, \dots, \mathbf{v}_n)$ is a subspace of $V$.
|
||
|
|
||
|
??? note "*Proof*:"
|
||
|
|
||
|
Let $b$ be a scalar and $\mathbf{u} \in \text{Span}(\mathbf{v}_1, \dots, \mathbf{v}_n)$ given by
|
||
|
|
||
|
$$
|
||
|
a_1 \mathbf{v}_1 + \dots + a_n \mathbf{v}_n,
|
||
|
$$
|
||
|
|
||
|
with scalars $a_1, \dots, a_n$. Since
|
||
|
|
||
|
$$
|
||
|
b \mathbf{u} = (b a_1)\mathbf{v}_1 + \dots + (b a_n)\mathbf{v}_n,
|
||
|
$$
|
||
|
|
||
|
it follows that $b \mathbf{u} \in \text{Span}(\mathbf{v}_1, \dots, \mathbf{v}_n)$.
|
||
|
|
||
|
If we also have $\mathbf{w} \in \text{Span}(\mathbf{v}_1, \dots, \mathbf{v}_n)$ given by
|
||
|
|
||
|
$$
|
||
|
b_1 \mathbf{v}_1 + \dots + b_n \mathbf{v}_n,
|
||
|
$$
|
||
|
|
||
|
with scalars $b_1, \dots, b_n$. Then
|
||
|
|
||
|
$$
|
||
|
\mathbf{u} + \mathbf{w} = (a_1 + b_1) \mathbf{v}_1 + \dots + (a_n + b_n)\mathbf{v}_n,
|
||
|
$$
|
||
|
|
||
|
it follows that $\mathbf{u} + \mathbf{w} \in \text{Span}(\mathbf{v}_1, \dots, \mathbf{v}_n)$ is a subspace of $V$.
|
||
|
|
||
|
For example, a vector $\mathbf{x} \in \mathbb{R}^3$ is in $\text{Span}(\mathbf{e}_1, \mathbf{e}_2)$ if and only if it lies in the $x_1 x_2$-plane in 3-space. Thus we can think of the $x_1 x_2$-plane as the geometrical representation of the subspace $\text{Span}(\mathbf{e}_1, \mathbf{e}_2)$.
|
||
|
|
||
|
> *Definition*: the set $\{\mathbf{v}_1, \dots, \mathbf{v}_n\}$ with $n \in \mathbb{N}$ is a spanning set for $V$ if and only if every vector $V$ can be written as a linear combination of $\mathbf{v}_1, \dots, \mathbf{v}_n$.
|
||
|
|
||
|
## Linear independence
|
||
|
|
||
|
We have the following observations.
|
||
|
|
||
|
> *Proposition*: if $\mathbf{v}_1, \dots, \mathbf{v}_n$ with $n \in \mathbb{N}$ span a vector space $V$ and one of these vectors can be written as a linear combination of the other $n-1$ vectors then thoses $n-1$ vectors span $V$.
|
||
|
|
||
|
??? note "*Proof*:"
|
||
|
|
||
|
suppose $\mathbf{v}_n$ with $n \in \mathbb{N}$ can be written as a linear combination of the vectors $\mathbf{v}_1, \dots, \mathbf{v}_{n-1}$ given by
|
||
|
|
||
|
$$
|
||
|
\mathbf{v}_n = a_1 \mathbf{v}_1 + \dots + a_{n-1} \mathbf{v}_{n-1}.
|
||
|
$$
|
||
|
|
||
|
Let $\mathbf{v}$ be any element of $V$. Since we have
|
||
|
|
||
|
$$
|
||
|
\begin{align*}
|
||
|
\mathbf{v} &= b_1 \mathbf{v}_1 + \dots + b_{n-1} \mathbf{v}_{n-1} + b_n \mathbf{v}_n, \\
|
||
|
&= b_1 \mathbf{v}_1 + \dots + b_{n-1} \mathbf{v}_{n-1} + b_n (a_1 \mathbf{v}_1 + \dots + a_{n-1} \mathbf{v}_{n-1}), \\
|
||
|
&= (b_1 + b_n a_1)\mathbf{v}_1 + \dots + (b_{n-1} + b_n a_{n-1}) \mathbf{v}_{n-1},
|
||
|
\end{align*}
|
||
|
$$
|
||
|
|
||
|
we can write any vector $\mathbf{v} \in V$ as a linear combination of $\mathbf{v}_1, \dots, \mathbf{v}_{n-1}$ and hence these vectors span $V$.
|
||
|
|
||
|
> *Proposition*: given $n$ vectors $\mathbf{v}_1, \dots, \mathbf{v}_n$ with $n \in \mathbb{N}$, it is possible to write one of the vectors as a linear combination of the other $n-1$ vectors if and only if there exist scalars $a_1, \dots, a_n$ not all zero such that
|
||
|
>
|
||
|
> $$
|
||
|
> a_1 \mathbf{v}_1 + \dots + a_n \mathbf{v}_n = \mathbf{0}.
|
||
|
> $$
|
||
|
|
||
|
??? note "*Proof*:"
|
||
|
|
||
|
Suppose that one of the vectors $\mathbf{v}_1, \dots, \mathbf{v}_n$ with $n \in \mathbb{N}$ can be written as a linear combination of the others
|
||
|
|
||
|
$$
|
||
|
\mathbf{v}_n = a_1 \mathbf{v}_1 + \dots + a_{n-1} \mathbf{v}_{n-1}.
|
||
|
$$
|
||
|
|
||
|
Subtracting $\mathbf{v}_n$ from both sides obtains
|
||
|
|
||
|
$$
|
||
|
a_1 \mathbf{v}_1 + \dots + a_{n-1} \mathbf{v}_{n-1} - \mathbf{v}_n = \mathbf{0},
|
||
|
$$
|
||
|
|
||
|
we have $a_n = -1$ and
|
||
|
|
||
|
$$
|
||
|
a_1 \mathbf{v}_1 + \dots + a_n\mathbf{v}_n = \mathbf{0}.
|
||
|
$$
|
||
|
|
||
|
We may use these oberservations to state the following definitions.
|
||
|
|
||
|
> *Definition*: the vectors $\mathbf{v}_1, \dots, \mathbf{v}_n$ in a vector space $V$ with $n \in \mathbb{N}$ are said to be **linearly independent** if
|
||
|
>
|
||
|
> $$
|
||
|
> a_1 \mathbf{v}_1 + \dots + a_n \mathbf{v}_n = \mathbf{0} \implies \forall i \in \{1, \dots, n\} [c_i = 0].
|
||
|
> $$
|
||
|
|
||
|
It follows from the above propositions that if $\{\mathbf{v}_1, \dots, \mathbf{v}_n\}$ is a minimal spanning set of a vector space $V$ then $\mathbf{v}_1, \dots, \mathbf{v}_n$ are linearly independent. A minimal spanning set is called a basis of the vector space.
|
||
|
|
||
|
> *Definition*: the vectors $\mathbf{v}_1, \dots, \mathbf{v}_n$ in a vector space $V$ with $n \in \mathbb{N}$ are said to be **linearly dependent** if there exists scalars $a_1, \dots, a_n$ not all zero such that
|
||
|
>
|
||
|
> $$
|
||
|
> a_1 \mathbf{v}_1 + \dots + a_n \mathbf{v}_n = \mathbf{0}.
|
||
|
> $$
|
||
|
|
||
|
It follows from the above propositions that if a set of vectors is linearly dependent then at least one vector is a linear combination of the other vectors.
|
||
|
|
||
|
> *Theorem*: let $\mathbf{x}_1, \dots, \mathbf{x}_n$ be vectors in $\mathbb{R}^n$ with $n \in \mathbb{N}$ and let $X = (\mathbf{x}_1, \dots, \mathbf{x}_n)$. The vectors $\mathbf{x}_1, \dots, \mathbf{x}_n$ will be linearly dependent if and only if $X$ is singular.
|
||
|
|
||
|
??? note "*Proof*:"
|
||
|
|
||
|
Let $\mathbf{x}_1, \dots, \mathbf{x}_n$ be vectors in $\mathbb{R}^n$ with $n \in \mathbb{N}$ and let $X = (\mathbf{x}_1, \dots, \mathbf{x}_n)$. Suppose we have the linear combination given by
|
||
|
|
||
|
$$
|
||
|
a_1 \mathbf{x}_1 + \dots + a_n \mathbf{x}_n = \mathbf{0},
|
||
|
$$
|
||
|
|
||
|
can be rewritten as a matrix equation by
|
||
|
|
||
|
$$
|
||
|
X\mathbf{a} = \mathbf{0},
|
||
|
$$
|
||
|
|
||
|
with $\mathbf{a} = (a_1, \dots, a_n)^T$. This equation will have a nontrivial solution if and only if $X$ is singular. Therefore $\mathbf{x}_1, \dots, \mathbf{x}_n$ will be linearly dependent if and only if $X$ is singular.
|
||
|
|
||
|
This result can be used to test whether $n$ vectors are linearly independent in $\mathbb{R}^n$ for $n \in \mathbb{N}$.
|
||
|
|
||
|
> *Theorem*: let $\mathbf{v}_1, \dots, \mathbf{v}_n$ be vectors in a vector space $V$ with $n \in \mathbb{N}$. A vector $\mathbf{v} \in \text{Span}(\mathbf{v}_1, \dots, \mathbf{v}_n)$ can be written uniquely as a linear combination of $\mathbf{v}_1, \dots, \mathbf{v}_n$ if and only if $\mathbf{v}_1, \dots, \mathbf{v}_n$ are linearly independent.
|
||
|
|
||
|
??? note "*Proof*:"
|
||
|
|
||
|
If $\mathbf{v} \in \text{Span}(\mathbf{v}_1, \dots \mathbf{v}_n)$ with $n \in \mathbb{N}$ then $\mathbf{v}$ can be written as a linear combination
|
||
|
|
||
|
$$
|
||
|
\mathbf{v} = a_1 \mathbf{v}_1 + \dots + a_n \mathbf{v}_n.
|
||
|
$$
|
||
|
|
||
|
Suppose that $\mathbf{v}$ can also be expressed as a linear combination
|
||
|
|
||
|
$$
|
||
|
\mathbf{v} = b_1 \mathbf{v}_1 + \dots + b_n \mathbf{v}_n.
|
||
|
$$
|
||
|
|
||
|
If $\mathbf{v}_1, \dots \mathbf{v}_n$ are linearly independent then subtracting both expressions yields
|
||
|
|
||
|
$$
|
||
|
(a_1 - b_1)\mathbf{v}_1 + \dots + (a_n - b_n)\mathbf{v}_n = \mathbf{0}.
|
||
|
$$
|
||
|
|
||
|
By the linear independence of $\mathbf{v}_1, \dots \mathbf{v}_n$, the coefficients must all be 0, hence
|
||
|
|
||
|
$$
|
||
|
a_1 = b_1,\; \dots \;, a_n = b_n
|
||
|
$$
|
||
|
|
||
|
therefore the representation of $\mathbf{v}$ is unique when $\mathbf{v}_1, \dots \mathbf{v}_n$ are linearly independent.
|
||
|
|
||
|
On the other hand if $\mathbf{v}_1, \dots \mathbf{v}_n$ are linearly dependent then the coefficients must not all be 0 and $a_i \neq b_i$ for some $i \in \{1, \dots, n\}$. Therefore the representation of $\mathbf{v}$ is not unique when $\mathbf{v}_1, \dots \mathbf{v}_n$ are linearly dependent.
|
||
|
|