Vector Spaces
A real vector space is a nonempty set \(V\) of vectors on which are
defined addition and scalar multiplication, defined to
satisfy the following eight axioms:
\(\forall u, v, w \in V\) and scalars \(c, d \in \mathbb{R}\),
- Commutativity of vector addition: \(u + v = v + u\)
- Associativity of vector addition: \((u + v) + w = u + (v + w)\)
- Existence of an additive identity: \(\exists 0 \in V\) s.t. \(u + 0 = u\)
- Existence of an additive inverse: \(\forall u \in V, \exists -u\) s.t. \(u + (-u) = 0\)
- Existence of a multiplicative identity: \(1u = u\)
- Distributivity of scalar multiplication with respect to vector addition : \(c(u + v) = cu + cv\)
- Distributivity of scalar multiplication with respect to scalar addditon: \((c + d)u = cu + du\)
- Compatibility of scalar multiplication: \(c(du)=(cd)u\)
From now on, the term "vector space" refers to a "real" vector space. If the context requires
a complex vector space, where scalars \(c, d \in \mathbb{C}\), it will be explicitly mentioned.
To study more general cases, you will need concepts from abstract algebra, but for now, this definition suffices.
A straightforward example of a vector space is \(\mathbb{R}^n\), where \(n \geq 1\). This is why the concept is introduced here.
By definition, \(\forall u \in V \text{and scalar } c\),
\[0u =0\]
\[c0 =0\]
\[-u =(-1)u\]
\[cu =0 \Longrightarrow c=0 \text{ or } u =0\]
A subspace of a vector space \(V\) is a subset \(H \subseteq V\) that satisfies the following properties:
- The zero vector of \(V\) is in \(H\)
- \(H\) is closed under vector addition: \(\forall u, v \in H, \quad u+v \in H\)
- \(H\) is closed under scalar multiplication: \(\forall u \in H, \text{ any scalar } c, cu \in H\)
For example, the vector space \(\mathbb{R}^2\) is NOT subspace of \(\mathbb{R}^3\) because \(\mathbb{R}^2\)
is NOT a subset of \(\mathbb{R}^3\).
Recall that a linear combination of vectors is any "sum" of "scalar multiples" of vectors. Also, the
\(\text{Span }\{v_1, v_2, \cdots, v_p\}\) represents the set of all linear combinations of \(v_1, v_2, \cdots, v_p\).
Thus, we get the following result:
Theorem 1:
If \(v_1, \cdots, v_p \in V\), then \(\text{Span }\{v_1, \cdots, v_p \}\) is a subspace of \(V\).
Null Space
The null space of \(A \in \mathbb{R}^{m \times n} \) is the set of all solutions
to the homogeneous equation \(Ax =0\).
\[
\text{Nul } A = \{x \in \mathbb{R}^n | Ax=0 \in \mathbb{R}^m \}
\]
In other words, \(\forall x \in \text{Nul } A\) are mapped to the zero vector \(0 \in \mathbb{R}^m\)
by the linear transformation \(x \mapsto Ax\). This implies that \(\text{Nul } A\) is a subspace of \(\mathbb{R}^n\).
When discussing a general linear transformation \(T: V \mapsto W\) (not necessarily a matrix transformation),
the null space is also referred to as the kernel of \(T\) denoted by \(\text{Ker } T\).
It is defined as:
\[
\text{Ker } T = \{u \in V \mid T(u) = 0 \in W\}.
\]
Consider the augmented matrix for \(Ax=0\) and its reduced row echelon form(rref).
\[
\begin{bmatrix} 2 & 4 & 6 & 8 & 0 \\ 1 & 3 & 5 & 7 & 0 \end{bmatrix}
\xrightarrow{\text{rref}}
\begin{bmatrix} 1 & 0 & -1 & -2 & 0 \\ 0 & 1 & 2 & 3 & 0 \end{bmatrix}
\]
\(x_3 \text{ and } x_4\) are free variable, and we can express the solution as:
\[
\begin{bmatrix} x_1\\ x_2\\ x_3 \\ x_4\\ \end{bmatrix}
=
x_3\begin{bmatrix} 1\\ -2\\ 1 \\ 0\\ \end{bmatrix}
+
x_4\begin{bmatrix} 2\\ -3\\ 0 \\ 1\\ \end{bmatrix}
\]
Thus, a spanning set for \(\text{Nul } A\) is
\[\left\{
\begin{bmatrix} 1\\ -2\\ 1 \\ 0\\ \end{bmatrix},
\begin{bmatrix} 2\\ -3\\ 0 \\ 1\\ \end{bmatrix}
\right\}.
\]
This means that every linear combination of these two vectors lies in \(\text{Nul } A\).
Column Space
The column space of \(A \in \mathbb{R}^{m \times n} \) is the set of all linear combinations of the
columns of \(A = \begin{bmatrix} a_1 & a_2 & \cdots & a_n \end{bmatrix}\).
\[
\begin{align*}
\text{Col } A &= \text{Span }\{a_1, a_2, \cdots, a_n\} \\\\
&= \{a \in \mathbb{R}^m \mid a = Ax, \, x \in \mathbb{R}^n \}
\end{align*}
\]
Clearly, \(\text{Col } A\) is a subspace of \(\mathbb{R}^m\) and represents the range of the
linear transformation \(x \mapsto Ax\).
Additionally, \(\text{Col } A^T\) is called the row space of \(A\) and is denoted by \(\text{Row } A\).
The row space is the set of all linear combinations of the row vectors of \(A\) and is a subspace of \(\mathbb{R}^n\).
Basis
Let \(H\) be a subspace of a vector space \(V\).
An indexed set of vectors \(\mathcal{B} = \{b_1, \cdots, b_p\} \text{ in } V\) is a basis for
\(H\) if \(\mathcal{B}\) is a linearly independent set and \(H = \text{Span }\{b_1, \cdots, b_p\}\).
For example, the columns of the identity matrix \(I_n\) form the standard basis
for \(\mathbb{R}^n\).
Another example is the set of columns of an \(n \times n\) invertible matrix, which forms a basis for \(\mathbb{R}^n\)
because, by the invertible matrix theorem, these columns are linearly independent and span \(\mathbb{R}^n\).
Previously, we found a "spanning" set for \(\text{Nul } A\):
\[
\left\{
\begin{bmatrix} 1\\ -2\\ 1 \\ 0\\ \end{bmatrix},
\begin{bmatrix} 2\\ -3\\ 0 \\ 1\\ \end{bmatrix}
\right\}
\]
This set is actually a basis for \(\text{Nul } A\).
On the other hand, a basis for \(\text{Col } A\) must be a linearly independent spanning set with
no redundant vectors. Consider:
\[
A = \begin{bmatrix} 2 & 4 & 6 & 8 \\ 1 & 3 & 5 & 7 \end{bmatrix}
\xrightarrow{\text{ref}}
\begin{bmatrix} 1 & 2 & 3 & 4 \\ 0 & 1 & 2 & 3 \end{bmatrix} = B
\]
Here, columns \(a_3\) and \(a_4\) are linear combinations of earlier columns:
\[
a_3 = -a_1 + 2a_2, \quad a_4 = -2a_1 + 3a_2.
\]
The pivot columns of \(B\) are linearly independent, and because \(A\) is row equivalent
to \(B\), the pivot columns of \(A\) (namely \(a_1\) and \(a_2\)) form a basis for \(\text{Col } A\).
Thus, the basis for \(\text{Col } A\) is:
\[
\left\{
\begin{bmatrix} 2 \\ 1 \end{bmatrix}, \,
\begin{bmatrix} 4 \\ 3 \end{bmatrix}
\right\}.
\]
Also, the representation of \(a_3\) and \(a_4\) in terms of \(a_1\) and \(a_2\) is unique.
This fact is formalized in:
Theorem 2:
Let \(\mathcal{B} = \{b_1, \cdots, b_n\}\) be a basis for a vector space \(V\).
Then, there exsits a unique set of scalars \(c_1, \cdots, c_n\) such that:
\[
x = c_1b_1 + \cdots + c_nb_n.
\]
Proof:
Since \(\mathcal{B}\) spans \(V\), there exist scalars such that
\[x = c_1b_1 + \cdots + c_nb_n \tag{1}\]
Assume that \(x\) also has another representation with different scalars \(d_1, \cdots, d_n\):
\[x = d_1b_1 + \cdots + d_nb_n \tag{2}\]
Subtracting (2) from (1):
\[x - x = (c_1 - d_1)b_1 + \cdots + (c_n-d_n)b_n\]
To hold this equation, since \(\mathcal{B}\) is linearly independent, each coefficient
must satisfy \(c_i - d_i = 0\) \((1 \leq i \leq n)\).
Thus, \(c_i = d_i\). In other words, the representation is unique.
Coordinate Systems
Let \(\mathcal{B} = \{b_1, \cdots, b_n\}\) be a basis for a vector space \(V\) and \(x \in V\).
The \(\mathcal{B}\)-coordinates of \(x\) are weights \(c_1, \cdots, c_n \) such that:
\[x = c_1b_1 + \cdots + c_nb_n.\]
The vector
\[
[x]_{\mathcal{B}} = \begin{bmatrix} c_1\\ \cdots \\ c_n \\ \end{bmatrix}
\]
is called the coordinate vector of \(x\).
The mapping \(x \mapsto [x]_{\mathcal{B}}\) is the coordinate mapping, which is
a one-to-one linear transformation from \(V\) onto \(\mathbb{R}^n\).
The coordinate mapping is isomorphism from \(V\) onto \(\mathbb{R}^n\).
It allows us to analize an "unfamiliar" vector space \(V\) via the "familiar" vector space \(\mathbb{R}^n\),
preserving all operations and structures.
Rank
The dimension of a vector space \(V\) is the number of vectors in a basis for \(V\).
The rank of a matrix \(A\) is the dimension of the column space of \(A\).
\[
\begin{align*}
\text{rank }A &= \dim (\text{Col } A) \\\\
&= \text{ the number of pivot columns of } A
\end{align*}
\]
Note that since \(\text{Row } A = \text{Col } A^T\), we get \(\dim \text{Row } A = \text{rank }A^T\).
Additionally, \(\dim \text{Nul } A \) is the number of free variables in \(Ax=0\). Equivalently,
it is the number of non-pivot colums of \(A\).
Thus, the following relationship holds:
Theorem 3:
For an \(m \times n\) matrix \(A\),
\[\text{rank }A + \dim (\text{Nul } A )= n. \]