2.6. Vector Spaces

Vector Spaces¶

When we first introduced vectors back in Chapter 2.1, I mentioned that they support two key operations: addition and scalar multiplication. These two operations combine through linear combinations, as we’ve seen repeatedly throughout Chapter 2.

So far, we’ve mostly dealt with Euclidean vectors, which are the vectors that we’ve treated as ordered lists of numbers in $\mathbb{R}^n$. (The term “Euclidean” refers to the fact that the vectors have some geometric meaning; we have thought of a vector as an arrow from the origin to a point in space. The term Euclidean is named after the Greek mathematician Euclid, who was one of the pioneers of the study of geometry.)

However, it’s sometimes useful to consider vectors in a more abstract setting. Euclidean vectors are not the only mathematica objects that support addition and scalar multiplication!

Together, the two properties above tell us that any linear combination of vectors in $V$ is also a vector in $V$.

$\mathbb{R}^2$ is a vector space: if you take any two vectors in $\mathbb{R}^2$ and add them together, you will still have a vector in $\mathbb{R}^2$. Multiply a vector in $\mathbb{R}^2$ by a scalar, and you’ll still have a vector in $\mathbb{R}^2$.

The same holds for $\mathbb{R}^3$, $\mathbb{R}^4$, and more generally, $\mathbb{R}^n$. These are all examples of vector spaces.

What might be a vector space that isn’t made up of our regular Euclidean vectors? One possible vector space is the set of all polynomials with a degree less than or equal to 3 (i.e. degree “at most 3”). This vector space, denoted by $\mathcal{P}_3$, is the set of all polynomials of the form

where $a_3, a_2, a_1, a_0$ are real numbers (potentially equal to 0).

$u(x)$, $v(x)$, and $w(x)$ are all elements of $\mathcal{P}_3$; think of each of these polynomials as a vector in $\mathcal{P}_3$.

Why is $\mathcal{P}_3$ a vector space? If you add any two polynomials with a degree of at most 3 together, you will still have a polynomial with degree at most 3. Likewise, if you multiply a polynomial with degree at most 3 by a scalar, you will still have a polynomial with degree at most 3.

There are other examples of abstract vector spaces, too, like the set of all matrices with real entries, or the set of all continuous functions from $\mathbb{R}$ to $\mathbb{R}$. In the definition of a vector space, I mentioned that the objects in the space should support addition and scalar multiplication. We know how these operations work for our traditional definition of vectors, but what do these operations look like for generic mathematical objects?

Technically, we could be even more abstract and not restrict the scalars to be real numbers, and instead allow them to be complex numbers. Such a vector space is called a complex vector space. But, we’ll stick with real vector spaces in this class, where the scalars involved in scalar multiplication are real numbers.

And even more practically, we’ll stick to focusing on the vector space $\mathbb{R}^n$. Still, I wanted to provide you with some context for how to generalize these ideas.

Subspaces¶

As I emphasized above, we’ll stick to working with vector spaces of the form $\mathbb{R}^n$, made up of Euclidean vectors.

A theme that we’ve studied over the past few sections is that of a subspace: a subset of a vector space.

• The span of one vector in $\mathbb{R}^2$ is a line through the origin, which is a 1-dimensional subspace of $\mathbb{R}^2$.

• The span of two non-collinear vectors in $\mathbb{R}^3$ is a plane through the origin, which is a 2-dimensional subspace of $\mathbb{R}^3$.

Let me be a bit more precise with what a subspace is, given our new framing of vector spaces.

The simpler way of thinking of Properties 2 and 3 above is that if you take any two vectors $\vec u, \vec v \in S$, then any linear combination $c\vec u + d\vec v$ must also be in $S$. If you combine elements in a subspace, they can’t “escape” the subspace.

Properties 2 and 3 above sound familiar, since they were the defining properties of a vector space. Indeed, a subspace is a vector space on its own, it’s just that it’s also a subset of a larger vector space, meaning all of its elements are also in $V$. The “sub” in “subspace” means “under” or “within”. A subspace can also just be the entire vector space itself – $\mathbb{R}^2$ is a subspace of $\mathbb{R}^2$, for example – which is why I’ll usually phrase definitions and theorems in terms of subspaces, rather than vector spaces.

Sometimes, other math texts will use set notation to denote subspaces, like $S \subseteq V$, but I’ll try and avoid unnecessary notation like that here.

Many years ago, one of the great math teachers I had used the following example to explain what the term closure meant:

If you put two elephants in a cage, and one day a third animal comes out, that third animal must still be an elephant. - J. Bruce White, 1938-2025

The point is not just that elephants are animals, but that if you take any two elephants and add them together, you’ll still have an elephant.

Let’s look at several possible subspaces of larger vector spaces, and see if they are subspaces (and if not, why not).

Example: First Component is 1 ❌¶

Is the set of all vectors in $\mathbb{R}^2$ whose first component is 1 a subspace of $\mathbb{R}^2$? (Notice that when I ask whether a set is a subspace, I have to specify the vector space it’s possibly a subspace of.)

In set notation, I could write this set as

When I’m presented with a set like this, I like to think of some example vectors in the set, and see if those examples satisfy the properties of a subspace. If those examples satisfy the properties, then I’ll try and prove that the properties hold for any two vectors in the set.

This set contains vectors like $\begin{bmatrix} 1 \\ 2 \end{bmatrix}$ and $\begin{bmatrix} 1 \\ -5 \end{bmatrix}$. Remember, for this set to be a subspace, it must be closed under addition and scalar multiplication. But, if I add these two example vectors together, I get

which is not a vector in the set, since its first component is not 1. So, this set is not closed under addition, and so it is not a subspace of $\mathbb{R}^2$. I don’t need to prove anything in a more general form here, since I’ve found a counterexample.

Also, this set doesn’t contain the zero vector $\begin{bmatrix} 0 \\ 0 \end{bmatrix}$, which it needs to in order to be a subspace.

Example: First Component is 0 ✅¶

Is the set of all vectors in $\mathbb{R}^2$ whose first component is 0 a subspace of $\mathbb{R}^2$? This is the set

This set does contain the zero vector $\begin{bmatrix} 0 \\ 0 \end{bmatrix}$, which is a good start.

Some example vectors in this set are $\begin{bmatrix} 0 \\ 2 \end{bmatrix}$ and $\begin{bmatrix} 0 \\ -5 \end{bmatrix}$. Adding these vectors together gives $\begin{bmatrix} 0 \\ -3 \end{bmatrix}$, which is also in the set.

That’s a good start – but can we verify that any linear combination of vectors in the set is another vector in the set?

To do so, let’s pick two arbitrary vectors in the set, $\vec u = \begin{bmatrix} 0 \\ x \end{bmatrix}$ and $\vec v = \begin{bmatrix} 0 \\ y \end{bmatrix}$. A linear combination of these two vectors is $c \vec u + d \vec v$, or

No matter what scalars $c$ and $d$ we pick, and no matter what the values of scalars $x$ and $y$ are, the sum $c \vec u + d \vec v$ is always a vector in $\mathbb{R}^2$ with a first component of 0 and a second component of $cx + dy$. In other words, any linear combination of vectors in the set is also a vector in the set. So, this set is closed under addition and scalar multiplication, and so it is a subspace of $\mathbb{R}^2$.

Example: Unit Vectors ❌¶

Is the set of all unit vectors in $\mathbb{R}^5$ a subspace of $\mathbb{R}^5$? (There’s nothing special about $\mathbb{R}^5$; I just wanted to move away from examples in $\mathbb{R}^2$.) This is the set

This is not a subspace of $\mathbb{R}^5$, which we can identify quickly by realizing it doesn’t contain the zero vector $\begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix}$.

If we instead changed the set to be all vectors in $\mathbb{R}^5$ with a norm of less than or equal to 1, i.e.

the resulting set would contain the zero vector in $\mathbb{R}^5$, but still wouldn’t be a subspace, since it’s not closed under addition or scalar multiplication. $\begin{bmatrix} 1 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix}$ is in the set, but if we multiply it by 2, we get $\begin{bmatrix} 2 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix}$, which is not in the set.

Example: Non-Negative Coordinates ❌¶

Is the set of all vectors in $\mathbb{R}^2$ with non-negative coordinates a subspace of $\mathbb{R}^2$? This is the set

Graphically, this set represents the top-right quadrant (also known as the first quadrant) of the $xy$-plane. If we add two vectors in this set, the third vector will also be in the set, since the sum of two non-negative numbers is non-negative. However, if we multiply a vector in this set by a negative scalar, the resulting vector will not be in the set.

Example: Line through Origin ✅¶

Consider the set of all vectors in $\mathbb{R}^3$ that lie on the line spanned by $\begin{bmatrix} 1 \\ -3 \\ 2 \end{bmatrix}$. Is this a set a subspace of $\mathbb{R}^3$? This is the set

Visually, this line looks like

If we pick any two vectors on this line, any of their linear combinations will also lie on this line. Any vector on this line is a scalar multiple of $\begin{bmatrix} 1 \\ -3 \\ 2 \end{bmatrix}$, so two arbitrary vectors on this line might look like

Then, a linear combination of $\color{orange}\vec u$ and $\color{#3d81f6}\vec v$ is

which is just another vector on the line! So, since the set is closed under linear combinations, it is a subspace of $\mathbb{R}^3$.

Spans are Subspaces¶

The last example we walked through above, $\{{ \begin{bmatrix} 1 \\ -3 \\ 2 \end{bmatrix} \mid t \in \mathbb{R} \}}$, is a subspace of $\mathbb{R}^3$. It is also the span of the vector $\begin{bmatrix} 1 \\ -3 \\ 2 \end{bmatrix}$. It turns out that this correspondence between spans and subspaces is true in general.

This is true essentially by the definition of a span. The span of a collection of vectors is defined to be the set of all of their possible linear combinations. If you pick any two vectors in the span, they’re both made of the same building blocks (spanning vectors), so if you make a linear combination of these two new vectors, that’ll still be made up of the same building blocks.

Example: Span of Two Vectors in $\mathbb{R}^3$ ✅¶

As an example, consider the span of the vectors

$\text{span}(\{{\color{orange}\vec u}, {\color{#3d81f6}\vec v}\})$ contains all vectors of the form $c{\color{orange}\vec u} + d{\color{#3d81f6}\vec v}$. Let’s pick two such vectors, say

Both $\vec x$ and $\vec y$ are in $\text{span}(\{{\color{orange}\vec u}, {\color{#3d81f6}\vec v}\})$. Now, let’s take a linear combination of these two new vectors, say, $5 \vec x - \frac{1}{2} \vec y$, and check if that’s also in $\text{span}(\{{\color{orange}\vec u}, {\color{#3d81f6}\vec v}\})$. This linear combination is

But, simplifying this expression shows us that it’s equivalent to

which clearly shows that a linear combination of $\vec x$ and $\vec y$ is really just a linear combination of $\color{orange} \vec u$ and $\color{#3d81f6} \vec v$.

All of this is to say, if you pick any two vectors in $\text{span}(\{{\color{orange}\vec u}, {\color{#3d81f6}\vec v}\})$, their linear combinations will also be in $\text{span}(\{{\color{orange}\vec u}, {\color{#3d81f6}\vec v}\})$, so $\text{span}(\{{\color{orange}\vec u}, {\color{#3d81f6}\vec v}\})$ is closed under addition and scalar multiplication. It also contains the zero vector (since you could take $0{\color{orange}\vec u} + 0{\color{#3d81f6}\vec v} = \vec 0$), so this span is a subspace of $\mathbb{R}^3$.

Let’s not forget that the span of two non-collinear vectors in $\mathbb{R}^3$ is a plane. So, since $\text{span}(\{{\color{orange}\vec u}, {\color{#3d81f6}\vec v}\})$ is a subspace, the corresponding plane is also a subspace of $\mathbb{R}^3$ (because it’s the same thing as the span of the vectors).

Chapter 2.5 shows us how to find the equation of a plane in standard form ($ax + by + cz + d = 0$) given two non-collinear vectors in the plane. That specific equation in this case happens to be $15x - 3y + 4z = 0$, which means the set of vectors $\begin{bmatrix} x \\ y \\ z \end{bmatrix}$ that satisfy this equation form a subspace of $\mathbb{R}^3$.

Example: Plane $x + y + z = 0$ ✅¶

What if we’re instead given a plane, rather than the span of a collection of vectors?

It turns out that not only is every span a subspace, but every subspace can be spanned by some collection of vectors. The plane $x + y + z = 0$ is a subspace of $\mathbb{R}^3$. Can we write it as the span of a collection of vectors? Sure we can. We know that it only takes two non-collinear vectors to span a plane, so all we need to do is pick two vectors that lie in $x + y + z = 0$.

Arbitrarily, let’s pick the vectors $\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}$ and $\begin{bmatrix} 2 \\ 1 \\ -3 \end{bmatrix}$. These vectors aren’t collinear (as a reminder, this means that one is not a scalar multiple of the other), so they span a plane.

So, the plane $x + y + z = 0$ is the span of the vectors $\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}$ and $\begin{bmatrix} 2 \\ 1 \\ -3 \end{bmatrix}$, and so it’s a subspace of $\mathbb{R}^3$. These are not the only two vectors that span this plane; there are infinitely many pairs of vectors that can be said to span this plane. I just picked a pair that happens to have nice numbers.

I’ve plotted the plane below, without the vectors $\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}$ and $\begin{bmatrix} 2 \\ 1 \\ -3 \end{bmatrix}$, to emphasize that you should think of planes through the origin as subspaces in their own right.

Example: Plane $x + y + z = 5$ ❌¶

What about a plane that doesn’t pass through the origin? Is the plane $x + y + z = 5$ a subspace of $\mathbb{R}^3$?

Remember, subspaces must contain the zero vector, but the vector $\begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}$ doesn’t satisfy $x + y + z = 5$. So, the plane $x + y + z = 5$ is not a subspace of $\mathbb{R}^3$.

Equivalently, you know that the span of any set of vectors must also always contain the zero vector, since you’re always able to create the zero vector by taking 0 of each of the spanning vectors, $0{\color{#3d81f6}\vec v_1} + 0{\color{#3d81f6}\vec v_2} + \cdots + 0{\color{#3d81f6}\vec v_d} = \vec 0$. So, if a set does not contain the zero vector, it can’t be the span of any set of vectors, but as we just saw, all subspaces must be spanned by some set of vectors. If it can’t be a span, it can’t be a subspace, and so the plane $x + y + z = 5$ is not a subspace of $\mathbb{R}^3$.

Basis and Dimension¶

In Chapter 2.4, we looked at how to find linearly independent subsets with the same span as a given set of vectors. We like the vectors we’re dealing with to be linearly independent, so that any of their linear combinations are unique and can only be created in one way. (This was proved in Chapter 2.4.)

The specific example we looked at was

And, through the algorithm introduced at the bottom of that section, we found that the vectors $\vec v_1, \vec v_2, \vec v_5$ alone have the same span as the original set of 5 vectors, but are also linearly independent.

The vectors $\vec v_3$ and $\vec v_4$ can be written as linear combinations of $\vec v_1, \vec v_2, \vec v_5$, so they are not needed to describe the span of the original set of 5 vectors. In particular,

There is a special name for a set of vectors that are both linearly independent and span a subspace, like $\vec v_1, \vec v_2, \vec v_5$ above.

In the example above, the vectors $\vec v_1, \vec v_2, \vec v_5$ are a basis for the span of the original set of 5 vectors, or equivalently, a basis for the subspace of $\mathbb{R}^3$ containing all of the vectors that can be written as linear combinations of the original 5 vectors.

Think of a basis as the minimum set of vectors you need to describe every vector in a subspace:

• If you remove a vector from a basis, you’ll no longer be able to reach every vector in the subspace.

• If you add an extra vector to a basis, the new set will no longer be linearly independent.

Again, continuing with the example above, since we’ve found a basis containing 3 vectors for the span of the original set of 5 vectors, the dimension of the span of the original set of 5 vectors is 3 (not 5). So, I might say

A basis for a subspace is not unique. Sure, $\vec v_1, \vec v_2, \vec v_5$ is a basis for the span of the original set of 5 vectors, but so is $\vec v_3, \vec v_4, \vec v_5$, another collection of 3 vectors.

And, since both of these bases ($\vec v_1, \vec v_2, \vec v_5$ and $\vec v_3, \vec v_4, \vec v_5$) involve 3 linearly independent vectors in $\mathbb{R}^3$, they span all of $\mathbb{R}^3$. What’s another set of linearly independent vectors in $\mathbb{R}^3$ that span $\mathbb{R}^3$? There’s the easy choice, of $\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$.

Standard Basis for $\mathbb{R}^n$¶

We say the standard basis for $\mathbb{R}^n$ is the set of vectors $\begin{bmatrix} 1 \\ 0 \\ 0 \\ \vdots \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \\ \vdots \\ 0 \end{bmatrix}, \ldots, \begin{bmatrix} 0 \\ 0 \\ 0 \\ \vdots \\ 1 \end{bmatrix}$. Sometimes these vectors are written as $\vec e_1, \vec e_2, \ldots, \vec e_n$. In addition to being a basis for $\mathbb{R}^n$, they’re also unit vectors, and orthogonal.

Since the standard basis for $\mathbb{R}^n$ is, well, a basis, and any basis for a subspace has the same number of elements, we know that $\mathbb{R}^n$ is $n$-dimensional.

In $\mathbb{R}^2$, the standard basis consists of $\vec e_1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}$ and $\vec e_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}$. $\vec e_1$ and $\vec e_2$ are linearly independent (neither is a scalar multiple of the other), and they span all of $\mathbb{R}^2$, which means that any other vector in $\mathbb{R}^2$ can be written as a linear combination of them.

Pick your favorite vector with two components, say $\begin{bmatrix} 5 \\ -12 \end{bmatrix}$, and you’ll be able to write it as a linear combination of the standard basis.

Let’s look at a few more examples of possible bases for given subspaces.

Examples¶

Finding a Basis for a Subspace¶

Often, we’ll need to find a basis for a given subspace. If the subspace in question is the span of a set of vectors, we can use the algorithm from the bottom of Chapter 2.4 to find a basis.

given v_1, v_2, ..., v_d
initialize linearly independent set S = {v_1}
for i = 2 to d:
    if v_i is not a linear combination of S:
        add v_i to S

All this algorithm does is remove linearly dependent vectors from left to right, so that the remaining vectors are linearly independent.

For example, if

A basis for $\text{span}\left( \left\{ \vec v_1, \vec v_2, \vec v_3 \right\} \right)$ is $\left\{ \vec v_1, \vec v_2 \right\}$, since these two vectors are linearly independent and span the same subspace as the original three vectors; $\vec v_3$ is a linear combination of $\vec v_1$ and $\vec v_2$, specifically $\vec v_3 = -2 \vec v_1 - \vec v_2$.

Equivalently, $\{ \vec v_1, \vec v_3 \}$ is also a basis for the same subspace, since we can equivalently say $\vec v_2 = - 2 \vec v_1 - \vec v_3$. Either way, $\text{dim}\left( \text{span}\left( \left\{ \vec v_1, \vec v_2, \vec v_3 \right\} \right) \right) = 2$.

See the bottom of Chapter 2.4 for another worked example.

What about subspaces that aren’t expressed explicitly as a span – how do we find a basis for them? For instance, consider the subspace of $\mathbb{R}^3$ containing vectors $\begin{bmatrix} x \\ y \\ z \end{bmatrix}$ such that $2x - 3y + 4z = 0$. This is a plane through the origin, and so we know it’s spanned by two non-collinear vectors in $\mathbb{R}^3$.

Some subspaces aren’t explicitly expressed as a span, but we can still find a basis for them. We looked at an example of this earlier in this section, before we knew what a basis was. Specifically, we saw the plane of points $\begin{bmatrix} x \\ y \\ z \end{bmatrix}$ such that $x + y + z = 0$ is spanned by $\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}$ and $\begin{bmatrix} 2 \\ 1 \\ -3 \end{bmatrix}$. Since these two vectors are linearly independent, they form a basis for the subspace (a basis of dimension 2).