Linear Combinations

DEFINITION 3.1.10 (Linear Span)   Let $V ({\mathbb{F}})$ be a vector space and let $S = \{{\mathbf u}_1, {\mathbf u}_2, \ldots, {\mathbf u}_n \}$ be a non-empty subset of $V.$ The linear span of $S$ is the set defined by

$$L(S) = \{\alpha_1 {\mathbf u}_1 + \alpha_2 {\mathbf u}_2 + \cdots + \alpha_n {\mathbf u}_n : \alpha_i \in {\mathbb{F}}, 1 \leq i \leq n \}$$

If $S$ is an empty set we define $L(S) = \{{\mathbf 0} \}.$

EXAMPLE 3.1.11  

1. Note that $(4,5,5)$ is a linear combination of $(1,0,0),$ $(1,1,0),$ and $(1,1,1)$ as $(4,5,5) = 5 (1,1,1) - 1 (1,0,0) + 0 (1,1,0).$

   For each vector, the LINEAR COMBINATION IN TERMS OF THE VECTORS #MATH1223# MATHEND000# AND MATHEND000# IS UNIQUE.

2. Is $(4,5,5)$ a linear combination of $(1,2,3), (-1,1,4)$ and $(3,3,2)?$
   Solution: We want to find $\alpha_1, \alpha_2, \alpha_3 \in {\mathbb{R}}$ such that

   $${} \alpha_1 (1,2,3) + \alpha_2 (-1,1,4)+ \alpha_3 (3,3,2)= (4,5,5).$$ (3.1.1)

   
   

   Check that $3 (1, 2, 3) + (-1) (-1,1,4) + 0 (3,3,2) = (4, 5, 5).$ Also, in this case, the vector $(4,5,5)$ DOES NOT HAVE A UNIQUE EXPRESSION AS LINEAR COMBINATION OF VECTORS #MATH1228# MATHEND000# AND MATHEND000#.

3. Verify that $(4,5,5)$ is not a linear combination of the vectors $(1,2,1)$ and $(1,1,0)$ ?
4. The linear span of $S = \{(1,1,1), (2,1,3) \}$ over ${\mathbb{R}}$ is
   

   $$L(S) = \{\alpha (1,1,1) + \beta (2,1,3) : \alpha, \beta \in {\mathbb{R}} \}$$

   $$= \{(\alpha + 2 \beta, \alpha + \beta, \alpha + 3 \beta ) : \alpha, \beta \in {\mathbb{R}} \}$$

   $$= \{(x,y,z) \in {\mathbb{R}}^3 : 2 x - y = z \}.$$

   
   as $2 ({\alpha}+ 2 \beta) - ({\alpha}+ \beta) = {\alpha}+ 3 \beta$ , and if $z = 2 x - y$ , take ${\alpha}= 2 y - x$ and $\beta = x - y$ .

LEMMA 3.1.12 (Linear Span is a subspace)   Let $V ({\mathbb{F}})$ be a vector space and let $S$ be a non-empty subset of $V.$ Then $L(S)$ is a subspace of $V ({\mathbb{F}}).$

Proof. By definition, $S \subset L(S)$ and hence $L(S)$ is non-empty subset of $V.$ Let ${\mathbf u}, {\mathbf v}\in L(S).$ Then, for $1 \leq i \leq n$ there exist vectors ${\mathbf w}_i \in S,$ and scalars ${\alpha}_i, \beta_i \in {\mathbb{F}}$ such that ${\mathbf u}= {\alpha}_1 {\mathbf w}_1 + {\alpha}_2 {\mathbf w}_2 + \cdots + {\alpha}_n {\mathbf w}_n$ and ${\mathbf v}= \beta_1 {\mathbf w}_1 + \beta_2 {\mathbf w}_2 + \cdots + \beta_n {\mathbf w}_n.$ Hence,

$${\mathbf u}+ {\mathbf v}= ({\alpha}_1 + \beta) {\mathbf w}_1 + \cdots + ({\alpha}_n + \beta_n) {\mathbf w}_n \in L(S).$$

Thus, $L(S)$ is a vector subspace of $V ({\mathbb{F}}).$ height6pt width 6pt depth 0pt

Remark 3.1.13   Let $V ({\mathbb{F}})$ be a vector space and $W \subset V$ be a subspace. If $S \subset W,$ then $L(S) \subset W$ is a subspace of $W$ as $W$ is a vector space in its own right.

THEOREM 3.1.14   Let $S$ be a non-empty subset of a vector space $V.$ Then $L(S)$ is the smallest subspace of $V$ containing $S.$

Proof. For every ${\mathbf u}\in S, \; {\mathbf u}= 1.{\mathbf u}\in L(S)$ and therefore, $S \subseteq L(S).$ To show $L(S)$ is the smallest subspace of $V$ containing $S,$ consider any subspace $W$ of $V$ containing $S.$ Then by Proposition 3.1.13, $L(S) \subseteq W$ and hence the result follows. height6pt width 6pt depth 0pt

DEFINITION 3.1.15   Let $A$ be an $m \times n$ matrix with real entries. Then using the rows ${\mathbf a}_1^t, {\mathbf a}_2^t, \ldots, {\mathbf a}_m^t \in {\mathbb{R}}^n$ and columns ${\mathbf b}_1, {\mathbf b}_2, \ldots, {\mathbf b}_n \in {\mathbb{R}}^m,$ we define

1. ${\mathbf Row Space} (A) = L({\mathbf a}_1, {\mathbf a}_2, \ldots, {\mathbf a}_m ),$
2. ${\mathbf Column Space} (A) = L({\mathbf b}_1, {\mathbf b}_2, \ldots, {\mathbf b}_n),$
3. ${\mathbf Null Space} (A),$ denoted ${\cal N}(A)$ as $\{{\mathbf x}^{t} \in {\mathbb{R}}^n : A {\mathbf x}= {\mathbf 0}\}.$
4. ${\mathbf Range} (A),$ denoted ${\mbox{Im }}(A) = \{{\mathbf y}: A {\mathbf x}= {\mathbf y}{\mbox{ for some }} {\mathbf x}^t \in {\mathbb{R}}^n \}.$

Note that the ``column space" of a matrix $A$ consists of all ${\mathbf b}$ such that $A {\mathbf x}= {\mathbf b}$ has a solution. Hence, ${\mathbf Column Space}(A) = {\mathbf Range}(A).$

LEMMA 3.1.16   Let $A$ be a real $m \times n$ matrix. Suppose $B = E A$ for some elementary matrix $E.$ Then ${\mbox{ Row Space}}(A) = {\mbox{ Row Space}}(B).$

Proof. We prove the result for the elementary matrix $E_{ij}(c),$ where $c \neq 0$ and $i < j.$ Let ${\mathbf a}_1^t, {\mathbf a}_2^t, \ldots, {\mathbf a}_m^t$ be the rows of the matrix $A.$ Then $B = E_{ij}(c) A$ gives us

$${\mbox{ Row Space}}(B) = L({\mathbf a}_1, \ldots, {\mathbf a}_{i-1}, {\mathbf a}_i + c {\mathbf a}_j, \ldots, {\mathbf a}_m)$$

$$= \left\{ {\alpha}_1 {\mathbf a}_1+ \cdots + {\alpha}_{i-1} {\mathbf a}_{i-1}+ {\alpha}_i ({\mathbf a}_i + c {\mathbf a}_j)+ \cdots \right.$$

$$\hspace{1in} \left. + {\alpha}_m {\mathbf a}_m : \;\; {\alpha}_\ell \in {\mathbb{R}}, 1 \leq \ell \leq m \right\}$$

$$= \left\{ \sum_{\ell = 1}^m {\alpha}_\ell {\mathbf a}_\ell + {\alph... ...t c {\mathbf a}_j : {\alpha}_\ell \in {\mathbb{R}}, 1 \leq \ell \leq m \right\}$$

$$= \left\{ \sum_{\ell = 1}^m \beta_\ell {\mathbf a}_\ell : \beta_\ell \in {\mathbb{R}}, 1 \leq \ell \leq m \right\}$$

$$= L({\mathbf a}_1, \ldots, {\mathbf a}_{i-1}, {\mathbf a}_i, \ldots, {\mathbf a}_m)$$

$$= {\mbox{ Row Space}}(A)$$

height6pt width 6pt depth 0pt

THEOREM 3.1.17   Let $A$ be an $m \times n$ matrix with real entries. Then

1. ${\cal N}(A)$ is a subspace of ${\mathbb{R}}^n$ ;
2. the non-zero row vectors of a matrix in row-reduced form, forms a basis for the row-space. Hence $\dim({\mbox{ Row Space}}(A) ) = {\mbox{ row rank of }}(A).$

Proof. Part $1)$ can be easily proved. Let $A$ be an $m \times n$ matrix. For part $2),$ let $D$ be the row-reduced form of $A$ with non-zero rows ${\mathbf d}_1^t, {\mathbf d}_2^t, \ldots, {\mathbf d}_r^t.$ Then $B = E_k E_{k-1} \cdots E_2 E_1 A$ for some elementary matrices $E_1, E_2, \ldots, E_k.$ Then, a repeated application of Lemma 3.1.16 implies ${\mbox{ Row Space}}(A) = {\mbox{ Row Space}}(B).$ That is, if the rows of the matrix $A$ are ${\mathbf a}_1^t, {\mathbf a}_2^t, \ldots, {\mathbf a}_m^t,$ then

$$L({\mathbf a}_1, {\mathbf a}_2, \ldots, {\mathbf a}_m) = L({\mathbf b}_1, {\mathbf b}_2, \ldots, {\mathbf b}_r).$$

Hence the required result follows. height6pt width 6pt depth 0pt

EXERCISE 3.1.18  

1. Show that any two row-equivalent matrices have the same row space. Give examples to show that the column space of two row-equivalent matrices need not be same.
2. Find all the vector subspaces of ${\mathbb{R}}^2.$
3. Find the conditions on the real numbers $a, b, c, d$ so that the set $\{(x,y,z) \in {\mathbb{R}}^3 : \; a x + b y + c z =d \}$ is a subspace of ${\mathbb{R}}^3$ .
4. Show that the vector space ${\cal P}_5({\mathbb{R}})$ is a subspace of ${\cal P}({\mathbb{R}})$ . Further show that if $W \subset {\cal P}_5({\mathbb{R}})$ consists of all polynomials of degree $5$ , then $W$ is not a subspace.
5. Conisder the vector space $V$ given in Example 5. Determine all its vector subspaces.
6. Let $P$ and $Q$ be two subspaces of a vector space $V.$ Show that $P \cap Q$ is a subspace of $V.$ Also show that $P \cup Q$ need not be a subspace of $V.$ When is $P \cup Q$ a subspace of $V?$
7. Let $P$ and $Q$ be two subspaces of a vector space $V.$ Define $P + Q = \{{\mathbf u}+ {\mathbf v}: {\mathbf u}\in P, {\mathbf v}\in Q \}.$ Show that $P + Q$ is a subspace of $V.$ Also show that $L(P \cup Q) = P + Q.$
8. Let $S = \{x_1, x_2, x_3, x_4 \}$ where $x_1 = (1,0,0,0), \; x_2 = (1,1,0,0), \; x_3 = (1,2,0,0), \; x_4 = (1,1,1,0).$ Determine all $x_i$ such that $L(S) = L(S \setminus \{x_i\}).$
9. Let $C([-1,1])$ be the set of all continuous functions on the interval $[-1, 1]$ (cf. Example 3.1.4.12). Let
   

   $$W_1 = \{f \in C([-1,1]) : f(0.2) = 0 \}, \;\; {\mbox{and }}$$

   $$W_2 = \{f \in C([-1, 1]) : f'(\frac{1}{4}) {\mbox{exists }} \}.$$

   
   Are $W_1, W_2$ subspaces of $C([-1,1])$ ?
10. Let $V = \{(x, y) : x, y \in {\mathbb{R}}\}$ over ${\mathbb{R}}.$ Define $(x, y) \oplus (x_1, y_1) = ( x + x_1, 0)$ and $\alpha \odot (x, y) = ( \alpha x, 0).$ Show that $V$ is not a vector space over ${\mathbb{R}}$ .
11. Recall that $M_n({\mathbb{R}})$ is the real vector space of all $n \times n$ real matrices. Prove that the following subsets are subspaces of $M_n({\mathbb{R}}).$
    1. ${\mbox{sl}}_n = \{ A \in M_n({\mathbb{R}}) \; : \; {\mbox{trace}}(A) = 0 \}$
    2. ${\mbox{Sym}}_n = \{ A \in M_n({\mathbb{R}}) \; : \; A = A^t \}$
    3. ${\mbox{Skew}}_n = \{ A \in M_n({\mathbb{R}}) \; : \; A + A^t = {\mathbf 0}\}$
12. Let $V = {\mathbb{R}}.$ Define $x \oplus y = x - y$ and $\alpha \odot x = - \alpha x.$ Which vector space axioms are not satisfied here?

In this section, we saw that a vector space has infinite number of vectors. Hence, one can start with any finite collection of vectors and obtain their span. It means that any vector space contains infinite number of other vector subspaces. Therefore, the following questions arise:

1. What are the conditions under which, the linear span of two distinct sets are the same?
2. Is it possible to find/choose vectors so that the linear span of the chosen vectors is the whole vector space itself?
3. Suppose we are able to choose certain vectors whose linear span is the whole space. Can we find the minimum number of such vectors?

We try to answer these questions in the subsequent sections.