+ This notebook is part of lecture 9 *Independence, basis and dimension* in the OCW MIT course 18.06 by Prof Gilbert Strang [1]
+ Created by me, Dr Juan H Klopper
    + Head of Acute Care Surgery
    + Groote Schuur Hospital
    + University Cape Town
    + <a href="mailto:juan.klopper@uct.ac.za">Email me with your thoughts, comments, suggestions and corrections</a> 
<a rel="license" href="http://creativecommons.org/licenses/by-nc/4.0/"></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/InteractiveResource" property="dct:title" rel="dct:type">Linear Algebra OCW MIT18.06</span> <span xmlns:cc="http://creativecommons.org/ns#" property="cc:attributionName">IPython notebook [2] study notes by Dr Juan H Klopper</span> is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc/4.0/">Creative Commons Attribution-NonCommercial 4.0 International License</a>.

+ [1] <a href="http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/index.htm">OCW MIT 18.06</a>
+ [2] Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org


```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```


```python
#import numpy as np
from sympy import init_printing, Matrix, symbols
#import matplotlib.pyplot as plt
#import seaborn as sns
#from IPython.display import Image
from warnings import filterwarnings

init_printing(use_latex = 'mathjax') # Pretty Latex printing to the screen
#%matplotlib inline
filterwarnings('ignore')
```

# Independence
# Spanning
# Basis
# Dimension

## Independence

* Vectors are linearly independent if
    * No combination of these vectors results in the zero vector (except the zero combinations)
$$ { c }_{ 1 }{ x }_{ 1 }+{ c }_{ 2 }{ x }_{ 2 }+\dots +{ c }_{ n }{ x }_{ n }\neq 0,\quad { c }_{ i }\neq 0 $$
    * In 2-space, this means that they should noy be on the same line through the origin
    * In 3-space they should not be on the same line through the origin or on a plane through the origin
    * In higher-dimensional space they should not be on the same line through the origin or a hyperplane through the origin

* If they are independent by the constraints above, only a zero combination of them will results in zero
* If there are vectors in the nullspace (apart from the zero vector), then (there is a linear combination that will give zero and) then the vectors are not linearly independent


```python
A= Matrix([[1, 2, 4], [3, 1, 4]])
A
```

* Here we will have a rank of 2 (2 pivots) and 3 unknowns and 2 rows
* Thus, *r* = *m* (full row rank)
* We are left with *n* - *r* freen variable, i.e. 3 - 2 = 1 and will have one vector in the nullspace


```python
A.rref()
```


```python
A.nullspace()
```

### Another way to state independence

* Consider the columns of the matrix A as vectors v<sub>1</sub>, v<sub>2</sub>, ..., v<sub>n</sub>

* If *r* = *n* then the nullspace only contains the zero vector and the column vectors are linearly independent

## Spanning

* If we have a set of linearly independent vector that all their linear combinations (including zero) span a subspace ( in this instance a column space)

* We are particularly interested in a set of (column) vectors (in a matrix) that are linearly independent and span a subspace
* This leads us to the next topic, **basis**

## Basis

* A set of vectors (in a space *W*) with the properties
    * They are linearly independent
    * They span the space (linear combinations of them fill the space)

* Up until now we looked at columns in a matrix A
* It is more common in textbooks to look at a space first and ask about basis vectors, spanning vectors, dimension, etc

* So let's look at &#8477;<sup>3</sup>
* The obvious set of basis vectors are
$$ \hat {i}, \quad \hat {j}, \quad \hat {k} $$

* What about
$$ \begin{bmatrix} 1 \\ 1 \\ 2 \end{bmatrix},\quad \begin{bmatrix} 2 \\ 2 \\ 5 \end{bmatrix} $$
* So, are they linearly independent and do they span &#8477;<sup>3</sup>?|


```python
A = Matrix([[1, 2], [1, 2], [2, 5]])
A
```

* Here we will have *r* = 2, *n* = 2 and thus a (*n* - *r* = 0) zero nullspace


```python
A.nullspace()
```


```python
A.rref()
```

* For now, our intuition is that they will not span &#8477;<sup>3</sup>
* This intuition is correct, because all their linear combinations will only fill a plane through the origin
* The zero combination does result in the zero vector, though, so they do fill a subspace of &#8477;<sup>3</sup>
* Some textbooks refer to this as *V* = &#8477;<sup>n</sup>, with *W* a subspace of *V*

* If we added a column vector that is a linear combination of these, it will also fall in the plane and thus not be linearly independent (there will be a vector in the nullspace other than the zero vector


```python
A = Matrix([[1, 2, 3], [1, 2, 3], [2, 5, 7]])
A.nullspace()
```


```python
A.rref()
```

* Indeed, we have a column without a pivot and thus a free variable

* Let's add another, such that we have


```python
A = Matrix([[1, 2, 3], [1, 2, 3], [2, 5, 8]])
A.rref()
```

* Again, a column without a pivot and sure enough, we'll find a vector (other than the zero vector) in the nullspace


```python
A.nullspace()
```

### The special case of a square matrix

* If we now end up with a square matrix, we need only look at it's determinant, i.e., is it invertible


```python
A.det() # .det() calculates the determinant
```

* Indeed the determinant is zero as expected

## Dimension

* Given a (sub)space, every basis for that (sub)space has the same number of vectors (there are usally more than one basis for every (sub)space)
* This called the dimension of a (sub)space

## Important point to remember

* The (sub)space which a set of (column) vectors (matrix of coefficients, A) span, is the set of possible **b**-values

## More examples

#### Example problem

* Consider the column space


```python
A = Matrix([[1, 2, 3, 1], [1, 1, 2, 1], [1, 2, 3, 1]])
```


```python
A
```

* There are *n* = 4 unknowns, *m* = 3 unknowns
* We note that column 1 = column 4
* We note that with 4 unknowns we are dealing with &#8477;<sup>4</sup>
* In essence, there are at most three independent columns, thus the matrix cannot be a basis for &#8477;<sup>4</sup>
* It is possible for them to span the column space (don't get confused by column space (&#8477;<sup>4</sup>) and matrix here)


```python
A.nullspace()
```


```python
A.rref()
```

* As we can see here (columns three and four have free variables, i.e. no pivots)
$$ { x }_{ 1 }+0{ x }_{ 2 }+{ x }_{ 3 }+{ x }_{ 4 }=0\\ 0{ x }_{ 1 }+1{ x }_{ 2 }+{ x }_{ 3 }+{ 0 }_{ 4 }=0\\ { x }_{ 4 }={ c }_{ 2 }\\ { x }_{ 3 }={ c }_{ 1 }\\ \therefore \quad { x }_{ 2 }=-{ c }_{ 1 }\\ \therefore \quad { x }_{ 1 }=-{ c }_{ 1 }-{ c }_{ 2 }\\ \begin{bmatrix} { x }_{ 1 } \\ { x }_{ 2 } \\ { x }_{ 3 } \\ { x }_{ 4 } \end{bmatrix}=\begin{bmatrix} -{ c }_{ 1 }-{ c }_{ 2 } \\ -{ c }_{ 1 } \\ { c }_{ 1 } \\ { c }_{ 2 } \end{bmatrix}=\begin{bmatrix} -{ c }_{ 1 } \\ -{ c }_{ 1 } \\ { c }_{ 1 } \\ 0 \end{bmatrix}+\begin{bmatrix} -{ c }_{ 2 } \\ 0 \\ 0 \\ { c }_{ 2 } \end{bmatrix}={ c }_{ 1 }\begin{bmatrix} -1 \\ -1 \\ 1 \\ 0 \end{bmatrix}+{ c }_{ 2 }\begin{bmatrix} -1 \\ 0 \\ 0 \\ 1 \end{bmatrix} $$

* The rank of the matrix is two (it is the number of pivot columns)
* This matrix space thus have two basis vectors (column vectors 1 and 2) and we say the dimension of this space is two
* Remember, a matrix has a rank, which is the dimension of a column space (the column space representing the space 'produced' by the column vectors)
* We talk about the rank of a matrix, rank(A) and the column space of a matrix, C(A)

* In summary we have two basis above (they span a space)
    * Any two vectors that are not linearly dependent will also span this space, they can't help but to
    * dimC(A)= *r*
    * The nullspace will have *n* - *r* vectors (the dimension of the null space equal the number of free variables)


```python

```

3c431ed62539dd7358860025572f7e1b6d895a38
ipynb
Jupyter Notebook
_math/MIT_OCW_18_06_Linear_algebra/I_10_Independence_Spanning_Basis_Dimension.ipynb
aixpact/data-science
f04a54595fbc2d797918d450b979fd4c2eabac15
MIT
2020-07-22T23:12:39.000Z
2020-07-25T02:30:48.000Z
_math/MIT_OCW_18_06_Linear_algebra/I_10_Independence_Spanning_Basis_Dimension.ipynb
aixpact/data-science
f04a54595fbc2d797918d450b979fd4c2eabac15
MIT
_math/MIT_OCW_18_06_Linear_algebra/I_10_Independence_Spanning_Basis_Dimension.ipynb
aixpact/data-science
f04a54595fbc2d797918d450b979fd4c2eabac15
MIT
Qwen/Qwen-72B
1. YES
2. YES