---

# Chapter 4: Linear Algebra

---

## Vector
Vector may be a list of one dimension numbers with each index stored the same symantics in my understanding

\begin{equation}\vec{v_1} = \begin{bmatrix}2 \\ 1 \end{bmatrix} 
=\begin{bmatrix}v_1 \\ v_2 \end{bmatrix}
=\begin{bmatrix}v_i \\ v_j \end{bmatrix}
=\begin{bmatrix}v_x \\ v_y \end{bmatrix}
\end{equation}


\begin{equation}\vec{v_2} = \begin{bmatrix}4 \\ -3 \\2 \end{bmatrix}
=\begin{bmatrix}v_1 \\ v_2 \\ v_3 \end{bmatrix}  
=\begin{bmatrix}v_i \\ v_j \\ v_k \end{bmatrix}  
=\begin{bmatrix}v_x \\ v_y \\ v_z \end{bmatrix}  
\end{equation}

\begin{equation}\overrightarrow{grades} = \begin{bmatrix}exam_1 \\ exam_2 \\ exam_3 \\ exam_4 \end{bmatrix}
=\begin{bmatrix}95 \\ 80 \\ 75 \\ 62 \end{bmatrix}  
\end{equation}

Simple built in ``list`` is satisfied but most of the time we use numpy. In this chapter the book leads us to build our own vector data type from scratch. Create data type ``Vector`` from ``typing`` using ``List``. We will use the ``Vector`` to define the function arguements and return value. ``height_weight_age`` is a vector store heigh,weight, and age, while ``grades`` the score from exams 4 times of a one person.





```python
from typing import List

Vector = List[float]

height_weight_age = [70,  # inches,
                     170, # pounds,
                     40 ] # years

grades = [95,   # exam1
          80,   # exam2
          75,   # exam3
          62 ]  # exam4

```

### Add/Subtract
To add/subract vector is, add/subract each element together. The vector must be the same dimension.

\begin{equation}
\vec{v} = \begin{bmatrix}3 \\ -1 \end{bmatrix} \;\;\; \vec{s} = \begin{bmatrix}2 \\ 4 \end{bmatrix}\end{equation} 

\begin{equation}
\begin{aligned}
\vec{t}= \vec{v} + \vec{s} &= \begin{bmatrix}v_x + s_x  \\ v_y + s_y \end{bmatrix} \\  
&= \begin{bmatrix}3 +2 \\ -1 + 4 \end{bmatrix} \\  
&= \begin{bmatrix}5 \\ 3\end{bmatrix} \\  
\end{aligned}
\tag{Vector plus Vecotr}\label{Vector plus Vecotr}
\end{equation}




```python
def add(v: Vector, w: Vector) -> Vector:
    """Adds corresponding elements"""
    assert len(v) == len(w), "vectors must be the same length"

    return [v_i + w_i for v_i, w_i in zip(v, w)]

assert add([1, 2, 3], [4, 5, 6]) == [5, 7, 9]

def subtract(v: Vector, w: Vector) -> Vector:
    """Subtracts corresponding elements"""
    assert len(v) == len(w), "vectors must be the same length"

    return [v_i - w_i for v_i, w_i in zip(v, w)]

assert subtract([5, 7, 9], [4, 5, 6]) == [1, 2, 3]



grades2 = [45,   # exam1
          50,   # exam2
          47,   # exam3
          42 ]  # exam4
add(grades,grades2), subtract(grades,grades2)
```




    ([140, 130, 122, 104], [50, 30, 28, 20])



### Sum of Vectors
Interesting technic to assert and summary in python style, ``all()`` and using the  list comprehension to sum the list of  vectors instead loop call function ``add``.


```python

def vector_sum(vectors: List[Vector]) -> Vector:
    """Sums all corresponding elements"""
    # Check that vectors is not empty
    assert vectors, "no vectors provided!"

    # Check the vectors are all the same size
    num_elements = len(vectors[0])
    assert all(len(v) == num_elements for v in vectors), "different sizes!"

    # the i-th element of the result is the sum of every vector[i]
    return [sum(vector[i] for vector in vectors)
            for i in range(num_elements)]

assert vector_sum([[1, 2], [3, 4], [5, 6], [7, 8]]) == [16, 20]

```

### Vector scalar multiplication and mean

Vector can multiply by scalar as decribe below.

\begin{equation}
\begin{aligned}
\lambda\vec{v}&= \lambda \times \begin{bmatrix}v_x \\ v_y \end{bmatrix} = \begin{bmatrix}\lambda\times v_x  \\ \lambda\times v_y \end{bmatrix} \\  
\vec{v_2}&= 3 \times \vec{v_1} =  3 \times \begin{bmatrix}v_x \\ v_y \end{bmatrix} = \begin{bmatrix}3\times 2  \\ 3\times 1 \end{bmatrix} = \begin{bmatrix}6  \\ 3 \end{bmatrix} \\  
 &= \vec{v_1} + \vec{v_1} + \vec{v_1} = \begin{bmatrix}v_x \\ v_y \end{bmatrix} + \begin{bmatrix}v_x \\ v_y \end{bmatrix} + \begin{bmatrix}v_x \\ v_y \end{bmatrix} = \begin{bmatrix} 2+2+2  \\ 1+1+1 \end{bmatrix}  = \begin{bmatrix}6  \\ 3 \end{bmatrix}
\end{aligned}
\tag{Scalar Multiply}\label{Scalar Multiply}
\end{equation}





```python
def scalar_multiply(c: float, v: Vector) -> Vector:
    """Multiplies every element by c"""
    return [c * v_i for v_i in v]

assert scalar_multiply(2, [1, 2, 3]) == [2, 4, 6]

def vector_mean(vectors: List[Vector]) -> Vector:
    """Computes the element-wise average"""
    n = len(vectors)
    return scalar_multiply(1/n, vector_sum(vectors))

assert vector_mean([[1, 2], [3, 4], [5, 6]]) == [3, 4]

```

### DOT and sum sqaure

Vector DOT products of two vectors define by the equations, note that the dot produc is commutative

\begin{equation}
\begin{aligned}
\vec{v} \cdot \vec{s} = \vec{s} \cdot \vec{v} = \mathbf{v}^\mathbf{T}\mathbf{s}  &=\sum_{i=0}^{i=n} {v_i \cdot s_i}  \\ 
&= (v_{1} \cdot s_{1}) + (v_{2} \cdot s_{2}) ... + \; (v_{n} \cdot s_{n}) \\ 
&= (v_{x} \cdot s_{x}) + (v_{y} \cdot s_{y}) ... + \; (v_{n} \cdot s_{n}) \\  
&= (3 \cdot 2 ) + (-1 \cdot  4) \\  
&= (3 \times 2) + (-1 \times 4) \\
&= 2 \\  
\end{aligned}
\tag{Dot Product}\label{Dot Product}
\end{equation}

Dot product of a vector with itself equivalent to the sum square of all elements.

\begin{equation}
\begin{aligned}
\vec{v} \cdot \vec{v}  &=\sum_{i=0}^{i=n} {v_i \cdot v_i}  \\
&= (v_{1} \cdot v_{1}) + (v_{2} \cdot v_{2}) ... + \; (v_{n} \cdot v_{n}) \\ 
&= (1 \cdot  1) + (2 \cdot  2) + (3 \cdot  3) \\  
&= (1 \times 1) + (2 \times 2) + (3 \times 3) \\
&= (1^2) + (2^2) + (3^2) \\
&= 14 \\  
\end{aligned}
\tag{Sum of Sqaure}\label{Sum of Square}
\end{equation}


```python
def dot(v: Vector, w: Vector) -> float:
    """Computes v_1 * w_1 + ... + v_n * w_n"""
    assert len(v) == len(w), "vectors must be same length"

    return sum(v_i * w_i for v_i, w_i in zip(v, w))

assert dot([1, 2, 3], [4, 5, 6]) == 32  # 1 * 4 + 2 * 5 + 3 * 6

def sum_of_squares(v: Vector) -> float:
    """Returns v_1 * v_1 + ... + v_n * v_n"""
    return dot(v, v)

assert sum_of_squares([1, 2, 3]) == 14  # 1 * 1 + 2 * 2 + 3 * 3


```

### Vector magnitude and Distances
Magnitude of the vector is the square root of the sum of square of all elements 


\begin{equation}
\begin{aligned}
\|\vec{v}\| &= \sqrt{v_{1}^{2} + v_{2}^{2} ... + v_{n}^{2}}  \\
\|\vec{v}\| &= \sqrt{v_{1}^{2} + v_{2}^{2}} \\
            &= \sqrt{3^{2} + 4^{2}} \\
            &= \sqrt{25} \; = 5 
\end{aligned} 
\tag{Magnitude}\label{Magnitude}
\end{equation}

Calculate the distance of vectors in the cartesian system or points co-ordinate 


$$
\vec{v}=\begin{bmatrix}3 \\ 4 \end{bmatrix} \;\;\; \vec{w} = \begin{bmatrix}8 \\ 6 \end{bmatrix} \\ \\
$$



\begin{equation}
\begin{aligned}
Square\;Distance(\vec{v},\vec{w}) &=  \sum_{i=0}^{i=n} {(v_i - w_i)^2}  \\
    &=(v_{1} - w_{1})^{2} + (v_{2} - w_{2})^{2} \\ 
    &=(3 - 8)^{2} + (4 -6)^{2} \\
    &=-5^{2} + -2^{2} \\
    &=29 
\end{aligned} 
\tag{Sqaure Disctance}\label{Sqaure Disctance}
\end{equation}





\begin{equation}
\begin{aligned}
Distance(\vec{v},\vec{w}) &= \sqrt{ \sum_{i=0}^{i=n} {(v_i - w_i)^2}}  \\
    &= \sqrt{ (v_{1} - w_{1})^{2} + (v_{2} - w_{2})^{2}} \\
    &= \sqrt{ (3 - 8)^{2} + (4 -6)^{2}} \\
    &= \sqrt{-5^{2} + -2^{2}} \\
    &= \sqrt{29} \; \approx 5.385
\end{aligned} 
\tag{Distance}\label{Distance}
\end{equation}


```python
import math

def magnitude(v: Vector) -> float:
    """Returns the magnitude (or length) of v"""
    return math.sqrt(sum_of_squares(v))   # math.sqrt is square root function

assert magnitude([3, 4]) == 5

def squared_distance(v: Vector, w: Vector) -> float:
    """Computes (v_1 - w_1) ** 2 + ... + (v_n - w_n) ** 2"""
    return sum_of_squares(subtract(v, w))
print(squared_distance([3, 4],[8,6]))

def distance(v: Vector, w: Vector) -> float:
    """Computes the distance between v and w"""
    return math.sqrt(squared_distance(v, w))
print(distance([3, 4],[8,6]))

def distance(v: Vector, w: Vector) -> float:  # type: ignore
    return magnitude(subtract(v, w))
print(distance([3, 4],[8,6]))
```

    29
    5.385164807134504
    5.385164807134504


---

## Matrix
Two dimensions row and column  data. 



\begin{align*}
A_{M\times N} &=
\begin{bmatrix} 
        a_{11} & a_{12} & \dots   & a_{1N} \\  
        a_{21} & a_{22} & \dots   & a_{2N} \\  
        \vdots & \vdots & \ddots & \vdots \\ 
        a_{M1} & a_{M2} & \dots  & a_{MN}
\end{bmatrix} 
\\ \\
A_{3\times3} &= 
\begin{bmatrix} 
    v_1 & v_2 & v_3 \\  
    v_4 & v_5 & v_6 \\ 
    v_7 & v_8 & v_9
    \end{bmatrix} 
= \begin{bmatrix} 
    1 & 2 & 3 \\  
    4 & 5 & 6 \\ 
    7 & 8 & 9
    \end{bmatrix}     
\end{align*} 

Note that if a dimension is one, the matrix can be considerd as vector. If the row is fixed to one, we call **row vector** and if the vector is fixed column to be one, we call it is a **column vector**.

\begin{align*}\text{Column vector} &= \begin{bmatrix}2 \\ 1 \\ 3 \end{bmatrix} \\
\text{Row vector} &= \begin{bmatrix}2 & 1 & 3 \end{bmatrix} 
\end{align*}

And column vector can be **Transpose** to row vector and vice versa. One dimension which one element may be considered as **Saclar**, isnt't it?

In python usually use **numpy** but simple ``list`` or ``List`` also implement the matrix, in the book it intruduce the Matrix type from ``List``.


```python
# Another type alias
Matrix = List[List[float]]

A = [[1, 2, 3],  # A has 2 rows and 3 columns
     [4, 5, 6]]

B = [[1, 2],     # B has 3 rows and 2 columns
     [3, 4],
     [5, 6]]
ROW_VECTOR = [[1,2,3]]
COL_VECTOR = [[1],[2],[3]]
```

### Matrix Dimension
Dimension of the matrix determine by the shape of the matrix list (or numpy array)


```python
from typing import Tuple

def shape(A: Matrix) -> Tuple[int, int]:
    """Returns (# of rows of A, # of columns of A)"""
    num_rows = len(A)
    num_cols = len(A[0]) if A else 0   # number of elements in first row
    return num_rows, num_cols

assert shape([[1, 2, 3], [4, 5, 6]]) == (2, 3)  # 2 rows, 3 columns

def get_row(A: Matrix, i: int) -> Vector:
    """Returns the i-th row of A (as a Vector)"""
    return A[i]             # A[i] is already the ith row

def get_column(A: Matrix, j: int) -> Vector:
    """Returns the j-th column of A (as a Vector)"""
    return [A_i[j]          # jth element of row A_i
            for A_i in A]   # for each row A_i

print(shape(A))
print(shape(B))
print('Row vector ',ROW_VECTOR, ' shape ',shape(ROW_VECTOR))
print('Row vector ',COL_VECTOR, ' shape ',shape(COL_VECTOR))
```

    (2, 3)
    (3, 2)
    Row vector  [[1, 2, 3]]  shape  (1, 3)
    Row vector  [[1], [2], [3]]  shape  (3, 1)


### Identity Matrix

Matrix that all diagonal element ($a_{1 1},a_{2 2}, \dots a_{n n}$) is one and the remains are all zero called **identity matrix**

\begin{bmatrix} 
      1 & 0 & 0 & 0 \\ 
      0 & 1 & 0 & 0 \\ 
      0 & 0 & 1 & 0 \\
      0 & 0 & 0 & 1 
\end{bmatrix}

Function ``make_metrix(num_rows, num_cols, entry_fn)`` accepts number of rows and columns with a ``Callabel[[int,int]],float`` to return a matrix. The function ``identity_matrix(n:int)`` that retrun $ N\times N$ identity matrix, noticed the ``lambda`` send to ``make_matrix`` as a callable object. 


```python
from typing import Callable

def make_matrix(num_rows: int,
                num_cols: int,
                entry_fn: Callable[[int, int], float]) -> Matrix:
    """
    Returns a num_rows x num_cols matrix
    whose (i,j)-th entry is entry_fn(i, j)
    """
    return [[entry_fn(i, j)             # given i, create a list
             for j in range(num_cols)]  #   [entry_fn(i, 0), ... ]
            for i in range(num_rows)]   # create one list for each i

def identity_matrix(n: int) -> Matrix:
    """Returns the n x n identity matrix"""
    return make_matrix(n, n, lambda i, j: 1 if i == j else 0)

assert identity_matrix(5) == [[1, 0, 0, 0, 0],
                              [0, 1, 0, 0, 0],
                              [0, 0, 1, 0, 0],
                              [0, 0, 0, 1, 0],
                              [0, 0, 0, 0, 1]]
```

Remains code from the book is about matrix representation for data from chapter one.


```python
data = [[70, 170, 40],
        [65, 120, 26],
        [77, 250, 19],
        # ....
       ]

friendships = [(0, 1), (0, 2), (1, 2), (1, 3), (2, 3), (3, 4),
               (4, 5), (5, 6), (5, 7), (6, 8), (7, 8), (8, 9)]

#            user 0  1  2  3  4  5  6  7  8  9
#
friend_matrix = [[0, 1, 1, 0, 0, 0, 0, 0, 0, 0],  # user 0
                 [1, 0, 1, 1, 0, 0, 0, 0, 0, 0],  # user 1
                 [1, 1, 0, 1, 0, 0, 0, 0, 0, 0],  # user 2
                 [0, 1, 1, 0, 1, 0, 0, 0, 0, 0],  # user 3
                 [0, 0, 0, 1, 0, 1, 0, 0, 0, 0],  # user 4
                 [0, 0, 0, 0, 1, 0, 1, 1, 0, 0],  # user 5
                 [0, 0, 0, 0, 0, 1, 0, 0, 1, 0],  # user 6
                 [0, 0, 0, 0, 0, 1, 0, 0, 1, 0],  # user 7
                 [0, 0, 0, 0, 0, 0, 1, 1, 0, 1],  # user 8
                 [0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]  # user 9

assert friend_matrix[0][2] == 1, "0 and 2 are friends"
assert friend_matrix[0][8] == 0, "0 and 8 are not friends"

# only need to look at one row
friends_of_five = [i
                   for i, is_friend in enumerate(friend_matrix[5])
                   if is_friend]


```


```python

```


```python

```


```python

```


```python

```




```python
import numpy as np
a = np.array([1, 2, 3])
display(a)
display(a.transpose())
display(a.dot(a.transpose()))


```


    array([1, 2, 3])



    array([1, 2, 3])



    14



```python
a.shape = (3,1)
display(a)
display(a.transpose())
display(a.dot(a.transpose()))
        
```


    array([[1],
           [2],
           [3]])



    array([[1, 2, 3]])



    array([[1, 2, 3],
           [2, 4, 6],
           [3, 6, 9]])



```python

```

dc9869c5abe6bdfe3893996adb21b6adea177f2e
ipynb
Jupyter Notebook
notebook/Chapter_04.ipynb
ray-ronnaret/data-science-from-scratch
cfbbff996a615a58ee55fa094c82bdaa7d86d8de
MIT
notebook/Chapter_04.ipynb
ray-ronnaret/data-science-from-scratch
cfbbff996a615a58ee55fa094c82bdaa7d86d8de
MIT
notebook/Chapter_04.ipynb
ray-ronnaret/data-science-from-scratch
cfbbff996a615a58ee55fa094c82bdaa7d86d8de
MIT
Qwen/Qwen-72B
1. YES
2. YES