# Vertical Line Test


```
import matplotlib.pyplot as plt
import numpy as np
```

## 1.1 Create two graphs, one that passes the vertical line test and one that does not.


```
plt.axhline(y=2)
plt.title("passes the vertical line test")
plt.show()
```


```
plt.axvline(x=2)
plt.title("fails the vertical line test")
plt.show()
```

## 1.2 Why are graphs that don't pass the vertical line test not considered "functions?"

A function cannot have the same input (x value/domain value) mapped to multiple outputs (y value/co-domain value). 

# Functions as Relations

## 2.1 Which of the following relations are functions? Why?

\begin{align}
\text{Relation 1: } \{(1, 2), (3, 2), (1, 3)\}
\\
\text{Relation 2: } \{(1, 3), (2, 3), (6, 7)\}
\\
\text{Relation 3: } \{(9, 4), (2, 1), (9, 6)\}
\\
\text{Relation 4: } \{(6, 2), (8, 3), (6, 4)\}
\\
\text{Relation 5: } \{(2, 6), (2, 7), (2, 4)\}
\end{align}

Relations 2 is the only one here that is a function because there are not repeated x values. In other words, there are no two different y values that are mapped to by the same x value.

# Functions as a mapping between dimensions


## 3.1 for the following functions what is the dimensionality of the domain (input) and codomain (range/output)?

\begin{align}
m(𝑥_1,𝑥_2,𝑥_3)=(x_1+x_2, x_1+x_3, x_2+x_3)
\\
n(𝑥_1,𝑥_2,𝑥_3,𝑥_4)=(x_2^2 + x_3, x_2x_4)
\end{align}

function m: 3D -> 3D

function n: 4D -> 2D

## 3.2 Do you think it's possible to create a function that maps from a lower dimensional space to a higher dimensional space? If so, provide an example.

Yes this is possible. For more information on functions from lower dimensions of space that map to higher dimensions of space, google the terms injective and surjective in relation to Linear Transformations:

<https://youtu.be/xKNX8BUWR0g>

Example:

$f(x) = (x, x+1)$

# Vector Transformations

## 4.1 Plug the corresponding unit vectors into each function. Use the output vectors to create a transformation matrix.

\begin{align}
p(\begin{bmatrix}x_1 \\ x_2 \end{bmatrix}) = \begin{bmatrix} x_1 + 3x_2 \\2 x_2 - x_1 \\  \end{bmatrix}
\\
\\
q(\begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix}) = \begin{bmatrix} 4x_1 + x_2 + 2x_3 \\2 x_2 - x_1 + 3x_3 \\ 5x_1 - 2x_3 + x_2  \end{bmatrix}
\end{align}


---


\begin{align}
p(\begin{bmatrix} 1 \\ 0 \end{bmatrix}) = \begin{bmatrix} x_1 + 3x_2 \\ -x_1+ 2x_2 \\  \end{bmatrix} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}
\end{align}

\begin{align}
p(\begin{bmatrix} 0 \\ 1 \end{bmatrix}) = \begin{bmatrix} x_1 + 3x_2 \\ -x_1+ 2x_2 \\  \end{bmatrix} = \begin{bmatrix} 3 \\ 2 \end{bmatrix}
\end{align}

\begin{align}
T = \begin{bmatrix} 1 & 3 \\ -1 & 2 \end{bmatrix}
\end{align}

---

\begin{align}
q(\begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix}) = \begin{bmatrix} 4x_1 + x_2 + 2x_3 \\ -x_1 + 2x_2  + 3x_3 \\ 5x_1 + x_2 - 2x_3  \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \\ 5\end{bmatrix}
\end{align}

\begin{align}
q(\begin{bmatrix}0 \\ 1 \\ 0 \end{bmatrix}) = \begin{bmatrix} 4x_1 + x_2 + 2x_3 \\ -x_1 + 2x_2  + 3x_3 \\ 5x_1 + x_2 - 2x_3  \end{bmatrix}= \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}
\end{align}

\begin{align}
q(\begin{bmatrix}0 \\ 0 \\ 1 \end{bmatrix}) = \begin{bmatrix} 4x_1 + x_2 + 2x_3 \\ -x_1 + 2x_2  + 3x_3 \\ 5x_1 + x_2 - 2x_3  \end{bmatrix}= \begin{bmatrix} 2 \\ 3 \\ -2\end{bmatrix}
\end{align}

\begin{align}
T = \begin{bmatrix} 4 & 1 & 2 \\ -1 & 2 & 3 \\ 5 & 1 & -2  \end{bmatrix}
\end{align}

---

## 4.2 Verify that your transformation matrices are correct by choosing an input matrix and calculating the result both via the traditional functions above and also via vector-matrix multiplication.


```
# Transformation Matrix
T1 = np.array([[1,3],[-1,2]])
# Input Vector
v1 = np.array([2,3])

# Product found by hand-calculating
product_by_hand_1 = [11, 4]

# Product found with NumPy
np.matmul(T1, v1)
```




    array([11,  4])




```
# Transformation Matrix
T2 = np.array([[4,1,2],[-1,2,3],[5,1,-2]])
# Input Vector
v2 = np.array([1,2,3])

# Product found by hand-calculating
product_by_hand_2 = [12, 12, 1]

# Product found with NumPy
np.matmul(T2, v2)
```




    array([12, 12,  1])



# Eigenvalues and Eigenvectors

## 5.1 In your own words, give an explanation for the intuition behind eigenvalues and eigenvectors.

An eigenvector is a vector that does not change orientation (direction in which it is pointing) during a linear transformation. An eigenvector always comes paired with a corresponding eigenvalue. Every linear transformation has at least one eigenvector-eigenvalue pair. The eigenvalue describes how much the eigenvector gets scaled by (stretched or squished) during the transformation.

# The Curse of Dimensionality

## 6.1 What are some of the challenges of working with high dimensional spaces?

- Increased computational inefficiency (searching)
- Increased data redundancy with increasing number of columns.
- More difficult to visualize/explore the data.
- Measures of euclidean distance breaks down in high-dimensional spaces
- Increased sparcity of data.
- As the number of parameters in a model grows while number of available observations remains fixed, overfitting a predictive model becomes more certain.

## 6.2 What is the rule of thumb for how many observations you should have compared to parameters in your model?

You should have 5x as many observations (rows) as you do parameters (columns) in a machine learning model. 

# Principal Component Analysis

## 7.1 Code for loading and cleaning the 2013 national dataset from the [Housing Affordability Data System (HADS)](https://www.huduser.gov/portal/datasets/hads/hads.html) --housing data, can be found below. 

## Perform PCA on the processed dataset `national_processed` (Make sure you standardize your data!) and then make a scatterplot of PC1 against PC2. Some of our discussion and work around PCA with this dataset will continue during tomorrow's lecture and assignment.

 
The code below will read in the dataset and perform categorical encoding of the categorical variables to ensure that we're only working with numeric columns from our dataset. Start adding your PCA code at the bottom of the provided code.


```
from urllib.request import urlopen
from zipfile import ZipFile
from io import BytesIO
import os.path
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Read Natinal Data 
national_url = 'https://www.huduser.gov/portal/datasets/hads/hads2013n_ASCII.zip'
national_file = 'thads2013n.txt'

if os.path.exists(national_file):
    national = pd.read_csv(national_file)
else: 
    z_national = urlopen(national_url)
    zip_national = ZipFile(BytesIO(z_national.read())).extract(national_file)
    national = pd.read_csv(zip_national)

print(national.shape)
national.head()
```

    (64535, 99)





<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>CONTROL</th>
      <th>AGE1</th>
      <th>METRO3</th>
      <th>REGION</th>
      <th>LMED</th>
      <th>FMR</th>
      <th>L30</th>
      <th>L50</th>
      <th>L80</th>
      <th>IPOV</th>
      <th>BEDRMS</th>
      <th>BUILT</th>
      <th>STATUS</th>
      <th>TYPE</th>
      <th>VALUE</th>
      <th>VACANCY</th>
      <th>TENURE</th>
      <th>NUNITS</th>
      <th>ROOMS</th>
      <th>WEIGHT</th>
      <th>PER</th>
      <th>ZINC2</th>
      <th>ZADEQ</th>
      <th>ZSMHC</th>
      <th>STRUCTURETYPE</th>
      <th>OWNRENT</th>
      <th>UTILITY</th>
      <th>OTHERCOST</th>
      <th>COST06</th>
      <th>COST12</th>
      <th>COST08</th>
      <th>COSTMED</th>
      <th>TOTSAL</th>
      <th>ASSISTED</th>
      <th>GLMED</th>
      <th>GL30</th>
      <th>GL50</th>
      <th>GL80</th>
      <th>APLMED</th>
      <th>ABL30</th>
      <th>...</th>
      <th>COST08RELPOVCAT</th>
      <th>COST08RELFMRPCT</th>
      <th>COST08RELFMRCAT</th>
      <th>COST12RELAMIPCT</th>
      <th>COST12RELAMICAT</th>
      <th>COST12RELPOVPCT</th>
      <th>COST12RELPOVCAT</th>
      <th>COST12RELFMRPCT</th>
      <th>COST12RELFMRCAT</th>
      <th>COSTMedRELAMIPCT</th>
      <th>COSTMedRELAMICAT</th>
      <th>COSTMedRELPOVPCT</th>
      <th>COSTMedRELPOVCAT</th>
      <th>COSTMedRELFMRPCT</th>
      <th>COSTMedRELFMRCAT</th>
      <th>FMTZADEQ</th>
      <th>FMTMETRO3</th>
      <th>FMTBUILT</th>
      <th>FMTSTRUCTURETYPE</th>
      <th>FMTBEDRMS</th>
      <th>FMTOWNRENT</th>
      <th>FMTCOST06RELPOVCAT</th>
      <th>FMTCOST08RELPOVCAT</th>
      <th>FMTCOST12RELPOVCAT</th>
      <th>FMTCOSTMEDRELPOVCAT</th>
      <th>FMTINCRELPOVCAT</th>
      <th>FMTCOST06RELFMRCAT</th>
      <th>FMTCOST08RELFMRCAT</th>
      <th>FMTCOST12RELFMRCAT</th>
      <th>FMTCOSTMEDRELFMRCAT</th>
      <th>FMTINCRELFMRCAT</th>
      <th>FMTCOST06RELAMICAT</th>
      <th>FMTCOST08RELAMICAT</th>
      <th>FMTCOST12RELAMICAT</th>
      <th>FMTCOSTMEDRELAMICAT</th>
      <th>FMTINCRELAMICAT</th>
      <th>FMTASSISTED</th>
      <th>FMTBURDEN</th>
      <th>FMTREGION</th>
      <th>FMTSTATUS</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>'100003130103'</td>
      <td>82</td>
      <td>'3'</td>
      <td>'1'</td>
      <td>73738</td>
      <td>956</td>
      <td>15738</td>
      <td>26213</td>
      <td>40322</td>
      <td>11067</td>
      <td>2</td>
      <td>2006</td>
      <td>'1'</td>
      <td>1</td>
      <td>40000</td>
      <td>-6</td>
      <td>'1'</td>
      <td>1</td>
      <td>6</td>
      <td>3117.394239</td>
      <td>1</td>
      <td>18021</td>
      <td>'1'</td>
      <td>533</td>
      <td>1</td>
      <td>'1'</td>
      <td>169.000000</td>
      <td>213.750000</td>
      <td>648.588189</td>
      <td>803.050535</td>
      <td>696.905247</td>
      <td>615.156712</td>
      <td>0</td>
      <td>-9</td>
      <td>73738</td>
      <td>15738</td>
      <td>26213</td>
      <td>40322</td>
      <td>51616.6</td>
      <td>20234.571429</td>
      <td>...</td>
      <td>4</td>
      <td>72.898038</td>
      <td>2</td>
      <td>48.402635</td>
      <td>2</td>
      <td>290.250487</td>
      <td>4</td>
      <td>84.001102</td>
      <td>2</td>
      <td>37.077624</td>
      <td>2</td>
      <td>222.339102</td>
      <td>4</td>
      <td>64.346936</td>
      <td>2</td>
      <td>'1 Adequate'</td>
      <td>'-5'</td>
      <td>'2000-2009'</td>
      <td>'1 Single Family'</td>
      <td>'2 2BR'</td>
      <td>'1 Owner'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'3 150-200% Poverty'</td>
      <td>'2 50.1 - 100% FMR'</td>
      <td>'2 50.1 - 100% FMR'</td>
      <td>'2 50.1 - 100% FMR'</td>
      <td>'2 50.1 - 100% FMR'</td>
      <td>'1 LTE 50% FMR'</td>
      <td>'2 30 - 50% AMI'</td>
      <td>'2 30 - 50% AMI'</td>
      <td>'2 30 - 50% AMI'</td>
      <td>'2 30 - 50% AMI'</td>
      <td>'2 30 - 50% AMI'</td>
      <td>'.'</td>
      <td>'2 30% to 50%'</td>
      <td>'-5'</td>
      <td>'-5'</td>
    </tr>
    <tr>
      <th>1</th>
      <td>'100006110249'</td>
      <td>50</td>
      <td>'5'</td>
      <td>'3'</td>
      <td>55846</td>
      <td>1100</td>
      <td>17165</td>
      <td>28604</td>
      <td>45744</td>
      <td>24218</td>
      <td>4</td>
      <td>1980</td>
      <td>'1'</td>
      <td>1</td>
      <td>130000</td>
      <td>-6</td>
      <td>'1'</td>
      <td>1</td>
      <td>6</td>
      <td>2150.725544</td>
      <td>4</td>
      <td>122961</td>
      <td>'1'</td>
      <td>487</td>
      <td>1</td>
      <td>'1'</td>
      <td>245.333333</td>
      <td>58.333333</td>
      <td>1167.640781</td>
      <td>1669.643405</td>
      <td>1324.671218</td>
      <td>1058.988479</td>
      <td>123000</td>
      <td>-9</td>
      <td>55846</td>
      <td>17165</td>
      <td>28604</td>
      <td>45744</td>
      <td>55846.0</td>
      <td>19911.400000</td>
      <td>...</td>
      <td>4</td>
      <td>120.424656</td>
      <td>3</td>
      <td>103.094063</td>
      <td>6</td>
      <td>275.768999</td>
      <td>4</td>
      <td>151.785764</td>
      <td>3</td>
      <td>65.388468</td>
      <td>4</td>
      <td>174.909320</td>
      <td>3</td>
      <td>96.271680</td>
      <td>2</td>
      <td>'1 Adequate'</td>
      <td>'-5'</td>
      <td>'1980-1989'</td>
      <td>'1 Single Family'</td>
      <td>'4 4BR+'</td>
      <td>'1 Owner'</td>
      <td>'3 150-200% Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'3 150-200% Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'2 50.1 - 100% FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'4 60 - 80% AMI'</td>
      <td>'4 60 - 80% AMI'</td>
      <td>'6 100 - 120% AMI'</td>
      <td>'4 60 - 80% AMI'</td>
      <td>'7 120% AMI +'</td>
      <td>'.'</td>
      <td>'1 Less than 30%'</td>
      <td>'-5'</td>
      <td>'-5'</td>
    </tr>
    <tr>
      <th>2</th>
      <td>'100006370140'</td>
      <td>53</td>
      <td>'5'</td>
      <td>'3'</td>
      <td>55846</td>
      <td>1100</td>
      <td>13750</td>
      <td>22897</td>
      <td>36614</td>
      <td>15470</td>
      <td>4</td>
      <td>1985</td>
      <td>'1'</td>
      <td>1</td>
      <td>150000</td>
      <td>-6</td>
      <td>'1'</td>
      <td>1</td>
      <td>7</td>
      <td>2213.789404</td>
      <td>2</td>
      <td>27974</td>
      <td>'1'</td>
      <td>1405</td>
      <td>1</td>
      <td>'1'</td>
      <td>159.000000</td>
      <td>37.500000</td>
      <td>1193.393209</td>
      <td>1772.627006</td>
      <td>1374.582175</td>
      <td>1068.025168</td>
      <td>28000</td>
      <td>-9</td>
      <td>55846</td>
      <td>13750</td>
      <td>22897</td>
      <td>36614</td>
      <td>44676.8</td>
      <td>19937.500000</td>
      <td>...</td>
      <td>4</td>
      <td>124.962016</td>
      <td>3</td>
      <td>109.452905</td>
      <td>6</td>
      <td>458.339239</td>
      <td>4</td>
      <td>161.147910</td>
      <td>3</td>
      <td>65.946449</td>
      <td>4</td>
      <td>276.153890</td>
      <td>4</td>
      <td>97.093197</td>
      <td>2</td>
      <td>'1 Adequate'</td>
      <td>'-5'</td>
      <td>'1980-1989'</td>
      <td>'1 Single Family'</td>
      <td>'4 4BR+'</td>
      <td>'1 Owner'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'3 150-200% Poverty'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'2 50.1 - 100% FMR'</td>
      <td>'2 50.1 - 100% FMR'</td>
      <td>'4 60 - 80% AMI'</td>
      <td>'5 80 - 100% AMI'</td>
      <td>'6 100 - 120% AMI'</td>
      <td>'4 60 - 80% AMI'</td>
      <td>'4 60 - 80% AMI'</td>
      <td>'.'</td>
      <td>'3 50% or More'</td>
      <td>'-5'</td>
      <td>'-5'</td>
    </tr>
    <tr>
      <th>3</th>
      <td>'100006520140'</td>
      <td>67</td>
      <td>'5'</td>
      <td>'3'</td>
      <td>55846</td>
      <td>949</td>
      <td>13750</td>
      <td>22897</td>
      <td>36614</td>
      <td>13964</td>
      <td>3</td>
      <td>1985</td>
      <td>'1'</td>
      <td>1</td>
      <td>200000</td>
      <td>-6</td>
      <td>'1'</td>
      <td>1</td>
      <td>6</td>
      <td>2364.585097</td>
      <td>2</td>
      <td>32220</td>
      <td>'1'</td>
      <td>279</td>
      <td>1</td>
      <td>'1'</td>
      <td>179.000000</td>
      <td>70.666667</td>
      <td>1578.857612</td>
      <td>2351.169341</td>
      <td>1820.442900</td>
      <td>1411.700224</td>
      <td>0</td>
      <td>-9</td>
      <td>55846</td>
      <td>13750</td>
      <td>22897</td>
      <td>36614</td>
      <td>44676.8</td>
      <td>17875.000000</td>
      <td>...</td>
      <td>4</td>
      <td>191.827492</td>
      <td>3</td>
      <td>161.926709</td>
      <td>7</td>
      <td>673.494512</td>
      <td>4</td>
      <td>247.752301</td>
      <td>3</td>
      <td>97.224801</td>
      <td>5</td>
      <td>404.382763</td>
      <td>4</td>
      <td>148.756610</td>
      <td>3</td>
      <td>'1 Adequate'</td>
      <td>'-5'</td>
      <td>'1980-1989'</td>
      <td>'1 Single Family'</td>
      <td>'3 3BR'</td>
      <td>'1 Owner'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'2 50.1 - 100% FMR'</td>
      <td>'6 100 - 120% AMI'</td>
      <td>'7 120% AMI +'</td>
      <td>'7 120% AMI +'</td>
      <td>'5 80 - 100% AMI'</td>
      <td>'4 60 - 80% AMI'</td>
      <td>'.'</td>
      <td>'1 Less than 30%'</td>
      <td>'-5'</td>
      <td>'-5'</td>
    </tr>
    <tr>
      <th>4</th>
      <td>'100007130148'</td>
      <td>26</td>
      <td>'1'</td>
      <td>'3'</td>
      <td>60991</td>
      <td>737</td>
      <td>14801</td>
      <td>24628</td>
      <td>39421</td>
      <td>15492</td>
      <td>2</td>
      <td>1980</td>
      <td>'1'</td>
      <td>1</td>
      <td>-6</td>
      <td>-6</td>
      <td>'2'</td>
      <td>100</td>
      <td>4</td>
      <td>2314.524902</td>
      <td>2</td>
      <td>96874</td>
      <td>'1'</td>
      <td>759</td>
      <td>5</td>
      <td>'2'</td>
      <td>146.000000</td>
      <td>12.500000</td>
      <td>759.000000</td>
      <td>759.000000</td>
      <td>759.000000</td>
      <td>759.000000</td>
      <td>96900</td>
      <td>0</td>
      <td>60991</td>
      <td>14801</td>
      <td>24628</td>
      <td>39421</td>
      <td>48792.8</td>
      <td>16651.125000</td>
      <td>...</td>
      <td>3</td>
      <td>102.985075</td>
      <td>3</td>
      <td>55.308707</td>
      <td>3</td>
      <td>195.972115</td>
      <td>3</td>
      <td>102.985075</td>
      <td>3</td>
      <td>55.308707</td>
      <td>3</td>
      <td>195.972115</td>
      <td>3</td>
      <td>102.985075</td>
      <td>3</td>
      <td>'1 Adequate'</td>
      <td>'Central City'</td>
      <td>'1980-1989'</td>
      <td>'5 50+ units'</td>
      <td>'2 2BR'</td>
      <td>'2 Renter'</td>
      <td>'3 150-200% Poverty'</td>
      <td>'3 150-200% Poverty'</td>
      <td>'3 150-200% Poverty'</td>
      <td>'3 150-200% Poverty'</td>
      <td>'4 200%+ Poverty'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'3 GT FMR'</td>
      <td>'3 50 - 60% AMI'</td>
      <td>'3 50 - 60% AMI'</td>
      <td>'3 50 - 60% AMI'</td>
      <td>'3 50 - 60% AMI'</td>
      <td>'7 120% AMI +'</td>
      <td>'0 Not Assisted'</td>
      <td>'1 Less than 30%'</td>
      <td>'-5'</td>
      <td>'-5'</td>
    </tr>
  </tbody>
</table>
<p>5 rows × 99 columns</p>
</div>




```
# Look at datatypes
# a lot of object datatypes even though they seem to be strings of numbers.
national.dtypes
```




    CONTROL                 object
    AGE1                     int64
    METRO3                  object
    REGION                  object
    LMED                     int64
    FMR                      int64
    L30                      int64
    L50                      int64
    L80                      int64
    IPOV                     int64
    BEDRMS                   int64
    BUILT                    int64
    STATUS                  object
    TYPE                     int64
    VALUE                    int64
    VACANCY                  int64
    TENURE                  object
    NUNITS                   int64
    ROOMS                    int64
    WEIGHT                 float64
    PER                      int64
    ZINC2                    int64
    ZADEQ                   object
    ZSMHC                    int64
    STRUCTURETYPE            int64
    OWNRENT                 object
    UTILITY                float64
    OTHERCOST              float64
    COST06                 float64
    COST12                 float64
                            ...   
    COSTMedRELAMICAT         int64
    COSTMedRELPOVPCT       float64
    COSTMedRELPOVCAT         int64
    COSTMedRELFMRPCT       float64
    COSTMedRELFMRCAT         int64
    FMTZADEQ                object
    FMTMETRO3               object
    FMTBUILT                object
    FMTSTRUCTURETYPE        object
    FMTBEDRMS               object
    FMTOWNRENT              object
    FMTCOST06RELPOVCAT      object
    FMTCOST08RELPOVCAT      object
    FMTCOST12RELPOVCAT      object
    FMTCOSTMEDRELPOVCAT     object
    FMTINCRELPOVCAT         object
    FMTCOST06RELFMRCAT      object
    FMTCOST08RELFMRCAT      object
    FMTCOST12RELFMRCAT      object
    FMTCOSTMEDRELFMRCAT     object
    FMTINCRELFMRCAT         object
    FMTCOST06RELAMICAT      object
    FMTCOST08RELAMICAT      object
    FMTCOST12RELAMICAT      object
    FMTCOSTMEDRELAMICAT     object
    FMTINCRELAMICAT         object
    FMTASSISTED             object
    FMTBURDEN               object
    FMTREGION               object
    FMTSTATUS               object
    Length: 99, dtype: object




```
# check for null values
national.isnull().sum().any()
```




    False




```
# check for number of categorical vs numeric columns
cat_cols = national.columns[national.dtypes=='object']
num_cols = national.columns[national.dtypes!='object']

print(f'{len(cat_cols)} categorical columns')
print(f'{len(num_cols)} numerical columns')
```

    32 categorical columns
    67 numerical columns



```
# We're making a copy of our data in case we mess something up.
national_processed = national.copy()

# Categorically Encode our Variables:
# They need to all be numeric before we do PCA.
# https://pbpython.com/categorical-encoding.html

# Cast categorical columns to "category" data type
national_processed[cat_cols] = national_processed[cat_cols].astype('category')

national_processed.dtypes
```




    CONTROL                category
    AGE1                      int64
    METRO3                 category
    REGION                 category
    LMED                      int64
    FMR                       int64
    L30                       int64
    L50                       int64
    L80                       int64
    IPOV                      int64
    BEDRMS                    int64
    BUILT                     int64
    STATUS                 category
    TYPE                      int64
    VALUE                     int64
    VACANCY                   int64
    TENURE                 category
    NUNITS                    int64
    ROOMS                     int64
    WEIGHT                  float64
    PER                       int64
    ZINC2                     int64
    ZADEQ                  category
    ZSMHC                     int64
    STRUCTURETYPE             int64
    OWNRENT                category
    UTILITY                 float64
    OTHERCOST               float64
    COST06                  float64
    COST12                  float64
                             ...   
    COSTMedRELAMICAT          int64
    COSTMedRELPOVPCT        float64
    COSTMedRELPOVCAT          int64
    COSTMedRELFMRPCT        float64
    COSTMedRELFMRCAT          int64
    FMTZADEQ               category
    FMTMETRO3              category
    FMTBUILT               category
    FMTSTRUCTURETYPE       category
    FMTBEDRMS              category
    FMTOWNRENT             category
    FMTCOST06RELPOVCAT     category
    FMTCOST08RELPOVCAT     category
    FMTCOST12RELPOVCAT     category
    FMTCOSTMEDRELPOVCAT    category
    FMTINCRELPOVCAT        category
    FMTCOST06RELFMRCAT     category
    FMTCOST08RELFMRCAT     category
    FMTCOST12RELFMRCAT     category
    FMTCOSTMEDRELFMRCAT    category
    FMTINCRELFMRCAT        category
    FMTCOST06RELAMICAT     category
    FMTCOST08RELAMICAT     category
    FMTCOST12RELAMICAT     category
    FMTCOSTMEDRELAMICAT    category
    FMTINCRELAMICAT        category
    FMTASSISTED            category
    FMTBURDEN              category
    FMTREGION              category
    FMTSTATUS              category
    Length: 99, dtype: object




```
# Replace all category cell values with their numeric category codes
for col in cat_cols:
  national_processed[col] = national_processed[col].cat.codes

print(national_processed.shape)
national_processed.head()
```

    (64535, 99)





<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>CONTROL</th>
      <th>AGE1</th>
      <th>METRO3</th>
      <th>REGION</th>
      <th>LMED</th>
      <th>FMR</th>
      <th>L30</th>
      <th>L50</th>
      <th>L80</th>
      <th>IPOV</th>
      <th>BEDRMS</th>
      <th>BUILT</th>
      <th>STATUS</th>
      <th>TYPE</th>
      <th>VALUE</th>
      <th>VACANCY</th>
      <th>TENURE</th>
      <th>NUNITS</th>
      <th>ROOMS</th>
      <th>WEIGHT</th>
      <th>PER</th>
      <th>ZINC2</th>
      <th>ZADEQ</th>
      <th>ZSMHC</th>
      <th>STRUCTURETYPE</th>
      <th>OWNRENT</th>
      <th>UTILITY</th>
      <th>OTHERCOST</th>
      <th>COST06</th>
      <th>COST12</th>
      <th>COST08</th>
      <th>COSTMED</th>
      <th>TOTSAL</th>
      <th>ASSISTED</th>
      <th>GLMED</th>
      <th>GL30</th>
      <th>GL50</th>
      <th>GL80</th>
      <th>APLMED</th>
      <th>ABL30</th>
      <th>...</th>
      <th>COST08RELPOVCAT</th>
      <th>COST08RELFMRPCT</th>
      <th>COST08RELFMRCAT</th>
      <th>COST12RELAMIPCT</th>
      <th>COST12RELAMICAT</th>
      <th>COST12RELPOVPCT</th>
      <th>COST12RELPOVCAT</th>
      <th>COST12RELFMRPCT</th>
      <th>COST12RELFMRCAT</th>
      <th>COSTMedRELAMIPCT</th>
      <th>COSTMedRELAMICAT</th>
      <th>COSTMedRELPOVPCT</th>
      <th>COSTMedRELPOVCAT</th>
      <th>COSTMedRELFMRPCT</th>
      <th>COSTMedRELFMRCAT</th>
      <th>FMTZADEQ</th>
      <th>FMTMETRO3</th>
      <th>FMTBUILT</th>
      <th>FMTSTRUCTURETYPE</th>
      <th>FMTBEDRMS</th>
      <th>FMTOWNRENT</th>
      <th>FMTCOST06RELPOVCAT</th>
      <th>FMTCOST08RELPOVCAT</th>
      <th>FMTCOST12RELPOVCAT</th>
      <th>FMTCOSTMEDRELPOVCAT</th>
      <th>FMTINCRELPOVCAT</th>
      <th>FMTCOST06RELFMRCAT</th>
      <th>FMTCOST08RELFMRCAT</th>
      <th>FMTCOST12RELFMRCAT</th>
      <th>FMTCOSTMEDRELFMRCAT</th>
      <th>FMTINCRELFMRCAT</th>
      <th>FMTCOST06RELAMICAT</th>
      <th>FMTCOST08RELAMICAT</th>
      <th>FMTCOST12RELAMICAT</th>
      <th>FMTCOSTMEDRELAMICAT</th>
      <th>FMTINCRELAMICAT</th>
      <th>FMTASSISTED</th>
      <th>FMTBURDEN</th>
      <th>FMTREGION</th>
      <th>FMTSTATUS</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0</td>
      <td>82</td>
      <td>2</td>
      <td>0</td>
      <td>73738</td>
      <td>956</td>
      <td>15738</td>
      <td>26213</td>
      <td>40322</td>
      <td>11067</td>
      <td>2</td>
      <td>2006</td>
      <td>0</td>
      <td>1</td>
      <td>40000</td>
      <td>-6</td>
      <td>1</td>
      <td>1</td>
      <td>6</td>
      <td>3117.394239</td>
      <td>1</td>
      <td>18021</td>
      <td>1</td>
      <td>533</td>
      <td>1</td>
      <td>0</td>
      <td>169.000000</td>
      <td>213.750000</td>
      <td>648.588189</td>
      <td>803.050535</td>
      <td>696.905247</td>
      <td>615.156712</td>
      <td>0</td>
      <td>-9</td>
      <td>73738</td>
      <td>15738</td>
      <td>26213</td>
      <td>40322</td>
      <td>51616.6</td>
      <td>20234.571429</td>
      <td>...</td>
      <td>4</td>
      <td>72.898038</td>
      <td>2</td>
      <td>48.402635</td>
      <td>2</td>
      <td>290.250487</td>
      <td>4</td>
      <td>84.001102</td>
      <td>2</td>
      <td>37.077624</td>
      <td>2</td>
      <td>222.339102</td>
      <td>4</td>
      <td>64.346936</td>
      <td>2</td>
      <td>1</td>
      <td>0</td>
      <td>5</td>
      <td>1</td>
      <td>2</td>
      <td>0</td>
      <td>4</td>
      <td>4</td>
      <td>4</td>
      <td>4</td>
      <td>3</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>2</td>
      <td>0</td>
      <td>2</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
      <td>50</td>
      <td>4</td>
      <td>2</td>
      <td>55846</td>
      <td>1100</td>
      <td>17165</td>
      <td>28604</td>
      <td>45744</td>
      <td>24218</td>
      <td>4</td>
      <td>1980</td>
      <td>0</td>
      <td>1</td>
      <td>130000</td>
      <td>-6</td>
      <td>1</td>
      <td>1</td>
      <td>6</td>
      <td>2150.725544</td>
      <td>4</td>
      <td>122961</td>
      <td>1</td>
      <td>487</td>
      <td>1</td>
      <td>0</td>
      <td>245.333333</td>
      <td>58.333333</td>
      <td>1167.640781</td>
      <td>1669.643405</td>
      <td>1324.671218</td>
      <td>1058.988479</td>
      <td>123000</td>
      <td>-9</td>
      <td>55846</td>
      <td>17165</td>
      <td>28604</td>
      <td>45744</td>
      <td>55846.0</td>
      <td>19911.400000</td>
      <td>...</td>
      <td>4</td>
      <td>120.424656</td>
      <td>3</td>
      <td>103.094063</td>
      <td>6</td>
      <td>275.768999</td>
      <td>4</td>
      <td>151.785764</td>
      <td>3</td>
      <td>65.388468</td>
      <td>4</td>
      <td>174.909320</td>
      <td>3</td>
      <td>96.271680</td>
      <td>2</td>
      <td>1</td>
      <td>0</td>
      <td>3</td>
      <td>1</td>
      <td>4</td>
      <td>0</td>
      <td>3</td>
      <td>4</td>
      <td>4</td>
      <td>3</td>
      <td>4</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>1</td>
      <td>3</td>
      <td>3</td>
      <td>3</td>
      <td>5</td>
      <td>3</td>
      <td>7</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>2</td>
      <td>53</td>
      <td>4</td>
      <td>2</td>
      <td>55846</td>
      <td>1100</td>
      <td>13750</td>
      <td>22897</td>
      <td>36614</td>
      <td>15470</td>
      <td>4</td>
      <td>1985</td>
      <td>0</td>
      <td>1</td>
      <td>150000</td>
      <td>-6</td>
      <td>1</td>
      <td>1</td>
      <td>7</td>
      <td>2213.789404</td>
      <td>2</td>
      <td>27974</td>
      <td>1</td>
      <td>1405</td>
      <td>1</td>
      <td>0</td>
      <td>159.000000</td>
      <td>37.500000</td>
      <td>1193.393209</td>
      <td>1772.627006</td>
      <td>1374.582175</td>
      <td>1068.025168</td>
      <td>28000</td>
      <td>-9</td>
      <td>55846</td>
      <td>13750</td>
      <td>22897</td>
      <td>36614</td>
      <td>44676.8</td>
      <td>19937.500000</td>
      <td>...</td>
      <td>4</td>
      <td>124.962016</td>
      <td>3</td>
      <td>109.452905</td>
      <td>6</td>
      <td>458.339239</td>
      <td>4</td>
      <td>161.147910</td>
      <td>3</td>
      <td>65.946449</td>
      <td>4</td>
      <td>276.153890</td>
      <td>4</td>
      <td>97.093197</td>
      <td>2</td>
      <td>1</td>
      <td>0</td>
      <td>3</td>
      <td>1</td>
      <td>4</td>
      <td>0</td>
      <td>4</td>
      <td>4</td>
      <td>4</td>
      <td>4</td>
      <td>3</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>1</td>
      <td>2</td>
      <td>3</td>
      <td>4</td>
      <td>5</td>
      <td>3</td>
      <td>4</td>
      <td>0</td>
      <td>3</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>3</td>
      <td>67</td>
      <td>4</td>
      <td>2</td>
      <td>55846</td>
      <td>949</td>
      <td>13750</td>
      <td>22897</td>
      <td>36614</td>
      <td>13964</td>
      <td>3</td>
      <td>1985</td>
      <td>0</td>
      <td>1</td>
      <td>200000</td>
      <td>-6</td>
      <td>1</td>
      <td>1</td>
      <td>6</td>
      <td>2364.585097</td>
      <td>2</td>
      <td>32220</td>
      <td>1</td>
      <td>279</td>
      <td>1</td>
      <td>0</td>
      <td>179.000000</td>
      <td>70.666667</td>
      <td>1578.857612</td>
      <td>2351.169341</td>
      <td>1820.442900</td>
      <td>1411.700224</td>
      <td>0</td>
      <td>-9</td>
      <td>55846</td>
      <td>13750</td>
      <td>22897</td>
      <td>36614</td>
      <td>44676.8</td>
      <td>17875.000000</td>
      <td>...</td>
      <td>4</td>
      <td>191.827492</td>
      <td>3</td>
      <td>161.926709</td>
      <td>7</td>
      <td>673.494512</td>
      <td>4</td>
      <td>247.752301</td>
      <td>3</td>
      <td>97.224801</td>
      <td>5</td>
      <td>404.382763</td>
      <td>4</td>
      <td>148.756610</td>
      <td>3</td>
      <td>1</td>
      <td>0</td>
      <td>3</td>
      <td>1</td>
      <td>3</td>
      <td>0</td>
      <td>4</td>
      <td>4</td>
      <td>4</td>
      <td>4</td>
      <td>4</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>5</td>
      <td>6</td>
      <td>6</td>
      <td>4</td>
      <td>4</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>4</th>
      <td>4</td>
      <td>26</td>
      <td>0</td>
      <td>2</td>
      <td>60991</td>
      <td>737</td>
      <td>14801</td>
      <td>24628</td>
      <td>39421</td>
      <td>15492</td>
      <td>2</td>
      <td>1980</td>
      <td>0</td>
      <td>1</td>
      <td>-6</td>
      <td>-6</td>
      <td>2</td>
      <td>100</td>
      <td>4</td>
      <td>2314.524902</td>
      <td>2</td>
      <td>96874</td>
      <td>1</td>
      <td>759</td>
      <td>5</td>
      <td>1</td>
      <td>146.000000</td>
      <td>12.500000</td>
      <td>759.000000</td>
      <td>759.000000</td>
      <td>759.000000</td>
      <td>759.000000</td>
      <td>96900</td>
      <td>0</td>
      <td>60991</td>
      <td>14801</td>
      <td>24628</td>
      <td>39421</td>
      <td>48792.8</td>
      <td>16651.125000</td>
      <td>...</td>
      <td>3</td>
      <td>102.985075</td>
      <td>3</td>
      <td>55.308707</td>
      <td>3</td>
      <td>195.972115</td>
      <td>3</td>
      <td>102.985075</td>
      <td>3</td>
      <td>55.308707</td>
      <td>3</td>
      <td>195.972115</td>
      <td>3</td>
      <td>102.985075</td>
      <td>3</td>
      <td>1</td>
      <td>1</td>
      <td>3</td>
      <td>5</td>
      <td>2</td>
      <td>1</td>
      <td>3</td>
      <td>3</td>
      <td>3</td>
      <td>3</td>
      <td>4</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>3</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>2</td>
      <td>7</td>
      <td>1</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
<p>5 rows × 99 columns</p>
</div>




```
# Now we only ahve numeric columns (ints and floats)
national_processed.dtypes
```




    CONTROL                  int32
    AGE1                     int64
    METRO3                    int8
    REGION                    int8
    LMED                     int64
    FMR                      int64
    L30                      int64
    L50                      int64
    L80                      int64
    IPOV                     int64
    BEDRMS                   int64
    BUILT                    int64
    STATUS                    int8
    TYPE                     int64
    VALUE                    int64
    VACANCY                  int64
    TENURE                    int8
    NUNITS                   int64
    ROOMS                    int64
    WEIGHT                 float64
    PER                      int64
    ZINC2                    int64
    ZADEQ                     int8
    ZSMHC                    int64
    STRUCTURETYPE            int64
    OWNRENT                   int8
    UTILITY                float64
    OTHERCOST              float64
    COST06                 float64
    COST12                 float64
                            ...   
    COSTMedRELAMICAT         int64
    COSTMedRELPOVPCT       float64
    COSTMedRELPOVCAT         int64
    COSTMedRELFMRPCT       float64
    COSTMedRELFMRCAT         int64
    FMTZADEQ                  int8
    FMTMETRO3                 int8
    FMTBUILT                  int8
    FMTSTRUCTURETYPE          int8
    FMTBEDRMS                 int8
    FMTOWNRENT                int8
    FMTCOST06RELPOVCAT        int8
    FMTCOST08RELPOVCAT        int8
    FMTCOST12RELPOVCAT        int8
    FMTCOSTMEDRELPOVCAT       int8
    FMTINCRELPOVCAT           int8
    FMTCOST06RELFMRCAT        int8
    FMTCOST08RELFMRCAT        int8
    FMTCOST12RELFMRCAT        int8
    FMTCOSTMEDRELFMRCAT       int8
    FMTINCRELFMRCAT           int8
    FMTCOST06RELAMICAT        int8
    FMTCOST08RELAMICAT        int8
    FMTCOST12RELAMICAT        int8
    FMTCOSTMEDRELAMICAT       int8
    FMTINCRELAMICAT           int8
    FMTASSISTED               int8
    FMTBURDEN                 int8
    FMTREGION                 int8
    FMTSTATUS                 int8
    Length: 99, dtype: object



---


```
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
```


```
# There isn't a super clear Y varaible in this dataset, so we'll just 
# pretend that our whole dataset is our X matrix for now.

# Make a copy to save our work at this checkpoint
df = national_processed.copy()
# Turn our dataframe into a numpy array
X = df.values
# instantiate our standard scaler object
scaler = StandardScaler()
# Standardize our data
Z = scaler.fit_transform(X)
```


```
# Instantiate our PCA object
pca = PCA(2)
transformed_data = pca.fit_transform(Z)
```


```
transformed_data
```




    array([[-2.57657018, -1.37612728],
           [ 2.04064284, -0.98806146],
           [ 1.21527025,  0.37016814],
           ...,
           [ 1.83162816, -2.95570216],
           [-5.02514474, -2.52857809],
           [-9.89816404, -0.80284263]])




```
plt.scatter(transformed_data[:,0], transformed_data[:,1])
plt.xlabel("PC1")
plt.ylabel("PC2")
plt.title("Principal Component 1 vs Principal Component 2")
plt.show()
```

# Stretch Goals

## 1) Perform further data exploration on the HADS national dataset (the version before we one-hot encoded it) Make scatterplots and see if you can see any resemblance between the original scatterplots and the plot of the principal components that you made in 7.1. 

(You may or may not not see very much resemblance depending on the variables you choose, and that's ok!)

## 2) Study "Scree Plots" and then try and make one for your PCA dataset. How many principal conponents do you need to retain in order for your PCs to contain 90% of the explained variance? 

We will present this topic formally at the beginning of tomorrow's lecture, so if you figure this stretch goal out, you're ahead of the game. 

## 3) Explore further the intuition behind eigenvalues and eigenvectors by creating your very own eigenfaces:

Prioritize self-study over this stretch goal if you are not semi-comfortable with the topics of PCA, Eigenvalues, and Eigenvectors.



You don't necessarily have to use this resource, but this will get you started: 
[Eigenface Tutorial](https://sandipanweb.wordpress.com/2018/01/06/eigenfaces-and-a-simple-face-detector-with-pca-svd-in-python/)

6fc2da77ce806062bfb92010b10ada73a2eec82d
ipynb
Jupyter Notebook
curriculum/unit-1-statistics-fundamentals/sprint-3-linear-algebra/module4-clustering/module-3.ipynb
BrianThomasRoss/lambda-school
6140db5cb5a43d0a367e9a08dc216e8bec9fb323
MIT
curriculum/unit-1-statistics-fundamentals/sprint-3-linear-algebra/module4-clustering/module-3.ipynb
BrianThomasRoss/lambda-school
6140db5cb5a43d0a367e9a08dc216e8bec9fb323
MIT
curriculum/unit-1-statistics-fundamentals/sprint-3-linear-algebra/module4-clustering/module-3.ipynb
BrianThomasRoss/lambda-school
6140db5cb5a43d0a367e9a08dc216e8bec9fb323
MIT
Qwen/Qwen-72B
1. YES
2. YES