# Vertical Line Test

## 1.1 Create two graphs, one that passes the vertical line test and one that does not.


```python
import matplotlib.pyplot as plt

plt.plot([0,1,2], [1,3,9])
plt.xlim(0, 9)
plt.ylim(0, 9)
plt.show();
```


```python
plt.plot([1,1,2], [1,3,9])
plt.xlim(0, 9)
plt.ylim(0, 9)
plt.show();
```

## 1.2 Why are graphs that don't pass the vertical line test not considered "functions?"

Functions have one and only one output for a given input.

# Functions as Relations

## 2.1 Which of the following relations are functions? Why?

\begin{align}
\text{Relation 1: } \{(1, 2), (3, 2), (1, 3)\}
\\
\text{Relation 2: } \{(1, 3), (2, 3), (6, 7)\}
\\
\text{Relation 3: } \{(9, 4), (2, 1), (9, 6)\}
\\
\text{Relation 4: } \{(6, 2), (8, 3), (6, 4)\}
\\
\text{Relation 5: } \{(2, 6), (2, 7), (2, 4)\}
\end{align}

Only Relation 2. The other functions have multiple outputs for the same input.

# Functions as a mapping between dimensions


## 3.1 for the following functions what is the dimensionality of the domain (input) and codomain (range/output)?

\begin{align}
m(𝑥_1,𝑥_2,𝑥_3)=(x_1+x_2, x_1+x_3, x_2+x_3)
\\
n(𝑥_1,𝑥_2,𝑥_3,𝑥_4)=(x_2^2 + x_3, x_2x_4)
\end{align}

**m**

input: R^3

output: R^3

**n**

input: R^3

output: R^2

## 3.2 Do you think it's possible to create a function that maps from a lower dimensional space to a higher dimensional space? If so, provide an example.

Sure
\begin{align}
m(𝑥_1,𝑥_2)=(x_1, x_2, x_1+x_2)
\end{align}

# Vector Transformations

## 4.1 Plug the corresponding unit vectors into each function. Use the output vectors to create a transformation matrix.

\begin{align}
p(\begin{bmatrix}x_1 \\ x_2 \end{bmatrix}) = \begin{bmatrix} x_1 + 3x_2 \\2 x_2 - x_1 \\  \end{bmatrix}
\\
\\
p(\begin{bmatrix}x_1 \\ x_2 \end{bmatrix}) = x_1\begin{bmatrix} 1 \\-1 \\  \end{bmatrix} + x_2\begin{bmatrix} 3 \\2 \\  \end{bmatrix}
\\
\\
p(\begin{bmatrix}x_1 \\ x_2 \end{bmatrix}) = \begin{bmatrix} x_1 \\x_2 \\  \end{bmatrix} \begin{bmatrix} 1 &3 \\-1 &2 \\  \end{bmatrix}
\\
\\
q(\begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix}) = \begin{bmatrix} 4x_1 + x_2 + 2x_3 \\2 x_2 - x_1 + 3x_3 \\ 5x_1 - 2x_3 + x_2  \end{bmatrix}
\\
\\
q(\begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix}) = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}\begin{bmatrix} 4 & 1 & 2 \\ -1 & 2 & 3 \\ 5 & 1 & -2 \end{bmatrix}
\\
\end{align}

## 4.2 Verify that your transformation matrices are correct by choosing an input matrix and calculating the result both via the traditional functions above and also via vector-matrix multiplication.

```
p(4, 7) =
    x_1 = 4 + 3*7 = 25
    x_2 = 2*7 - 4 = 10
```


```python
import numpy as np

p_trans = np.array([[ 1, 3],
                    [-1, 2]])
p_test = np.array([[4],
                   [7]])
np.matmul(p_trans, p_test)
```




    array([[25],
           [10]])



```
q(1, 2, 3) =
    x_1 = 4*1 + 2 + 2*3 = 12
    x_2 = 2*2 - 1 + 3*3 = 12
    x_3 = 5*1 - 2*3 + 2 = 1
```


```python
q_trans = np.array([[ 4, 1,  2],
                    [-1, 2,  3],
                    [ 5, 1, -2]])
q_test = np.array([[1],
                   [2],
                   [3]])
np.matmul(q_trans, q_test)
```




    array([[12],
           [12],
           [ 1]])



# Eigenvalues and Eigenvectors

## 5.1 In your own words, give an explanation for the intuition behind eigenvalues and eigenvectors.

**Eigenvectors** are the unique vectors of a space-transformation pair that are not re-oriented by the transformation of that space. However, the eigenvecotr may be scaled by this transformation, in which case this scalar is known as the **eigenvalue**.

# The Curse of Dimensionality

## 6.1 What are some of the challenges of working with high dimensional spaces?

With more dimensions, the more "distant" vectors are from each other, the less meaningful that distance is (they become equidistant), and the greater the threat of over-fitting.

## 6.2 What is the rule of thumb for how many observations you should have compared to parameters in your model?

5 * number of dimensions

# Principal Component Analysis

## 7.1 Load the UCI Machine Learning Repository's [Iris Dataset](https://gist.githubusercontent.com/curran/a08a1080b88344b0c8a7/raw/d546eaee765268bf2f487608c537c05e22e4b221/iris.csv) and use PCA to isolate the dataset's first and second principal components and plot them on a graph. 


```python
import pandas as pd

df = pd.read_csv('https://gist.githubusercontent.com/curran/a08a1080b88344b0c8a7/'
                 'raw/d546eaee765268bf2f487608c537c05e22e4b221/iris.csv')
target = 'species'
features = [col for col in df if col != target]
X = df[features].copy()

for feature in X:
    X[feature] = (X[feature] - X[feature].mean()) / X[feature].std(ddof=1)

covariance_matrix = np.cov(X.T)

eig_values, eig_vectors = np.linalg.eig(covariance_matrix)

P = eig_vectors.T.dot(X.T)

P
```




    array([[-2.25698063e+00, -2.07945912e+00, -2.36004408e+00,
            -2.29650366e+00, -2.38080159e+00, -2.06362348e+00,
            -2.43754534e+00, -2.22638327e+00, -2.33413810e+00,
            -2.18136797e+00, -2.15626287e+00, -2.31960686e+00,
            -2.21665672e+00, -2.63090249e+00, -2.18497165e+00,
            -2.24394778e+00, -2.19539570e+00, -2.18286636e+00,
            -1.88775015e+00, -2.33213620e+00, -1.90816387e+00,
            -2.19728429e+00, -2.76490710e+00, -1.81433338e+00,
            -2.22077769e+00, -1.95048969e+00, -2.04521166e+00,
            -2.16095426e+00, -2.13315968e+00, -2.26121491e+00,
            -2.13739396e+00, -1.82582143e+00, -2.59949432e+00,
            -2.42981077e+00, -2.18136797e+00, -2.20373717e+00,
            -2.03759040e+00, -2.18136797e+00, -2.42781878e+00,
            -2.16329995e+00, -2.27889274e+00, -1.86545777e+00,
            -2.54929405e+00, -1.95772074e+00, -2.12624970e+00,
            -2.06842817e+00, -2.37330742e+00, -2.39018435e+00,
            -2.21934620e+00, -2.19858869e+00,  1.10030752e+00,
             7.30035752e-01,  1.23796222e+00,  3.95980711e-01,
             1.06901266e+00,  3.83174476e-01,  7.46215186e-01,
            -4.96201068e-01,  9.23129797e-01,  4.95143781e-03,
            -1.24281108e-01,  4.37265239e-01,  5.49792127e-01,
             7.14770518e-01, -3.71339807e-02,  8.72966018e-01,
             3.46844441e-01,  1.52880381e-01,  1.21124542e+00,
             1.56417164e-01,  7.35791136e-01,  4.70792484e-01,
             1.22388808e+00,  6.27279600e-01,  6.98133985e-01,
             8.70620328e-01,  1.25003446e+00,  1.35370481e+00,
             6.59915360e-01, -4.71236447e-02,  1.21128417e-01,
             1.40710866e-02,  2.35222819e-01,  1.05316323e+00,
             2.20677797e-01,  4.30341477e-01,  1.04590946e+00,
             1.03241951e+00,  6.68436674e-02,  2.74505447e-01,
             2.71425765e-01,  6.21089831e-01,  3.28903506e-01,
            -3.72380115e-01,  2.81999618e-01,  8.87557702e-02,
             2.23607677e-01,  5.71967342e-01, -4.55486949e-01,
             2.51402252e-01,  1.84150339e+00,  1.14933941e+00,
             2.19898270e+00,  1.43388176e+00,  1.86165399e+00,
             2.74500070e+00,  3.57177896e-01,  2.29531637e+00,
             1.99505169e+00,  2.25998344e+00,  1.36134878e+00,
             1.59372546e+00,  1.87796051e+00,  1.24890257e+00,
             1.45917316e+00,  1.58649440e+00,  1.46636772e+00,
             2.42924030e+00,  3.29809227e+00,  1.24979406e+00,
             2.03368323e+00,  9.70663302e-01,  2.88838807e+00,
             1.32475564e+00,  1.69855041e+00,  1.95119099e+00,
             1.16799163e+00,  1.01637610e+00,  1.78004554e+00,
             1.85855159e+00,  2.42736549e+00,  2.30834923e+00,
             1.85415982e+00,  1.10756129e+00,  1.19347092e+00,
             2.79159729e+00,  1.57487926e+00,  1.34254677e+00,
             9.20349720e-01,  1.84736315e+00,  2.00942544e+00,
             1.89676253e+00,  1.14933941e+00,  2.03648602e+00,
             1.99500751e+00,  1.86427657e+00,  1.55328823e+00,
             1.51576710e+00,  1.37179555e+00,  9.56095566e-01],
           [-5.04015404e-01,  6.53216394e-01,  3.17413945e-01,
             5.73446613e-01, -6.72514411e-01, -1.51347827e+00,
            -7.43137171e-02, -2.46787172e-01,  1.09148977e+00,
             4.47131117e-01, -1.06702096e+00, -1.58057946e-01,
             7.06750478e-01,  9.35149145e-01, -1.88366805e+00,
            -2.71328133e+00, -1.50869601e+00, -5.12587094e-01,
            -1.42633236e+00, -1.15416686e+00, -4.29027880e-01,
            -9.49277150e-01, -4.87882574e-01, -1.06394362e-01,
            -1.61644638e-01,  6.05862870e-01, -2.65126115e-01,
            -5.50173363e-01, -3.35516398e-01,  3.13827252e-01,
             4.82326259e-01, -4.43780131e-01, -1.82237008e+00,
            -2.17809480e+00,  4.47131117e-01,  1.83722324e-01,
            -6.82669420e-01,  4.47131117e-01,  8.79223933e-01,
            -2.91749567e-01, -4.66429135e-01,  2.31991966e+00,
             4.52301130e-01, -4.95730895e-01, -1.16752081e+00,
             6.89607099e-01, -1.14679074e+00,  3.61180775e-01,
            -1.02205856e+00, -3.21302061e-02, -8.60230593e-01,
            -5.96636785e-01, -6.12769614e-01,  1.75229858e+00,
             2.11050863e-01,  5.89088966e-01, -7.76098609e-01,
             1.84269557e+00, -3.02295550e-02,  1.02596404e+00,
             2.64918765e+00,  5.86846859e-02,  1.76666308e+00,
             1.84815166e-01,  4.31350036e-01, -5.08295314e-01,
             1.89985179e-01,  7.88085297e-01,  1.62790202e+00,
             1.29875233e+00, -4.01126570e-01,  4.15217206e-01,
             9.37773165e-01,  4.15419947e-01,  6.32819273e-02,
            -2.49871518e-01,  8.23442389e-02, -3.27722366e-01,
             2.23597000e-01,  1.05368248e+00,  1.55837169e+00,
             1.56813894e+00,  7.73333046e-01,  6.34774729e-01,
             2.79909969e-01, -8.52281697e-01, -5.20453696e-01,
             1.38781717e+00,  2.11910814e-01,  1.32537578e+00,
             1.11570381e+00, -2.74506710e-02,  9.85598884e-01,
             2.01119458e+00,  8.51099455e-01,  1.74324544e-01,
             3.79214256e-01,  1.53206717e-01,  1.53432438e+00,
             5.93871222e-01, -8.68786147e-01,  6.98984451e-01,
            -5.52618781e-01,  4.98435418e-02, -2.90220536e-01,
            -7.85799704e-01,  1.55488557e+00, -4.08149357e-01,
             7.21448440e-01, -1.91502747e+00, -6.91631011e-01,
             4.26818953e-01, -4.12949339e-01,  1.16349352e+00,
             4.42664602e-01, -6.74774813e-01, -2.52347086e-01,
            -2.54822057e+00,  2.35343587e-03,  1.71184899e+00,
            -9.04369044e-01,  5.69267278e-01, -3.96463171e-01,
             4.85135293e-01, -1.01076228e+00, -9.99984474e-01,
             3.17831851e-01, -6.53241212e-02,  1.92627480e-01,
            -5.53527164e-01, -2.45830912e-01, -2.61741528e+00,
             1.84055790e-01,  2.94997832e-01,  8.14439294e-01,
            -8.41927658e-01, -1.06889360e+00, -4.20846092e-01,
            -1.91661621e-02, -6.70177572e-01, -6.08358978e-01,
            -6.83734258e-01,  6.98984451e-01, -8.61797778e-01,
            -1.04504904e+00, -3.81543631e-01,  9.02290843e-01,
            -2.65903772e-01, -1.01296839e+00,  2.22095406e-02],
           [-1.21536190e-01, -2.26492063e-01,  5.13077379e-02,
             9.85303586e-02,  2.13563034e-02,  3.05818064e-02,
             3.41055070e-01, -8.22987560e-02,  1.53049668e-01,
            -2.45736283e-01, -2.63127870e-01,  9.98311719e-02,
            -2.22469221e-01,  1.88937073e-01, -4.67912544e-01,
             3.24949356e-02, -1.35893901e-03, -3.84866610e-02,
            -3.69504958e-01,  1.36958894e-01, -4.14617866e-01,
             1.64188300e-01,  3.37705041e-01,  3.99277109e-02,
             1.23786731e-01, -2.97594061e-01,  9.17854889e-02,
            -2.00623375e-01, -2.64428684e-01,  7.52632970e-02,
            -6.76291966e-02, -2.64489180e-01,  5.12478331e-02,
            -7.90848134e-02, -2.45736283e-01, -2.17894561e-01,
            -4.77810860e-01, -2.45736283e-01,  2.00884605e-01,
            -1.69371127e-01,  4.06005234e-02, -1.93879097e-01,
             3.12524850e-01,  3.13704670e-01,  2.51949169e-01,
            -5.63701629e-02,  6.18945510e-02,  1.46365295e-01,
            -1.76055500e-01, -1.46104065e-01, -6.83266103e-01,
            -9.37527216e-02, -5.52993953e-01, -1.85074806e-02,
            -3.96120397e-01,  1.26374324e-01,  1.48159674e-01,
             2.54701975e-01, -5.93471704e-01,  5.41054466e-01,
            -3.96805148e-02,  2.06013327e-01, -7.58838045e-01,
            -6.70751344e-02,  1.97400140e-01, -5.01824672e-01,
             4.91185999e-01, -2.97607503e-01, -4.77809208e-01,
            -1.68023850e-01,  6.14713279e-01, -2.61841091e-01,
            -3.65480465e-01, -2.88994316e-01, -4.43282521e-01,
            -4.70572424e-01, -7.24416666e-01, -2.60585089e-01,
             8.70763929e-02, -3.14266373e-01, -1.44756789e-01,
            -2.35791505e-01, -1.47478817e-01,  1.06396795e-01,
             6.65330740e-01,  4.49226536e-01, -3.94819584e-01,
            -6.83145701e-01,  2.93146195e-01,  9.31327651e-02,
             9.78441041e-02, -1.92401979e-02, -1.95313754e-01,
             1.11809481e-01,  1.33671013e-01,  1.31009481e-01,
             1.58238888e-01, -2.69137780e-01,  1.95471326e-01,
             9.44335784e-02,  9.99412860e-01,  5.29690124e-01,
            -2.04809778e-01,  1.62844918e-01,  3.92688789e-01,
            -5.84275328e-01,  9.87691038e-01, -6.51982485e-01,
            -3.92754684e-01,  3.91207078e-01,  2.82333672e-01,
             2.32262719e-02,  2.44665886e-02,  5.80186592e-01,
             1.00075789e+00,  6.34525003e-01,  3.65351132e-02,
            -1.33752000e-01, -7.04573572e-01, -2.63738535e-01,
             2.31103895e-01,  8.26734145e-01, -8.58052287e-01,
            -4.69163193e-03,  2.94969701e-01, -4.21405305e-01,
             1.30215675e-01,  3.36913478e-01,  2.69101012e-01,
            -7.15114982e-01, -7.27795822e-01, -4.97951359e-01,
             3.52150542e-01, -1.82049724e-01, -1.62668826e-01,
            -5.45174572e-01,  9.40242708e-01,  1.79427607e-01,
             4.16000663e-01, -1.47708457e-02,  4.24492856e-01,
             1.27372654e-01,  5.29690124e-01,  3.34146638e-01,
             6.27167818e-01,  2.53682459e-01, -2.52971620e-02,
             1.78678613e-01,  9.28295177e-01,  5.27028592e-01],
           [ 2.29962838e-02,  1.02863642e-01,  2.77323192e-02,
            -6.60900544e-02, -3.72724209e-02,  4.38408784e-03,
            -3.79693659e-02, -2.54200029e-02, -2.67043670e-02,
            -3.97740567e-02,  1.52500486e-02, -1.34104994e-01,
             2.62183044e-03, -1.93571700e-02,  1.92138361e-01,
             4.69683560e-02,  1.86008435e-01,  9.15986427e-02,
             5.93293380e-02, -3.96952015e-02,  1.03256322e-02,
             5.75364099e-02,  1.77941178e-02,  1.49843605e-01,
            -2.70323255e-01,  4.36909201e-02,  6.63786282e-02,
             9.22964909e-03,  8.32649884e-02, -1.08485942e-01,
            -4.82172368e-02,  2.38342524e-01, -2.31148225e-01,
            -5.08506746e-02, -3.97740567e-02,  1.68056763e-01,
             1.94960180e-01, -3.97740567e-02, -9.92753256e-03,
             6.21944933e-03,  1.05365277e-01,  2.90719046e-01,
            -6.71860375e-02,  1.74954094e-01, -1.52717190e-01,
             1.39826548e-01, -1.53703647e-01, -4.93132199e-02,
            -1.63894036e-02,  4.86153365e-02,  3.48610735e-02,
             4.43889296e-03,  9.64105894e-03,  6.71728117e-02,
             1.05189268e-01, -2.39724981e-01, -7.80396265e-02,
            -3.92576228e-02, -2.90052501e-02, -2.82541090e-02,
             1.60866655e-02,  3.97183975e-02,  4.81922483e-02,
            -1.64006239e-01,  1.08661097e-01,  1.04790230e-01,
            -1.91418220e-01, -2.03639005e-01,  2.27452513e-01,
            -5.02448721e-02, -8.41695519e-02,  1.13863263e-01,
            -8.42013960e-03, -2.72581705e-01,  4.39341062e-02,
             1.01780030e-01,  4.06930920e-02,  6.67900380e-02,
            -3.62311585e-02,  6.57873163e-02, -7.84898495e-03,
            -3.10452570e-02,  2.43778864e-02, -1.82806816e-01,
            -2.54697124e-01, -1.10775062e-01,  3.71743283e-02,
             1.38664082e-01, -1.46998590e-01,  9.91430675e-03,
            -2.68941652e-01, -1.47229405e-01,  7.60105196e-03,
             2.10110819e-02, -1.06516920e-01, -2.29367584e-01,
            -1.32135972e-01, -1.93447983e-02,  2.00212649e-01,
            -5.81006331e-02, -5.08975254e-02, -4.02786435e-02,
             5.91025008e-02, -2.34972681e-01, -1.67257667e-02,
            -1.00542846e-01, -1.32543405e-01, -2.36420767e-01,
            -8.47100358e-02,  1.02563700e-01,  1.06653619e-01,
             5.87458960e-02,  1.45808492e-01,  9.93488551e-02,
             2.74103899e-01,  1.90009070e-01, -1.54916942e-01,
            -2.74741142e-01,  4.66000729e-02, -6.28568258e-02,
             1.66581983e-01,  2.72277324e-02, -1.25653335e-01,
             1.40128432e-01, -6.25308917e-02, -2.17729716e-01,
             1.25265815e-01, -9.03822942e-03,  3.11031008e-02,
            -2.06863755e-01, -1.67375294e-02, -2.12448695e-01,
             9.97054598e-02, -1.85120071e-01, -4.86773264e-01,
             2.95331758e-01,  3.34952108e-02, -2.15185647e-01,
             4.72840526e-03,  1.94224778e-01,  2.45940777e-01,
             4.67647757e-01, -4.02786435e-02,  4.41303571e-02,
             2.11878544e-01,  3.87592018e-01,  2.20583209e-01,
             1.18506037e-01,  2.40655734e-02, -1.63129308e-01]])




```python
# Using sklearn
from sklearn.decomposition import PCA

target = 'species'
features = [col for col in df if col != target]
X = df[features].copy()

pca = PCA(2)
pca.fit(X)
B = pca.transform(X)
```


```python
import seaborn as sns

sns.scatterplot(x=0, y=1, data=df[[target]].join(pd.DataFrame(B)), hue='species');
```

# Stretch Goal

1) Do NOT work on the stretch goal until you feel like you have a firm grasp of eigenvectors, eigenvalues, and PCA. Prioritize self-study over the stretch goal if you are not comfortable with those topics yet.

2) Explore further the intuition behind eigenvalues and eigenvectors by creating your very own eigenfaces:



You don't necessarily have to use this resource, but this will get you started: 
[Eigenface Tutorial](https://sandipanweb.wordpress.com/2018/01/06/eigenfaces-and-a-simple-face-detector-with-pca-svd-in-python/)

28783ce5b2830a0fe0cdc1ab475769cc21c83b9c
ipynb
Jupyter Notebook
05-Linear-Algebra/03_High_Dimensional_Data_Assignment.ipynb
shalevy1/data-science-journal
2a6beaf5bf328e257b638a695983457a9f3cd7ce
MIT
2019-03-05T04:44:48.000Z
2022-03-24T09:47:48.000Z
05-Linear-Algebra/03_High_Dimensional_Data_Assignment.ipynb
pesobreiro/data-science-journal
82a72b4ed5ce380988fac17b0acd97254c2b5c86
MIT
05-Linear-Algebra/03_High_Dimensional_Data_Assignment.ipynb
pesobreiro/data-science-journal
82a72b4ed5ce380988fac17b0acd97254c2b5c86
MIT
2019-03-07T05:08:03.000Z
2022-01-05T11:32:51.000Z
Qwen/Qwen-72B
1. YES
2. YES