### Conduct a PCA:

First we will import all packages and tools necessary for the Project...


```python
import numpy as np
import keras
import pandas
import matplotlib.pyplot as plt
%matplotlib inline
from sklearn import preprocessing
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
from sympy import *
init_printing(use_latex=True)
from IPython.display import display
```

We bring in all the training data taken, then get rid of any vectors that contain nan values because they will help train out network.  We use a scaler on the data to standardise it.  We then bring in our test data, for the Partial Component Analysis(PCA) and testing.  This will give us a better idea of weither or not we will have success with our data by examining the variance to features.  We also use the same function devised to transform our training data to transform our testing data, so the network is not misinformed about the relationships in the features.


```python
data = np.array(pandas.read_csv("./Demo/training_noavg.csv", header=0))
## bring in loc 0 -1 test data for PCA
data1 = np.array(pandas.read_csv("./Demo/test1.csv", header=0))
data2 = np.array(pandas.read_csv("./Demo/test2.csv", header=0))
data3 = np.array(pandas.read_csv("./Demo/test3.csv", header=0))
## Have to drop all teh rows that have nan values because they will not help with net
## clean out rows with nan values
data = data[~np.isnan(data).any(axis=1)]
data1 = data1[~np.isnan(data1).any(axis=1)]
data2 = data2[~np.isnan(data2).any(axis=1)]
data3 = data3[~np.isnan(data3).any(axis=1)]

print(data[:8])
print(data.shape)

data = np.vstack((data,data1,data2,data3))
print(data[:8])
data.shape
```


```python
# vectors AND class labels...
X = data[:,0:8] # 0 thru 30
Y = data[:,8] # 30

scaler = StandardScaler()
# standardize X .. will mean center data
X = scaler.fit_transform(X)

# Pretty-print with display()!
display(X.shape)
display(Y.shape)
display(Matrix(np.unique(Y)).T)
display(X[0:8])
```

Here we can see that our training data and testing data has been combined so we can analyse it all at the same time.  We use singular value decomposition, to examine each feature component and how it effects the total variance of the data.


```python
U,S,V = np.linalg.svd(X,full_matrices=True)

# Percent variance accounted for
plt.plot(100.0*S/np.sum(S))
plt.ylabel('% Var')
plt.xlabel('Singular Value')
plt.show()
```


```python
# Variance accounted for in the first two principal components
100.0*(S[0]+S[1])/np.sum(S)
```

We see that more than 53% of the variance is accounted for by the first two principal components, so they should give use some dece3nt information about the desicion boundries our network will see.


```python
# Scale the singular vectors, resulting in a rotated form of our mean-centered data
D = np.zeros([X.shape[0],X.shape[1]])
np.fill_diagonal(D,S)
Xrotated = np.dot(U,D)
# Extract just the first two principal components!
PCs = Xrotated[:,0:2]
PCs.shape
```

Here we use the matplot lib package and associate colors to the data that describes each class, as well as our testing data.


```python
# The x and y values come from the two
# Principal Components and the colors for
# each point are selected based on the
# corresponding class for each point...
plt.scatter(PCs[:,0],PCs[:,1],
color=[['red','green','blue','orange'][i] for i in Y.astype(int)])
plt.xlabel("PC1")
plt.ylabel("PC2")
plt.show()
```

The graph suggests that we will be able to create descision boundries fairly easily because our classses are very well grouped.  The orange dots represent the testing data, we can clearly see three seperate groupings that coincide with the three location classes.  This gives us a high level of confidence that our method will work.

At this point you should open up the Project_Multilayer_Net_2.ipynb and follow allong with building the network.  Then once that is complete, come back to this point and run through the rest of this demo to see the testing phase.

https://github.com/CSCI4850/S19-team2-project/blob/master/Demo/Project_Multilayer_Net_2.ipynb

Now that we have a trained network we will bring back in the training and testing data (you may have to re import the first cell with our packages), we can now see how to give our saved model a vector to predict.  THe testing data is stored as test vectors for each loacation.  We can see under the next cell that the first loaction accounts for the first 14 testing vectors and the 2nd location, the next 11.


```python
## bring in data to scale so that testing data is transformed with the same scaling function
data = np.array(pandas.read_csv("./Demo/training_noavg.csv", header=0))
## Have to drop all the rows that have nan values because they will not help with net
## clean out rows with nan values
data = data[~np.isnan(data).any(axis=1)]
X = data[:,0:8]

scaler = StandardScaler()
X = scaler.fit_transform(X)

test1 = np.array(pandas.read_csv("./Demo/test1.csv", header=0))
test2 = np.array(pandas.read_csv("./Demo/test2.csv", header=0))
test3 = np.array(pandas.read_csv("./Demo/test3.csv", header=0))
test1 = test1[:,0:8]
test2 = test2[:,0:8]
test3 = test3[:,0:8]
test1 = test1[~np.isnan(test1).any(axis=1)]
test2 = test2[~np.isnan(test2).any(axis=1)]
test3 = test3[~np.isnan(test3).any(axis=1)]
print(test1.shape)
print(test2.shape)
test = np.vstack((test1,test2,test3))

test = scaler.transform( test )

print(test.shape)
```

    (14, 8)
    (11, 8)
    (36, 8)


We no have the function used to standardize our data and we have run it on our testing data so that our network knows how to classify it properly.  We will then load in our netwrok model and weights.


```python
model = keras.models.load_model('./Demo/MLN.model')
model.load_weights('./Demo/MLN.weights')
```

Now it is simply a matter of using keras' predict function to provide us with the networks best guess at the vectors location class.  Here we use argmax to get the class label, and can look at the probability associated with each vector in relation to each class.


```python
preds = model.predict(test, batch_size=1, verbose=1, steps=None)

#print('Predicted:', preds[:])
for i in range(0,36):
    print(np.argmax(preds[i,]))
    print(preds[i])
```

    36/36 [==============================] - 0s 3ms/step
    0
    [9.9972159e-01 1.5968618e-04 1.1864946e-04]
    0
    [9.99707758e-01 1.87634781e-04 1.04638755e-04]
    0
    [9.9789029e-01 7.2141958e-04 1.3882895e-03]
    0
    [9.989511e-01 5.167439e-04 5.322162e-04]
    0
    [9.9261510e-01 7.5424381e-04 6.6306996e-03]
    0
    [9.9920374e-01 4.3745179e-04 3.5875969e-04]
    0
    [0.9904212  0.00144124 0.0081375 ]
    0
    [9.9963927e-01 1.9248696e-04 1.6824802e-04]
    0
    [9.9928242e-01 3.0315580e-04 4.1447039e-04]
    0
    [9.9974638e-01 1.5775024e-04 9.5761425e-05]
    0
    [9.9973148e-01 1.6664247e-04 1.0195105e-04]
    0
    [9.9973720e-01 1.7404708e-04 8.8845729e-05]
    0
    [9.9970835e-01 1.6403901e-04 1.2761576e-04]
    0
    [9.9974662e-01 1.7622366e-04 7.7121040e-05]
    1
    [0.00153411 0.995652   0.00281398]
    1
    [6.160449e-04 9.987489e-01 6.350375e-04]
    1
    [3.9185164e-04 9.9923086e-01 3.7728454e-04]
    1
    [2.7163792e-04 9.9942029e-01 3.0804746e-04]
    1
    [2.1198661e-04 9.9958497e-01 2.0300722e-04]
    1
    [5.1277818e-04 9.9856216e-01 9.2505309e-04]
    1
    [5.6899071e-04 9.9885035e-01 5.8059767e-04]
    1
    [5.9407944e-04 9.9840987e-01 9.9612435e-04]
    1
    [1.9460864e-04 9.9963331e-01 1.7212275e-04]
    1
    [1.9524986e-04 9.9964595e-01 1.5883044e-04]
    1
    [0.00122977 0.9976331  0.00113714]
    2
    [0.00764636 0.18166819 0.81068546]
    2
    [3.4702296e-04 1.4098064e-04 9.9951196e-01]
    2
    [0.00574952 0.00794102 0.9863094 ]
    2
    [0.00104963 0.00184738 0.99710304]
    2
    [2.5502822e-04 4.6062079e-04 9.9928433e-01]
    2
    [6.8109564e-04 6.4133300e-04 9.9867755e-01]
    2
    [1.3778178e-03 2.6749578e-04 9.9835461e-01]
    2
    [7.2980544e-04 3.2163865e-04 9.9894863e-01]
    2
    [4.0885728e-04 5.9418473e-04 9.9899691e-01]
    2
    [4.1726942e-04 2.4158432e-04 9.9934107e-01]
    2
    [6.9952821e-03 5.3660036e-04 9.9246806e-01]


Our network has predicted all 36 of the testing locations correctly(first 14 are loc 0, second 11 are loc1, final 11 are loc 2, as per class labels).  We now have a tool that can be integrated into an application that allows use to fingerprint a location and provide very accurate location information to a user for any prospective reason.  Based on the output we can increase the number of locations by a magnitude of 10^3 and still gain accurate predictions.  This would allow us to have hundreds or thousands of locations.  We will need to test this in the future. This tool has a farily simple structure and short amount of setup and train time.


```python

```

72cafef3a032d252b47ffd186c51d7696c538ea0
ipynb
Jupyter Notebook
Demo/.ipynb_checkpoints/demo_net-checkpoint.ipynb
holypolarpanda7/S19-team2-project
09b51f07849e3288dfa4ba91cf5d8d13909e35e2
MIT
Demo/.ipynb_checkpoints/demo_net-checkpoint.ipynb
holypolarpanda7/S19-team2-project
09b51f07849e3288dfa4ba91cf5d8d13909e35e2
MIT
Demo/.ipynb_checkpoints/demo_net-checkpoint.ipynb
holypolarpanda7/S19-team2-project
09b51f07849e3288dfa4ba91cf5d8d13909e35e2
MIT
Qwen/Qwen-72B
1. YES
2. YES