<jupyter_start><jupyter_text>MULTILAYER NN - TENSORFLOW 2.X<jupyter_code>import tensorflow as tf
print(tf.__version__)
!python --version
import keras
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras import regularizers
from tensorflow.keras.optimizers import SGD
from tensorflow.keras import utils
import numpy as np
batch_size = 100
n_inputs = 784
n_classes = 10
n_epochs = 50
(X_train, y_train), (X_test, y_test) = mnist.load_data()
#Reshape the two dimensional 28 x 28 pixels
#Flattened vector of 784 values
X_train = X_train.reshape(60000, n_inputs)
X_test = X_test.reshape(10000, n_inputs)
#Convert in float 32
X_train = X_train.astype(np.float32)
X_test = X_test.astype(np.float32)
#Normalize the values of image vectors to fit under 1
X_train /= 255
X_test /= 255
#Convert output data into one hot encoded format
y_train = utils.to_categorical(y_train, n_classes)
y_test = utils.to_categorical(y_test, n_classes)
y_test[1]<jupyter_output><empty_output><jupyter_text>Creating a sequential model<jupyter_code>model = Sequential()
#first layer = input vector
model.add(Dense(units=128, activation='sigmoid', input_shape=(n_inputs,), kernel_regularizer=regularizers.l2(0.01)))
model.add(Dense(units=128, activation='relu', kernel_regularizer=regularizers.l2(0.01)))
model.add(Dense(units=128, activation='relu', kernel_regularizer=regularizers.l2(0.01)))
#output layer = number of classes
model.add(Dense(units=n_classes, activation='softmax'))<jupyter_output><empty_output><jupyter_text>Model Summary:<jupyter_code>model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 128)               100480    
_________________________________________________________________
dense_1 (Dense)              (None, 128)               16512     
_________________________________________________________________
dense_2 (Dense)              (None, 128)               16512     
_________________________________________________________________
dense_3 (Dense)              (None, 10)                1290      
=================================================================
Total params: 134,794
Trainable params: 134,794
Non-trainable params: 0
_________________________________________________________________<jupyter_text>Compiling the model:<jupyter_code>model.compile(loss='categorical_crossentropy',
              optimizer=SGD(),
              metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Training the model:<jupyter_code>model.fit(X_train, y_train, batch_size=batch_size, epochs=n_epochs)<jupyter_output>Epoch 1/50
600/600 [==============================] - 4s 5ms/step - loss: 6.3833 - accuracy: 0.4391
Epoch 2/50
600/600 [==============================] - 3s 5ms/step - loss: 4.8910 - accuracy: 0.6859
Epoch 3/50
600/600 [==============================] - 3s 5ms/step - loss: 3.7211 - accuracy: 0.7850
Epoch 4/50
600/600 [==============================] - 3s 5ms/step - loss: 3.0451 - accuracy: 0.8241
Epoch 5/50
600/600 [==============================] - 4s 6ms/step - loss: 2.5619 - accuracy: 0.8442
Epoch 6/50
600/600 [==============================] - 3s 5ms/step - loss: 2.1934 - accuracy: 0.8562
Epoch 7/50
600/600 [==============================] - 3s 5ms/step - loss: 1.9070 - accuracy: 0.8638
Epoch 8/50
600/600 [==============================] - 3s 5ms/step - loss: 1.6824 - accuracy: 0.8687
Epoch 9/50
600/600 [==============================] - 3s 5ms/step - loss: 1.5053 - accuracy: 0.8707
Epoch 10/50
600/600 [==============================] - 3s 5ms/step - loss: 1.3647 - accuracy: 0.8740[...]<jupyter_text>Evaluating the model:<jupyter_code>scores = model.evaluate(X_test, y_test)
print('loss:\n', scores[0])
print('accuracy:\n', scores[1])<jupyter_output>loss:
 0.7095614671707153
accuracy:
 0.8838000297546387<jupyter_text>Evaluating ADAM:<jupyter_code>model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])
model.fit(X_train, y_train, batch_size=batch_size, epochs=n_epochs)
scores = model.evaluate(X_test, y_test)
print('loss:\n', scores[0])
print('accuracy:\n', scores[1])<jupyter_output>loss:
 0.4335955083370209
accuracy:
 0.9340000152587891