<jupyter_start><jupyter_text>*Data Science Unit 4 Sprint 3 Assignment 2* Convolutional Neural Networks (CNNs)  Assignment- Part 1: Pre-Trained Model- Part 2: Custom CNN Model- Part 3: CNN with Data AugmentationYou will apply three different CNN models to a binary image classification model using Keras. Classify images of Mountains (`./data/train/mountain/*`) and images of forests (`./data/train/forest/*`). Treat mountains as the positive class (1) and the forest images as the negative (zero). |Mountain (+)|Forest (-)||---|---||![](https://github.com/LambdaSchool/DS-Unit-4-Sprint-3-Deep-Learning/blob/main/module2-convolutional-neural-networks/data/train/mountain/art1131.jpg?raw=1)|![](https://github.com/LambdaSchool/DS-Unit-4-Sprint-3-Deep-Learning/blob/main/module2-convolutional-neural-networks/data/validation/forest/cdmc317.jpg?raw=1)|The problem is relatively difficult given that the sample is tiny: there are about 350 observations per class. This sample size might be something that you can expect with prototyping an image classification problem/solution at work. Get accustomed to evaluating several different possible models.  Pre - Trained ModelLoad a pretrained network from Keras, [ResNet50](https://tfhub.dev/google/imagenet/resnet_v1_50/classification/1) - a 50 layer deep network trained to recognize [1000 objects](https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt). Starting usage:```pythonimport numpy as npfrom tensorflow.keras.applications.resnet50 import ResNet50from tensorflow.keras.preprocessing import imagefrom tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictionsfrom tensorflow.keras.layers import Dense, GlobalAveragePooling2Dfrom tensorflow.keras.models import Model  This is the functional APIresnet = ResNet50(weights='imagenet', include_top=False)```The `include_top` parameter in `ResNet50` will remove the full connected layers from the ResNet model. The next step is to turn off the training of the ResNet layers. We want to use the learned parameters without updating them in future training passes. ```pythonfor layer in resnet.layers:    layer.trainable = False```Using the Keras functional API, we will need to additional additional full connected layers to our model. We we removed the top layers, we removed all preivous fully connected layers. In other words, we kept only the feature processing portions of our network. You can expert with additional layers beyond what's listed here. The `GlobalAveragePooling2D` layer functions as a really fancy flatten function by taking the average of each of the last convolutional layer outputs (which is two dimensional still). ```pythonx = resnet.outputx = GlobalAveragePooling2D()(x)  This layer is a really fancy flattenx = Dense(1024, activation='relu')(x)predictions = Dense(1, activation='sigmoid')(x)model = Model(resnet.input, predictions)```Your assignment is to apply the transfer learning above to classify images of Mountains (`./data/train/mountain/*`) and images of forests (`./data/train/forest/*`). Treat mountains as the positive class (1) and the forest images as the negative (zero). Steps to complete assignment: 1. Load in Image Data into numpy arrays (`X`) 2. Create a `y` for the labels3. Train your model with pre-trained layers from resnet4. Report your model's accuracy  Load in DataThis surprisingly more difficult than it seems, because you are working with directories of images instead of a single file. This boiler plate will help you download a zipped version of the directory of images. The directory is organized into "train" and "validation" which you can use inside an `ImageGenerator` class to stream batches of images thru your model.    Download & Summarize the DataThis step is completed for you. Just run the cells and review the results.<jupyter_code>import tensorflow as tf
import os

_URL = 'https://github.com/LambdaSchool/DS-Unit-4-Sprint-3-Deep-Learning/blob/main/module2-convolutional-neural-networks/data.zip?raw=true'

path_to_zip = tf.keras.utils.get_file('./data.zip', origin=_URL, extract=True)
PATH = os.path.join(os.path.dirname(path_to_zip), 'data')
train_dir = os.path.join(PATH, 'train')
validation_dir = os.path.join(PATH, 'validation')
train_mountain_dir = os.path.join(train_dir, 'mountain')  # directory with our training cat pictures
train_forest_dir = os.path.join(train_dir, 'forest')  # directory with our training dog pictures
validation_mountain_dir = os.path.join(validation_dir, 'mountain')  # directory with our validation cat pictures
validation_forest_dir = os.path.join(validation_dir, 'forest')  # directory with our validation dog pictures
num_mountain_tr = len(os.listdir(train_mountain_dir))
num_forest_tr = len(os.listdir(train_forest_dir))

num_mountain_val = len(os.listdir(validation_mountain_dir))
num_forest_val = len(os.listdir(validation_forest_dir))

total_train = num_mountain_tr + num_forest_tr
total_val = num_mountain_val + num_forest_val
print('total training mountain images:', num_mountain_tr)
print('total training forest images:', num_forest_tr)

print('total validation mountain images:', num_mountain_val)
print('total validation forest images:', num_forest_val)
print("--")
print("Total training images:", total_train)
print("Total validation images:", total_val)<jupyter_output>total training mountain images: 254
total training forest images: 270
total validation mountain images: 125
total validation forest images: 62
--
Total training images: 524
Total validation images: 187<jupyter_text>Keras `ImageGenerator` to Process the DataThis step is completed for you, but please review the code. The `ImageGenerator` class reads in batches of data from a directory and pass them to the model one batch at a time. Just like large text files, this method is advantageous, because it stifles the need to load a bunch of images into memory. Check out the documentation for this class method: [Keras `ImageGenerator` Class](https://keras.io/preprocessing/image/imagedatagenerator-class). You'll expand it's use in the third assignment objective.<jupyter_code>batch_size = 16
epochs = 50
IMG_HEIGHT = 224
IMG_WIDTH = 224
from tensorflow.keras.preprocessing.image import ImageDataGenerator

train_image_generator = ImageDataGenerator(rescale=1./255) # Generator for our training data
validation_image_generator = ImageDataGenerator(rescale=1./255) # Generator for our validation data
train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,
                                                           directory=train_dir,
                                                           shuffle=True,
                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                           class_mode='binary')
val_data_gen = validation_image_generator.flow_from_directory(batch_size=batch_size,
                                                              directory=validation_dir,
                                                              target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                              class_mode='binary')<jupyter_output>Found 195 images belonging to 2 classes.<jupyter_text>Instatiate Model<jupyter_code>import numpy as np
 
from tensorflow.keras.applications.resnet50 import ResNet50
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictions
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
from tensorflow.keras.models import Model # This is the functional API

resnet = ResNet50(weights='imagenet', include_top=False)
for layer in resnet.layers:
  layer.trainable=False

x = resnet.output
x = GlobalAveragePooling2D()(x) # This layer is a really fancy flatten
x = Dense(1024, activation='relu')(x)
predictions = Dense(1, activation='sigmoid')(x)
model = Model(resnet.input, predictions)
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])
model.summary()<jupyter_output>Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/resnet/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5
94773248/94765736 [==============================] - 0s 0us/step
Model: "functional_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, None, None,  0                                            
__________________________________________________________________________________________________
conv1_pad (ZeroPadding2D)       (None, None, None, 3 0           input_1[0][0]                    
__________________________________________________________________________________________________
conv1_conv (Conv2D)             (None, None, None, 6 9472        conv1_pad[0][...]<jupyter_text>Fit Model<jupyter_code>history = model.fit(
    train_data_gen,
    steps_per_epoch=total_train // batch_size,
    epochs=epochs,
    validation_data=val_data_gen,
    validation_steps=total_val // batch_size
)<jupyter_output>Epoch 1/50
32/32 [==============================] - 3s 107ms/step - loss: 0.7980 - accuracy: 0.5808 - val_loss: 0.6329 - val_accuracy: 0.7273
Epoch 2/50
32/32 [==============================] - 2s 77ms/step - loss: 0.5624 - accuracy: 0.7685 - val_loss: 0.5171 - val_accuracy: 0.8352
Epoch 3/50
32/32 [==============================] - 2s 77ms/step - loss: 0.5067 - accuracy: 0.7964 - val_loss: 0.4290 - val_accuracy: 0.7784
Epoch 4/50
32/32 [==============================] - 2s 77ms/step - loss: 0.4855 - accuracy: 0.7645 - val_loss: 0.4455 - val_accuracy: 0.8295
Epoch 5/50
32/32 [==============================] - 3s 78ms/step - loss: 0.4156 - accuracy: 0.8383 - val_loss: 0.5267 - val_accuracy: 0.7727
Epoch 6/50
32/32 [==============================] - 2s 77ms/step - loss: 0.3944 - accuracy: 0.8423 - val_loss: 0.3575 - val_accuracy: 0.8239
Epoch 7/50
32/32 [==============================] - 2s 78ms/step - loss: 0.3543 - accuracy: 0.8723 - val_loss: 0.5511 - val_accuracy: 0.7386
Epoch 8/50
3[...]<jupyter_text>Custom CNN ModelIn this step, write and train your own convolutional neural network using Keras. You can use any architecture that suits you as long as it has at least one convolutional and one pooling layer at the beginning of the network - you can add more if you want.<jupyter_code># Define the Model
from tensorflow.keras import datasets
from tensorflow.keras.models import Sequential, Model # <- May Use
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dropout

my_model = Sequential([Conv2D(32,3, activation="relu", input_shape=(224,224,3)),
                       MaxPooling2D(),
                       Conv2D(32,3, activation="relu", input_shape=(224,224,3)),
                       MaxPooling2D(),
                       Conv2D(64,3, activation="relu", input_shape=(224,224,3)),
                       MaxPooling2D(),
                       Flatten(),
                       Dense(64, activation = "relu"),
                       Dropout(0.1),
                       Dense(1, activation="sigmoid")
                       ])
# Compile Model
my_model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])
# Fit Model
history = my_model.fit(
    train_data_gen,
    steps_per_epoch=total_train // batch_size,
    epochs=epochs,
    validation_data=val_data_gen,
    validation_steps=total_val // batch_size
)<jupyter_output>Epoch 1/50
32/32 [==============================] - 2s 66ms/step - loss: 0.5109 - accuracy: 0.7745 - val_loss: 0.2195 - val_accuracy: 0.9148
Epoch 2/50
32/32 [==============================] - 2s 61ms/step - loss: 0.2451 - accuracy: 0.9022 - val_loss: 0.3978 - val_accuracy: 0.8523
Epoch 3/50
32/32 [==============================] - 2s 63ms/step - loss: 0.1814 - accuracy: 0.9261 - val_loss: 0.1644 - val_accuracy: 0.9261
Epoch 4/50
32/32 [==============================] - 2s 63ms/step - loss: 0.1423 - accuracy: 0.9421 - val_loss: 0.2635 - val_accuracy: 0.8920
Epoch 5/50
32/32 [==============================] - 2s 64ms/step - loss: 0.1330 - accuracy: 0.9461 - val_loss: 0.1697 - val_accuracy: 0.9318
Epoch 6/50
32/32 [==============================] - 2s 64ms/step - loss: 0.1201 - accuracy: 0.9521 - val_loss: 0.1672 - val_accuracy: 0.9261
Epoch 7/50
32/32 [==============================] - 2s 64ms/step - loss: 0.1225 - accuracy: 0.9561 - val_loss: 0.2557 - val_accuracy: 0.9034
Epoch 8/50
32[...]<jupyter_text>Custom CNN Model with Image ManipulationsTo simulate an increase in a sample of image, you can apply image manipulation techniques: cropping, rotation, stretching, etc. Luckily Keras has some handy functions for us to apply these techniques to our mountain and forest example. Simply, you should be able to modify our image generator for the problem. Check out these resources to help you get started: 1. [Keras `ImageGenerator` Class](https://keras.io/preprocessing/image/imagedatagenerator-class)2. [Building a powerful image classifier with very little data](https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html)<jupyter_code>train_image_generator = ImageDataGenerator(
    rescale=1./255,
    rotation_range=20,
    width_shift_range=0.1,
    height_shift_range=0.1,
    shear_range=0.1,
    zoom_range=0.1,
    horizontal_flip=True) # Generator for our training data

validation_image_generator = ImageDataGenerator(
    rescale=1./255,
        rotation_range=20,
        width_shift_range=0.1,
        height_shift_range=0.1,
        shear_range=0.1,
        zoom_range=0.1,
        horizontal_flip=True) # Generator for our validation data

train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,
                                                           directory=train_dir,
                                                           shuffle=True,
                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                           class_mode='binary')

val_data_gen = validation_image_generator.flow_from_directory(batch_size=batch_size,
                                                              directory=validation_dir,
                                                              target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                              class_mode='binary')


history = my_model.fit(
    train_data_gen,
    steps_per_epoch=total_train // batch_size,
    epochs=epochs,
    validation_data=val_data_gen,
    validation_steps=total_val // batch_size
)<jupyter_output>Found 533 images belonging to 2 classes.
Found 195 images belonging to 2 classes.
Epoch 1/50
32/32 [==============================] - 9s 277ms/step - loss: 0.2894 - accuracy: 0.9002 - val_loss: 0.1725 - val_accuracy: 0.9375
Epoch 2/50
32/32 [==============================] - 9s 268ms/step - loss: 0.1700 - accuracy: 0.9142 - val_loss: 0.2902 - val_accuracy: 0.8864
Epoch 3/50
32/32 [==============================] - 9s 267ms/step - loss: 0.1433 - accuracy: 0.9501 - val_loss: 0.6274 - val_accuracy: 0.7784
Epoch 4/50
32/32 [==============================] - 9s 273ms/step - loss: 0.2231 - accuracy: 0.9102 - val_loss: 0.1382 - val_accuracy: 0.9659
Epoch 5/50
32/32 [==============================] - 9s 268ms/step - loss: 0.2036 - accuracy: 0.9202 - val_loss: 0.2110 - val_accuracy: 0.8807
Epoch 6/50
32/32 [==============================] - 9s 268ms/step - loss: 0.1298 - accuracy: 0.9361 - val_loss: 0.1668 - val_accuracy: 0.9432
Epoch 7/50
32/32 [==============================] - 8s 265ms/step [...]