<p>Well, <code>MobileNets</code> and all other imagenet based models down-sampling the image for 5 times(224 -> 7) and then do <code>GlobalAveragePooling2D</code> and then the output layers. </p>

<p>I think using 32*32 images on these models directly won't give you a good result, as the tensor shape would be 1*1 even before the <code>GlobalAveragePooling2D</code>.</p>

<p>Maybe you should try resize the image to like <a href="https://github.com/keras-team/keras-applications/blob/master/keras_applications/mobilenet_v2.py#L249" rel="nofollow noreferrer">96*96</a> or remove the first <a href="https://github.com/keras-team/keras-applications/blob/master/keras_applications/mobilenet_v2.py#L314" rel="nofollow noreferrer"><code>stride=2</code></a>. Take the <a href="https://arxiv.org/pdf/1707.07012.pdf" rel="nofollow noreferrer">NASNet paper</a> as reference, they use 4 poolings in both Cifar and ImageNet versions while only ImageNet version has <code>stride=2</code> in the first Convolution layer.</p>
Training Keras MobileNetV2 on CIFAR-100 (from scratch)
<p>I want to train MobileNetV2 from scratch on CIFAR-100 and I get the following results where it just stops learning after some while. </p>

<p><a href="https://i.stack.imgur.com/aNgEJ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/aNgEJ.png" alt="enter image description here"></a></p>

<p>Here is my code. I would like to see at least 60-70% validation accuracy and I wonder whether I have to pre-train it on imagenet or whether it is because CIFAR100 is just 32x32x3?
Due to some restrictions, I am using Keras 2.2.4 with tensorflow 1.12.0. </p>

<pre><code>from keras.applications.mobilenet_v2 import MobileNetV2
[..]

(x_train, y_train), (x_test, y_test) = cifar100.load_data()
x_train = x_train / 255
x_test = x_test / 255
y_train = np_utils.to_categorical(y_train, 100)
y_test = np_utils.to_categorical(y_test, 100)

input_tensor = Input(shape=(32,32,3))
x = MobileNetV2(include_top=False,
                  weights=None,
                  classes=100)(input_tensor)
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation='relu')(x)
x = Dense(512, activation='relu')(x)
preds = Dense(100, activation='softmax')(x)
model = Model(inputs=[input_tensor], outputs=[preds])

optimizer = Adam(lr=1e-3)
model.compile(loss="categorical_crossentropy",
                           optimizer=optimizer,
                           metrics=['accuracy'])

epochs = 300
batch_size = 64

callbacks = [ReduceLROnPlateau(monitor='val_loss', factor=np.sqrt(0.1), cooldown=0, patience=10, min_lr=1e-6)]
generator = ImageDataGenerator(rotation_range=15,
                                   width_shift_range=5. / 32,
                                   height_shift_range=5. / 32,
                                   horizontal_flip=True)
generator.fit(x_train)
model.fit_generator(generator.flow(x_train, y_train),
                             validation_data=(x_test, y_test),
                             steps_per_epoch=(len(x_train) // batch_size),
                             epochs=epochs, verbose=1,
                             callbacks=callbacks)
</code></pre>