<p>Stable GAN Training is an open research problem. Nevertheless I can give you two tips. If you stick with the original GAN Training routine and do not have absolute knowledge of what you are doing, use the DCGAN Architecture with the available hyperparameters as described in their paper (<a href="https://arxiv.org/pdf/1511.06434.pdf%C3%AF%C2%BC%E2%80%B0" rel="nofollow noreferrer">https://arxiv.org/pdf/1511.06434.pdf%C3%AF%C2%BC%E2%80%B0</a>). GAN Training is highly volatile and using other hyperparameter will lead to mode collapse or vanishing gradients. </p>

<p>The easier route with GANs is to use the Wasserstein GAN. Those are quite stable with abritrary architectures. However, I strongly suggest to use the hyperparameter suggested in their paper, because for me the training also collapsed for different hyperparameters. Improved Wasserstein GAN: [<a href="https://arxiv.org/pdf/1704.00028.pdf]" rel="nofollow noreferrer">https://arxiv.org/pdf/1704.00028.pdf]</a></p>
Adjusting GAN hyperparameters
<p><a href="https://i.stack.imgur.com/hWfC5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/hWfC5.png" alt="enter image description here"></a>
<a href="https://i.stack.imgur.com/ZA0Wi.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ZA0Wi.png" alt="enter image description here"></a></p>

<p>As shown in the two pictures above,while training a DCGAN model ,the gradient is not stable and fluctuate greatly.Because of this reason,the model can't draw a perfect image,even to draw a image that be recognized by human eyes.Does anybody can tell me how to adjust the parameter such as dropout rate or learning rate or something else to make the model run better?I will be great thankful to you!
Here is the model I have made before(Build with Keras):</p>

<p>the discriminator :</p>

<p>learn rate is 0.0005</p>

<p>dropout rate is 0.6</p>

<p>batch_size is 25</p>

<pre><code>dis=Sequential()

dis.add(Conv2D(depth*1, 5, strides=2, input_shape=(56,56,3),padding='same',kernel_initializer='RandomNormal', bias_initializer='zeros'))

dis.add(LeakyReLU(alpha=alp))

dis.add(Dropout(dropout))

dis.add(Conv2D(depth*2, 5, strides=2, padding='same',kernel_initializer='RandomNormal', bias_initializer='zeros'))

dis.add(LeakyReLU(alpha=alp))

dis.add(Dropout(dropout))

dis.add(Conv2D(depth*4, 5, strides=2, padding='same',kernel_initializer='RandomNormal', bias_initializer='zeros'))

dis.add(LeakyReLU(alpha=alp))

dis.add(Dropout(dropout))

dis.add(Conv2D(depth*8,5,strides=1,padding='same',kernel_initializer='RandomUniform', bias_initializer='zeros'))

dis.add(LeakyReLU(alpha=alp))

dis.add(Dropout(dropout))

dis.add(Flatten())

dis.add(Dense(1))

dis.add(Activation('sigmoid'))

dis.summary()

dis.compile(loss='binary_crossentropy',optimizer=RMSprop(lr=d_lr))
</code></pre>

<p>the generator and GAN model:</p>

<p>learning rate is 0.0001</p>

<p>momentum is 0.9</p>

<pre><code>gen=Sequential()

gen.add(Dense(dim*dim*dep,input_dim=100))

gen.add(BatchNormalization(momentum=momentum))

gen.add(Activation('relu'))

gen.add(Reshape((dim,dim,dep)))

gen.add(Dropout(dropout))

gen.add(UpSampling2D())

gen.add(Conv2DTranspose(int(dep/2),5,padding='same',kernel_initializer='RandomNormal', bias_initializer='RandomNormal'))

gen.add(BatchNormalization(momentum=momentum))

gen.add(Activation('relu'))

gen.add(UpSampling2D())

gen.add(Conv2DTranspose(int(dep/4),5,padding='same',kernel_initializer='RandomNormal', bias_initializer='RandomNormal'))

gen.add(BatchNormalization(momentum=momentum))

gen.add(Activation('relu'))

gen.add(UpSampling2D())

gen.add(Conv2DTranspose(int(dep/8),5,padding='same',kernel_initializer='RandomNormal', bias_initializer='RandomNormal'))

gen.add(BatchNormalization(momentum=momentum))

gen.add(Activation('relu'))

gen.add(Conv2DTranspose(3,5,padding='same',kernel_initializer='RandomNormal', bias_initializer='RandomNormal'))

gen.add(Activation('sigmoid'))

gen.summary()


GAN=Sequential()

GAN.add(gen)

GAN.add(dis)

GAN.compile(loss='binary_crossentropy',optimizer=RMSprop(lr=g_lr))
</code></pre>