<p>I guess you are using your own Dataset. If not consider using the same hyperparameter as in the DCGAN paper and first reproduce their results. Then use that network for your dataset. DCGANs especially with cross-entropy loss are known to be really tricky and extremely hyperparameter sensitive to get to run (see <a href="https://arxiv.org/abs/1711.10337" rel="nofollow noreferrer">https://arxiv.org/abs/1711.10337</a>). Especially, if you did not constrain your discriminator with some kind of gradient penalties (Gradient Penalty or Spectral norm).</p>

<p>In GAN's using your own ideas for the hyperparameters is mostly a bad idea, unless you either really know what you are doing or have loads of GPU power to burn on a large scale search. </p>
DC-GAN: Discriminator loss going up while generator loss goes down
<p>I cannot tell if this error is due to a technical mistake or hyper-parameters, but my DC-GAN's discriminator loss starts low and gradually climbs higher, slowing down around 8, whereas my generator loss goes way down. I ended it at about 60,000 epochs. Funny enough, the discriminator accuracy seems to be floating around 20-50%. Does anybody have any suggestions to fix the problem? Any help is appreciated.</p>

<p><strong>Important Info</strong></p>

<ul>
<li>Data Format: 472 320x224 Color PNG files.</li>
<li>Optimizer: <code>Adam(0.0002, 0.5)</code></li>
<li>Loss: Binary Cross-Entropy</li>
</ul>

<p><strong>A generated image after 50,000+ epochs: (Supposed to be a sneaker on a white background)</strong></p>

<p><a href="https://i.stack.imgur.com/aX5il.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/aX5il.png" alt="enter image description here"></a></p>

<p><strong>Discriminator Model:</strong></p>

<pre class="lang-python prettyprint-override"><code>    def build_discriminator(self):

        img_shape = (self.img_size[0], self.img_size[1], self.channels)

        model = Sequential()

        model.add(Conv2D(32, kernel_size=self.kernel_size, strides=2, input_shape=img_shape, padding="same"))  # 192x256 -&gt; 96x128
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))

        model.add(Conv2D(64, kernel_size=self.kernel_size, strides=2, padding="same"))  # 96x128 -&gt; 48x64
        model.add(ZeroPadding2D(padding=((0, 1), (0, 1))))
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))
        model.add(BatchNormalization(momentum=0.8))

        model.add(Conv2D(128, kernel_size=self.kernel_size, strides=2, padding="same"))  # 48x64 -&gt; 24x32
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))
        model.add(BatchNormalization(momentum=0.8))

        model.add(Conv2D(256, kernel_size=self.kernel_size, strides=1, padding="same"))  # 24x32 -&gt; 12x16
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))

        model.add(Conv2D(512, kernel_size=self.kernel_size, strides=1, padding="same"))  # 12x16 -&gt; 6x8
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))

        model.add(Flatten())
        model.add(Dense(1, activation='sigmoid'))

        model.summary()

        img = Input(shape=img_shape)
        validity = model(img)

        return Model(img, validity)
</code></pre>

<p><strong>Generator Model:</strong></p>

<pre class="lang-python prettyprint-override"><code>    def build_generator(self):

        noise_shape = (100,)

        model = Sequential()
        model.add(
            Dense(self.starting_filters * (self.img_size[0] // (2 ** self.upsample_layers))  *  (self.img_size[1] // (2 ** self.upsample_layers)),
                  activation="relu", input_shape=noise_shape))
        model.add(Reshape(((self.img_size[0] // (2 ** self.upsample_layers)),
                           (self.img_size[1] // (2 ** self.upsample_layers)),
                           self.starting_filters)))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 6x8 -&gt; 12x16
        model.add(Conv2D(1024, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 12x16 -&gt; 24x32
        model.add(Conv2D(512, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 24x32 -&gt; 48x64
        model.add(Conv2D(256, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 48x64 -&gt; 96x128
        model.add(Conv2D(128, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 96x128 -&gt; 192x256
        model.add(Conv2D(64, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(Conv2D(32, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(Conv2D(self.channels, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("tanh"))

        model.summary()

        noise = Input(shape=noise_shape)
        img = model(noise)

        return Model(noise, img)
</code></pre>