<p>You might have misread the source code, the first sample you gave is not averaging the resut of <code>D</code> to compute its loss but instead uses the binary cross-entropy.</p>
<p>To be more precise:</p>
<ul>
<li><p>The first method (<strong>&quot;GAN&quot;</strong>) uses the BCE loss to compute the loss terms for <code>D</code> and <code>G</code>. The standard GAN optimization objective for <code>D</code> is to minimize <code>E_x[log(D(x))] + E_z[log(1-D(G(z)))]</code>. <br><a href="https://github.com/NUS-Tim/Pytorch-WGAN/blob/master/models/gan.py#L83" rel="nofollow noreferrer"><em>Source code</em></a>:</p>
<pre><code>outputs = self.D(images)
d_loss_real = self.loss(outputs.flatten(), real_labels) # &lt;- bce loss
real_score = outputs

# Compute BCELoss using fake images
fake_images = self.G(z)
outputs = self.D(fake_images)
d_loss_fake = self.loss(outputs.flatten(), fake_labels) # &lt;- bce loss
fake_score = outputs

# Optimizie discriminator
d_loss = d_loss_real + d_loss_fake
self.D.zero_grad()
d_loss.backward()
self.d_optimizer.step()
</code></pre>
<p>For <code>d_loss_real</code> you optimize towards <code>1</code>s (output is considered <em>real</em>), while <code>d_loss_fake</code> optimizes towards <code>0</code>s (output is considered <em>fake</em>).</p>
</li>
<li><p>While the second (<strong>&quot;WCGAN&quot;</strong>) uses the Wasserstein loss (<a href="https://arxiv.org/pdf/1704.00028v3.pdf" rel="nofollow noreferrer">ref</a>) whereby we maximise for D the loss: <code>E_x[D(x)] - E_z[D(G(z))]</code>. <br><a href="https://github.com/NUS-Tim/Pytorch-WGAN/blob/master/models/wgan_gradient_penalty.py#L174" rel="nofollow noreferrer"><em>Source code</em></a>:</p>
<pre><code># Train discriminator
# WGAN - Training discriminator more iterations than generator
# Train with real images
d_loss_real = self.D(images)
d_loss_real = d_loss_real.mean()
d_loss_real.backward(mone)

# Train with fake images
z = self.get_torch_variable(torch.randn(self.batch_size, 100, 1, 1))

fake_images = self.G(z)
d_loss_fake = self.D(fake_images)
d_loss_fake = d_loss_fake.mean()
d_loss_fake.backward(one)

# [...]

Wasserstein_D = d_loss_real - d_loss_fake
</code></pre>
<p>By doing <code>d_loss_real.backward(mone)</code> you backpropage with a gradient of opposite sign, <em>i.e.</em> its's a gradient ascend, and you end up maximizing <code>d_loss_real</code>.</p>
</li>
</ul>
Difference between WGAN and WGAN-GP (Gradient Penalty)
<p>I just find that in the code here:</p>
<p><a href="https://github.com/NUS-Tim/Pytorch-WGAN/tree/master/models" rel="nofollow noreferrer">https://github.com/NUS-Tim/Pytorch-WGAN/tree/master/models</a></p>
<p>The &quot;generator&quot; loss, <code>G</code>, between WGAN and WGAN-GP is different, for WGAN:</p>
<pre><code>g_loss = self.D(fake_images)
g_loss = g_loss.mean().mean(0).view(1)
g_loss.backward(one) # !!!
g_cost = -g_loss
</code></pre>
<p>But for WGAN-GP:</p>
<pre><code>g_loss = self.D(fake_images)
g_loss = g_loss.mean()
g_loss.backward(mone) # !!!
g_cost = -g_loss
</code></pre>
<p>Why one is <code>one=1</code> and another is <code>mone=-1</code>?</p>