<p>This question is better suited to <a href="https://stats.stackexchange.com/">Cross Validated</a>. Based on my understanding from reading the 2014 GAN paper and other resources, the discriminator is kept at near optimal level such that it can provide better guidance to generator so it could improve. This is by first optimizing discriminator for some steps and then optimizing generator for one step.</p>
<p>Discriminator tries to maximize probability of assigning correct labels to both data from training example and the samples generated from generator. Simultaneously generator tries to minimize <code>log(1 - D(G(z)))</code> or maximize <code>log(D(G(z)))</code>.</p>
<p>At start both generator and discriminator are weak. Discriminator will mostly classify wrong and generated samples will be far from training distribution.</p>
<p>If generator can always completely fool discriminator then it will label all generated data as real. If the discriminator can perfectly distinguish between generated and real data then the generator will not know how to improve.</p>
<p>In practice, for a mini batch discriminator <code>D</code> is optimized (Section 3 of paper) for <code>k</code> steps then generator <code>G</code> is optimized for 1 step.</p>
<p><a href="https://i.stack.imgur.com/AMHeA.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/AMHeA.png" alt="enter image description here" /></a></p>
<p>If generator and discriminator are trained at the same time then generator won't get proper feedback from discriminator to generate better results and discriminator will misclassify generated, real data. I am not sure if it will converge and if it did it should take a lot of time.</p>
<p>If the discriminator is slightly better then it can catch some generated fake results, but not all. This in turn will allow generator improve its generation ability using feedback from discriminator to fool discriminator next time. Again in next iterations discriminator will keep improving and generator will also have to get better to fool discriminator. This result in gradual improvement of generated data that close matches training data.</p>
<p>Corrections for any mistakes welcome.</p>
<h4>References</h4>
<ul>
<li>GAN 2014 paper section 3, <a href="https://arxiv.org/pdf/1406.2661.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1406.2661.pdf</a>.</li>
<li>Coursera GAN training, <a href="https://www.coursera.org/lecture/build-basic-generative-adversarial-networks-gans/putting-it-all-together-gIAJ0" rel="nofollow noreferrer">https://www.coursera.org/lecture/build-basic-generative-adversarial-networks-gans/putting-it-all-together-gIAJ0</a>.</li>
</ul>
Can we train generator and discriminator at the same time?
<p>I recent start study GAN, but when I read about GAN's training algorithm it said the first step is train discriminator and then train generator in loop while they're not balance i'm confused about can't we train both at the same time??</p>