<p>This doesn't look like a checkerboard artifact honestly. Also I don't think discriminator would be the problem, it's usually about image restoration (generator or decoder).</p>
<p>Took a quick look at the MUNIT and what they use in <code>Decoder</code> is <code>torch.nn.Upsample</code> with nearest neighbor upsampling (exact code line <a href="https://github.com/NVlabs/MUNIT/blob/master/networks.py#L232" rel="nofollow noreferrer">here</a>).</p>
<p>You may try to use <code>torch.nn.Conv2d</code> followed by <a href="https://pytorch.org/docs/stable/generated/torch.nn.PixelShuffle.html" rel="nofollow noreferrer"><code>torch.nn.PixelShuffle</code></a>, something like this:</p>
<pre><code>import torch

in_channels = 32
upscale_factor = 2
out_channels = 16

upsampling = torch.nn.Sequential(
    torch.nn.Conv2d(
        in_channels,
        out_channels * upscale_factor * upscale_factor,
        kernel_size=3,
        padding=1,
    ),
    torch.nn.PixelShuffle(upscale_factor),
)

image = torch.randn(1, 32, 16, 16)

upsampling(image).shape  # [1, 16, 32, 32]
</code></pre>
<p>This allows neural network to learn how to upsample the image instead of merely using <code>torch.nn.Upsample</code> which the network has no control over (and using below trick it should also be free of checkerboard artifacts).</p>
<p>Additionally, ICNR initialization for <code>Conv2d</code> should also help (possible implementation <a href="https://github.com/pytorch/pytorch/pull/5429/commits/4fea7cb83366a95b2eb9f64a9161b35d0e412f66" rel="nofollow noreferrer">here</a> or <a href="https://github.com/szymonmaszke/torchlayers/blob/master/torchlayers/upsample.py#L116" rel="nofollow noreferrer">here</a>). This init scheme initializes weights to act similar to nearest neighbor upsampling at the beginning (research paper <a href="https://arxiv.org/pdf/1707.02937.pdf" rel="nofollow noreferrer">here</a>).</p>
Average Pooling layer in Deep Learning and gradient artifacts
<p>I know that in Convolution layers the kernel size needs to be a multiplication of stride or else it will produce artefacts in gradient calculations like the checkerboard problem.
Now does it also work like that in Pooling layers? I read somewhere that max pooling can also cause problems like that. Take this line in the discriminator for example:</p>
<pre><code>  self.downsample = nn.AvgPool2d(3, stride=2, padding=1, count_include_pad=False)
</code></pre>
<p>I have a model (MUNIT) with it, and this is the image it produced:</p>
<p><a href="https://i.stack.imgur.com/sxrEG.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/sxrEG.png" alt="enter image description here" /></a></p>
<p>It looks like the checkerboard problem, or at least a gradient problem but I checked my Convolution layers and didn't found the error described above. They all are of size 4 with stride 2 or an uneven size with stride of 1.</p>