We consider the question of how to assess generative adversarial networks, in particular with respect to whether or not they generalise beyond memorising the training data. We propose a simple procedure for assessing generative adversarial network performance based on a principled consideration of what the actual goal of generalisation is. Our approach involves using a test set to estimate the Wasserstein distance between the generative distribution produced by our procedure, and the underlying data distribution. We use this procedure to assess the performance of several modern generative adversarial network architectures. We find that this procedure is sensitive to the choice of ground metric on the underlying data space, and suggest a choice of ground metric that substantially improves performance.   We finally suggest that attending to the ground metric used in Wasserstein generative adversarial network training may be fruitful, and outline a concrete pathway towards doing so.
Assess whether or not your GAN is actually doing something other than memorizing the training data. Aims to provide a quality measure/test for GANs and proposes to evaluate the current approximation of a distribution learnt by a GAN by using Wasserstein distance between two distributions made of a sum of Diracs as a baseline performance.  This paper proposed a procedure for assessing the performance of GANs by re-considering the key of observation, using the procedure to test and improve the current GANs