Interpreting generative adversarial network (GAN) training as approximate divergence minimization has been
 theoretically insightful, has spurred discussion, and has lead to theoretically and practically interesting
 extensions such as f-GANs and Wasserstein GANs. For both classic GANs and f-GANs, there is an original variant of training and a "non-saturating" variant which uses an alternative form of generator gradient. The original variant is theoretically easier to study, but for GANs the alternative variant performs better in practice. The non-saturating scheme is often regarded as a simple modification to deal with optimization issues, but we show that in fact the non-saturating scheme for GANs is effectively optimizing a reverse KL-like f-divergence. We also develop a number of theoretical tools to help compare and classify f-divergences. We hope these results may help to clarify some of the theoretical discussion surrounding the divergence minimization view of GAN training. Generative adversarial networks (GANs) (Goodfellow et al., 2014) have enjoyed remarkable progress in recent years, producing images of striking fidelity, resolution and coherence (Karras et al., 2018; Miyato et al., 2018; Brock et al., 2018; Karras et al., 2019) . There has been much progress in both theoretical and practical aspects of understanding and performing GAN training (Nowozin et al., 2016; Mescheder et al., 2018; Gulrajani et al., 2017; Sønderby et al., 2017; Miyato et al., 2018; Karras et al., 2018; Brock et al., 2018; Karras et al., 2019) . One of the key considerations for GAN training is the scheme used to update the generator and critic. A rich avenue of developments has come from viewing GAN training as divergence minimization. Goodfellow et al. (2014) showed the conventional GAN training can be viewed as approximately minimizing the Jensen-Shannon divergence. f-GANs (Nowozin et al., 2016) approximately minimize f-divergences such as reverse KL in a principled way. Wasserstein GANs approximately minimize the Wasserstein metric, and combine solid theoretical underpinnings with strong practical results. Nevertheless a relatively unprincipled "non-saturating" scheme (Goodfellow et al., 2014) has continued to obtain groundbreaking results (Karras et al., 2019) and remains a state-of-the-art approach (Lucic et al., 2018) . The effect of the non-saturating scheme on training dynamics, and in particular whether it can be viewed as divergence minimization, has been source of discussion and some confusion since the original formulation of GAN training (Goodfellow et al., 2014) . The main result of this paper is to show that the non-saturating scheme approximately minimizes the f-divergence 4 KL( 1 2 p + 1 2 q p), which we refer to as the softened reverse KL divergence ( §6). This puts non-saturating training on a similar footing to Wasserstein GANs as a theoretically sound approach with strong empirical results. We also discuss how our results relate to previous attempts at this problem and attempt to clarify some of the confusion surrounding the divergence minimization view of non-saturating training. In order to better understand the qualitative behavior of different divergences such as softened reverse KL, we develop several tools. We show how to write f-divergences in a symmetry-preserving way, allowing easy visual comparison of f-divergences in a way that reflects their qualitative properties ( §7). We develop a rigorous formulation of tail weight which generalizes the notions of modeseeking and covering behavior ( §8). Using these tools we show that the softened reverse KL divergence is fairly similar to the reverse KL but very different to the Jensen-Shannon divergence approximately minimized by the original GAN training scheme. The precise practical effect of the non-saturating scheme and whether it can be motivated in a principled way have been a source of discussion and some confusion. In this section we review previous attempts to view non-saturating gradients as a form of divergence minimization. The original GAN paper claims that, compared to the saturating training scheme based on the Jensen-Shannon divergence, the non-saturating training scheme "results in the same fixed point of the dynamics of G and D but provides much stronger gradients early in learning." (Goodfellow et al., 2014, Section 3) . It is true that the original and non-saturating generator gradients give the same final result in the non-parametric case where q is unrestricted, but this is fairly trivial since both gradients lead to q = p, as do all divergences. It is even true that the dynamics of training are essentially the same for the original and non-saturating gradients when q ≈ p, but again this is fairly trivial since all f-divergences agree in this regime, as discussed in §3. However the "fixed point of the dynamics" is certainly not the same in the general case of parametric q (see §G for an empirical demonstration). Our results provide a precise way to view the relationship between saturating and non-saturating generator gradients: They are optimizing different f-divergences. The original f-GAN paper presents a simple argument that the "non-saturating" training scheme has the same fixed points and that the original and non-saturating generator gradients have the same direction (Nowozin et al., 2016, Section 3.2) 1 . However this argument is erroneous. It is true that if p ≈ q then (f * ) (f (u)) is approximately 1 everywhere, and so the original and non-saturating generator gradients are approximately equal, but this is true of any f-divergence. There is no guarantee that the regime p ≈ q will ever be approached in the general case where q belongs to a parametric family, it is not the case that the original and non-saturating generator gradients point in approximately the same direction in general (see §G for an empirical demonstration). In fact, the non-saturating form of generator gradient can have completely different qualitative behavior. For example, we show that the non-saturating KL scheme in fact optimizes reverse KL. A recent paper showed experimentally that the non-saturating generator gradient can successfully learn a distribution in a case where optimizing Jensen-Shannon divergence should fail, and used this to argue that perhaps it is not particularly helpful to view GANs as optimizing Jensen-Shannon divergence (Fedus et al., 2018) . The divergence optimized in practice for parametric critics is not exactly the divergence which would be optimized by the theoretically optimal critic, and this distinction seems particularly important in the situation where p and q initially have non-overlapping support. However the fact that non-saturating training is not optimizing Jensen-Shannon is also highly relevant to this discussion, since the gradient in the limit of zero noise is zero for Jensen-Shannon but sizeable for softened reverse KL. Thus the success of non-saturating GAN training in practice may be as much due to its optimizing a different divergence as it is to using an inexact critic. Arjovsky and Bottou correctly recognize that the non-saturating generator gradient results in approximately minimizing a different objective function and derive the function for classic GANs (Arjovsky & Bottou, 2017, Section 2.2.2) . The objective function there is expressed as KL(q p) − 2 JS(p, q) (1) which is a slightly convoluted form of the expression 2 KL( 1 2 p+ 1 2 q p) we derive below. The paper suggests the negative sign of the second term is "pushing for the distributions to be different, which seems like a fault in the update", whereas our expression for the divergence makes it clear that this is not an issue. Poole et al. (2016) present a very similar view to that presented in this paper, including recognizing that the generator and critic may be trained to optimize different f-divergences and interpreting the classic non-saturating generator gradient as a hybrid scheme of this form where the generator gradient is based on a new f-divergence (Poole et al., 2016) . However the f-divergence derived there is f (u) = log(1 + u −1 ), which differs from (50) by a factor of u + 1. We refer to this as the improved generator objectives for GANs (IGOG) divergence. It can be written as (1+u) 2 u 2 , and has (2, 0) tail weights. Figure 3 shows that this divergence is qualitatively quite similar to the softened reverse KL but is not identical. The source of the discrepancy between our results and theirs is matching the value instead of the gradient, and is described in detail in §A.
Non-saturating GAN training effectively minimizes a reverse KL-like f-divergence. This paper proposes a useful expression of the class of f-divergences, investigates theoretical properties of popular f-divergences from newly developed tools, and investigates GANs with the non-saturating training scheme.