While much recent work has targeted learning deep discrete latent variable models with variational inference, this setting remains challenging, and it is often necessary to make use of potentially high-variance gradient estimators in optimizing the ELBO. As an alternative, we propose to optimize a non-ELBO objective derived from the Bethe free energy approximation to an MRF's partition function. This objective gives rise to a saddle-point learning problem, which we train inference networks to approximately optimize. The derived objective requires no sampling, and can be efficiently computed for many MRFs of interest. We evaluate the proposed approach in learning high-order neural HMMs on text, and find that it often outperforms other approximate inference schemes in terms of true held-out log likelihood. At the same time, we find that all the approximate inference-based approaches to learning high-order neural HMMs we consider underperform learning with exact inference by a significant margin. There has been much recent interest in learning deep generative models with discrete latent variables BID29 BID28 BID12 BID25 BID15 Lee et al., 2018, inter alia) , especially in the case where these latent variables have structure -that is, where the interdependence between the discrete latents is modeled. Most recent work has focused on learning these models with variational inference BID14 , and in particular with variational autoencoders (VAEs) BID17 BID32 .Variational inference has a number of convenient properties, including that it involves the maximization of the evidence lower-bound (ELBO), a lower bound on the log marginal likelihood of the data. At the same time, when learning models with discrete latent variables variational inference may require the use of potentially high-variance gradient estimators, which are obtained during learning by sampling from the variational posterior; see Appendix A for an empirical investigation into the variance of various popular estimators when learning neural text HMMs with VAEs.In this paper we investigate learning discrete latent variable models with an alternative objective to the ELBO. In particular , we propose to approximate the intractable log marginal likelihood with an objective deriving from the Bethe free energy BID1 , a quantity which is intimately related to loopy belief propagation (LBP) BID30 BID45 BID9 BID9 , and which is the basis for "outer approximations" to the marginal polytope BID39 . The Bethe free energy is attractive because if all the factors in the factor graph associated with the model have low degree, it can often be evaluated efficiently, without any need for approximation by sampling (see Section 2). Of course, requiring all factors in the factor graph to be of low degree severely limits the expressiveness of directed graphical models. It does not, however , limit the expressiveness of markov random fields (MRFs) (i.e., undirected graphical models) as severely, since we can simply have an extremely loopy MRF, with arbitrary pairwise factors; see FIG1 (c) and Section 2.2.We accordingly propose to learn deep, undirected graphical models with latent variables, using a saddlepoint objective that makes use of the Bethe free energy approximation to the model's partition functions. We further amortize inference by using "inference networks" BID36 BID17 BID13 BID38 in optimizing the saddle-point objective. Unlike the ELBO, our objective will not form a lower bound on the log marginal likelihood, but an approximation to it. At the same time (and unlike other recent work on MRFs with a variational flavor BID21 BID23 ), this objective can be optimized efficiently, without sampling, and in our experiments in learning neural HMMs on text it outperforms other approximate inference methods in terms of held out log likelihood. We emphasize, however , that despite the improvement observed when training with the proposed objective, in our experiments all approximate inference methods were found to significantly underperform learning with exact inference; see Section 4.3. We begin with the results obtained by maximizing the true log marginal likelihood of the training data under both the directed ("Full" in Table 1 ) and undirected models ("Pairwise MRF" in Table 1 ), by backpropagating gradients through the relevant dynamic programs. These results establish how well our models perform under exact inference, and are shown in the last row of each subtable in Table 1 . We see that perplexities are roughly comparable between the directed and undirected models when trained with exact inference.We now consider the remaining directed HMM results of Table 1 , where the models are trained with approximate inference. In the first row of each "Full" subtable there, we show the result of maximizing the ELBO using a mean field-style posterior approximation and the REINFORCE BID43 gradient estimator, with an input-dependent baseline to reduce variance BID29 . The results are quite poor, with this approximate inference scheme leading to a gain of almost 200 points in perplexity over exact inference. Using the tighter IWAE BID3 objectives improves performance slightly in all cases, though the most dramatic performance improvement comes from using a first-order HMM posterior in maximizing the ELBO, which can be sampled from exactly using quantities calculated with the forward algorithm BID31 BID4 BID34 BID49 . While these results are encouraging, note that in general we may not have an exact dynamic program for sampling from a lower-order structured model, and that moreover we still appear to incur a perplexity penalty of more than 100 points over exact inference; see Appendix A for an empirical comparison of the variance of these estimators.Moving to the MRF results, the second row of each "Pairwise MRF" subtable in Table 1 contains the results of optimizing F as a saddle point problem. While this approach too underperforms exact inference by approximately 100 points in perplexity, somewhat remarkably it manages to consistently outperform the best approximate inference results for the directed models by a fair margin. The first row of each "Pairwise MRF" subtable in Table 1 attempts to determine whether the jump in perplexity when moving to the F objective is due to the approximate inference or to the approximate objective, by minimizing the F objective using the exact marginals, as calculated by a dynamic program. (Note that this is not equivalent to the negative log marginal likelihood, since the factor graphs are loopy). Interestingly, we see that this performs almost as well as the exact objective, suggesting that, at least for HMM models, the F objective is reasonable, and approximate inference remains the problem.Despite these encouraging results, we note that there are several drawbacks to the proposed approach. In particular, we find that in practice F indeed can over-or under-estimate perplexity. Moreover, while ELBO values are not perfectly correlated with their corresponding true perplexities, values of F seem even less correlated, which necessitates finding correlated proxies of perplexity that may be monitored during training. Finally, we note that explicitly calculating the projection onto the nullspace of A may be prohibitive for some models (e.g., large RBMs BID35 ), and so other approaches to tackling the constrained optimization problem are likely necessary. We have presented an objective for learning latent-variable MRFs based on the Bethe approximation to the partition function, which can often be efficiently evaluated and requires no sampling. This objective leads to slightly better held-out perplexities than other approximate inference methods when learning neural HMMs. Future work will examine scaling the proposed method to larger, non-sequential MRFs, and whether F -like objectives can be made to better correlate with the true perplexity.
Learning deep latent variable MRFs with a saddle-point objective derived from the Bethe partition function approximation. A method for learning deep latent-variable MRF with an optimization objective that utilizes Bethe free energy, that also solves the underlying constraints of Bethe free energy optimizations. An objective for learning latent variable MRFs based on Bethe free energy and amortized inference, different from optimizing the standard ELBO.