In this work we construct flexible joint distributions from low-dimensional conditional semi-implicit distributions. Explicitly defining the structure of the approximation allows to make the variational lower bound tighter, resulting in more accurate inference. Many recent advances in variational inference have been focused on different ways to estimate or bound the KL divergence between two complicated distributions. They made it possible to perform variational inference with hierarchical distributions (Ranganath et al., 2016; Titsias and Ruiz, 2018; Sobolev and Vetrov, 2019) , semi-implicit distributions (Yin and Zhou, 2018; Molchanov et al., 2019) and even fully implicit distributions (Mescheder et al., 2017; Shi et al., 2017; Huszár, 2017) . While these methods work well for low-dimensional cases, they can misbehave when the dimensionality of the problem grows. In this work, we focus on semi-implicit variational inference, and consider structured multi-dimensional distributions. We show that taking this structure into account, we can obtain a much tighter entropy bound and, consequentially, a much tighter evidence lower bound. We also demonstrate that structured semi-implicit variational inference can successfully capture the multi-modal nature of the posterior distribution in deep Gaussian processes, and show a way to construct and learn an autoregressive semi-implicit model.
Utilizing the structure of distributions improves semi-implicit variational inference