What would be learned by variational autoencoder(VAE) and what influence the disentanglement of VAE? This paper tries to preliminarily address VAE's intrinsic dimension, real factor, disentanglement and indicator issues theoretically in the idealistic situation and implementation issue practically through noise modeling perspective in the realistic case.   On intrinsic dimension issue, due to information conservation, the idealistic VAE learns and only learns intrinsic factor dimension. Besides, suggested by mutual information separation property, the constraint induced by Gaussian prior to the VAE objective encourages the information sparsity in dimension. On disentanglement issue,   subsequently, inspired by information conservation theorem the clarification on disentanglement in this paper is made. On real factor issue, due to factor equivalence, the idealistic VAE possibly learns any factor set in the equivalence class.   On indicator issue, the behavior of current disentanglement metric is discussed, and several performance indicators regarding the disentanglement and generating influence are subsequently raised to evaluate the performance of VAE model and to supervise the used factors. On implementation issue, the experiments under noise modeling and constraints empirically testify the theoretical analysis and also show their own characteristic in pursuing disentanglement. Variational AutoEncoder(VAE)s BID9 , Rezende et al. (2014) ) have shown their powerful human-like abilities: modelling causal relationship, unsupervisedly extracting disentangled factors/representation BID1 ) and generating signals with abundant diversities in a "latent-factor-controllable" way. Those capabilities enable the knowledge transferring through shared causes/factors among different tasks/experiences, emphasized as the important human advantages against the current machine by BID12 and compling with the ideal mental imagery mechanism in memory and thinking. Benefitted from those capabilities, VAEs have been widely applied to various applications, including disentangled representations learning of images and time series BID6 , BID11 , Mathieu et al. (2016) , BID3 ), few-shot and transfer learning (Rezende et al. (2016) , BID8 , BID7 ), causal relationships modeling (Louizos et al. (2017) ), pixel trajectory predicting (Walker et al. (2016) ), joint multi-modal inference learning (Suzuki et al. (2016) ), increasing diversity in imitation learning (Wang et al. (2017) ), generation with memory BID16 ) and etc.However, the lack of public theoretical study regarding the generating and inference procedure induced by VAEs is tripping the research process: Idealistic VAE learns and only learns the intrinsic factor dimension. The illustration of the information conservation theorem 1. Suppose that the oracle data, denoted by random variable x, is generated by y (with P independent unit Gaussian random variables) with a homeomorphism mapping x = φ(y ). Idealistic VAE will be forced to learn the factor z (with H independent unit Gaussian random variables) that generates the x with a homeomorphism mapping x = ψ(z). It yields z = ψ −1 • φ(y) and y = φ −1 • ψ(z). Then according to the information conservation theorem, it must hold that H = P . Gaussian-VAE (Kingma & Welling (2013) , Rezende et al. (2014) ) is an scalable unsupervised representation learning model BID6 ), and since Gaussian distribution can be continuously and reversibly mapping to many other distributions, the theoretical analysis on it is also instructive for other continuous latent factors VAE. DISPLAYFORM0
This paper tries to preliminarily address the disentanglement theoretically in the idealistic situation and practically through noise modelling perspective in the realistic case. Studies the importance of the noise modelling in Gaussian VAE and proposes to train the noise using Empirical-Bayes like fashion. Modifying how noise factors are treated when developing VAE models