0.125 (0.0091) & 0.161 (0.0051) & 0.115 (0.0082)[\
]{} \[-1.2em\][Mixture]{} & \[-0.5em\][$D$]{} & 0.127 (0.0027) & 0.098 (0.0061) & 0.104 (0.0015) & 0.085 (0.0019)[\
]{} & \[-0.5em\][$W$]{} & 0.320 (0.0025) & 0.165 (0.0059) & 0.276 (0.0021) & 0.197 (0.0031)[\
]{} \[-1.2em\][High-Dim.]{} & \[-0.5em\][$D$]{} & 0.093 (0.0010) & 0.080 (0.0012) & 0.092 (0.0008) & 0.065 (0.0005)[\
]{} & \[-0.5em\][$W$]{} & 0.222 (0.0027) & 0.158 (0.0029) & 0.212 (0.0030) & 0.093 (0.0015)[\
]{}

To further take into account the randomness in data generation, we simulate 30 replications of the data set in each setting, and compute the mean and standard errors of the distance metrics, shown in Table \[tab:latent\_estimation\_rep\]. Also included in this table is a high-dimensional data set with sample size $n=10000$ and dimension $p=100$: each dimension independently follows an exponential distribution with mean 2. We pre-train this data set using VAE for 10000 epochs, and then fit each method with 1000 more epochs. The mean and standard errors are computed over all dimensions. Both Figure \[fig:latent\_estimation\] and Table \[tab:latent\_estimation\_rep\] indicate that SAGD provides good bias reduction results for VAE in both simple and high-dimensional settings, which highlights the importance of optimizing the correct objective function in model fitting.

Generative Model 