the KL divergence term in ELBO becomes 0, and we call this phenomenon *KL-Vanishing*.

Although the *KL-Vanishing* point is a solution of Equation. \[vae\_elbo\], it is not actually we want. When *KL-Vanishing* takes place, Equation. \[vae\_elbo\] degenerates to $\log{p(x)}$. When the decoder is modeled by an RNN structure for text generation, it easily converges to a average language model of target sentences without regarding to $z$ under the degenerated objective. As a result, $z$ losses its ability of affecting $x$ and the decoder fits the average behavior of all $x$.

\[fig:latentspace\]

Previous works tried to avoid the *KL-Vanishing problem* by either breaking fact (1) or fact (2). Some of them put efforts into preventing $q(z|x)$ from collapsing to $p(z)$ by slowing down the optimizing procedure of the KL term. Others attempt to force the decoder to depends on the $z$ by weakening the decoder or making encoder more complicated. However, such methods fall into a dilemma: On the one hand, the objective function is optimized to reach its maximal. On the other hand, the optimal point should be avoided to prevent the occurrence of *KL-Vanishing*. These methods tried to find a good balance between the two contradictory facts without an 