decoder. Then a loss which measures the difference between the latent variable of CVAE and predicted variable from labeling network is added to the original objective function of the CVAE. Since this loss pulls the encoder towards the “best encoder” approximated by the labeling network and in the meanwhile original CVAE pulls encoder to the prior, an equilibrium will be reached where KL-vanishing can be avoided. Additionally, the labeling network introduces a continuous label for each target which essentially reflects the structural constraints of the latent space. Therefore, it guarantees each $z$ in the latent space corresponds to a unique target, thus improves the coverage of the generations in target space. We alternatively train the “labeling network" and the CVAE structure, and call this model Self Labeling Conditional Variational Auto Encoder (SLCVAE).

![Example of recommendation texts of a dress. Some of them emphasize on the linen material while some others emphasize on the color etc.[]{data-label="fig:one-to-many-example"}](case){width="0.95\linewidth"}

In summary our main contributions are: Firstly, we point out that the current CVAE objective function tends to encounter the *KL-Vanishing problem* due to the lack of explicit constraints on the connection between the latent variable and targets. Secondly, we propose the self labeling mechanism 