Stochastic Changes {#bg:sec-stochastic}
------------------

@kingma2013auto introduce variational auto-encoder, where you can stochastically draw a continuous latent variable ${\mathbf{z}}$ from a Gaussian distribution. The initialization of the generator ${\mathbf{h}}_0$ is based on this latent variable which is drawn. @bowman-etal-2016-generating use this concept for generating sentences from this continuous latent representation. This process of changing the encoder state ${\mathbf{h}}_e$ is can only be used with Kullback-Leibler (KL) Divergence training objective described in [§\[bg:sec-loss-kl\]]{}.

In [@wang-etal-2019-topic], VAE is used to guide the generation process with topics of a document. A gaussian mixture model is used to incorporate topics into latent variables. In [@Xu2020unsupervisedVAE], VAE is used to control for sentiment attribute in style transfer task by constraining the posterior mean to a learned probability simplex.

Such a design of controllable text generation works when the control attributes can be represented as latent variables for example style, topics, strategies etc. This design is difficult to work for content grounded text generation tasks where specific information, keywords or entities have to guide the generation process.

Decompose {#bg:sec-decompose}
---------

You can decompose the encoder representation ${\mathbf{h}}_e$ into multiple subspaces, each of which signifies a different attribute you would like to control. @liu2018learning split the encoder representation 