by providing different types of rewards like style reward, semantic reward and fluency reward in the reinforcement learning setup. The discriminator used to obtain the adversarial loss has to be jointly trained with the generator.

Arithmetic or Linear Transform {#arithmetic-or-linear-transform-1}
------------------------------

@hoang2016incorporating demonstrate three simple ways of changing the output ${\mathbf{o}}_t$ of an RNN to control for meta information like topic, keywords etc. They show that you can add the control vector ${\mathbf{s}}$ to ${\mathbf{o}}_t$. Hence the modified output ${\mathbf{\tilde{o}}}_t$ is ${\mathbf{\tilde{o}}}_t = {\mathbf{o}}_t + {\mathbf{s}}$. Similarly, you can create ${\mathbf{\tilde{o}}}_t$ by concatenating ${\mathbf{s}}$ to ${\mathbf{o}}_t$ (${\mathbf{\tilde{o}}}_t = [{\mathbf{o}}_t; {\mathbf{s}}]$). We can also build ${\mathbf{\tilde{o}}}_t$ using a perceptron layer dependent on ${\mathbf{s}}$ and ${\mathbf{o}}_t$. In this case, ${\mathbf{\tilde{o}}}_t$ is given by ${\mathbf{\tilde{o}}}_t = {\mathtt{tanh}}({\mathbf{W}}_o {\mathbf{o}}_t + {\mathbf{W}}_s {\mathbf{s}} + {\mathbf{b}}_o)$. In each of the three cases, the modified output ${\mathbf{\tilde{o}}}_t$ is then projected to the vocabulary space to predict the token ${\mathbf{\hat{x}}}_t$.

Training Objective {#sec:bg-train-obj}
==================

In this section we describe various methods used to control the generation using objective functions. The output ${\mathbf{o}}_t$ at each time step $t$ of the generation process is projected to the vocabulary space using a linear transform (${\mathbf{\hat{o}}}_t = {\mathbf{W}}_o {\mathbf{o}}_t + {\mathbf{b}}$). A 