token ${\mathbf{\hat{x}}}_t$ is predicted from the vocabulary by passing ${\mathbf{\hat{o}}}_t$ through a softmax function and taking the max value. The predicted token ${\mathbf{\hat{x}}}_t$ is compared with the reference token ${\mathbf{y}}_t$ using a loss function. This loss function can be tweaked to ensure that the generated text carries the desired control attributes.

General Loss Objective
----------------------

Here, we describe the loss objectives commonly used in natural language generation tasks. These loss objectives do not try to control for any attribute. Instead they try to ensure fluent, grammatical and diverse generations.

#### Cross Entropy Loss:

This is the basic loss used to compare the generated tokens with the reference tokens and is used in all text generation process. At each time step $t$, the generation has to predict a token from the vocabulary. Hence, it could be seen as a classification problem with number of classes being equal to vocabulary size. The categorical cross entropy loss is given by: $$- \Sigma^{M}_{c=1} {\mathbf{y}}_{t, c} {\mathtt{log}} (p_{t, c})$$ where $p_{t, c}$ is the probability of the token $c$ at time step $t$. Note that $p_t = {\mathtt{softmax}} ({\mathbf{\tilde{o}}}_t)$ is the probability distribution over the vocabulary.

#### Unlikelihood loss: 

This maintains a set of negative 