\odot {\mathbf{Y}}{\mathbf{g}} + \\
    {\mathbf{q}}_t \odot ({\mathbf{1}}^{T}_{L} {\mathbf{Z}} {\mathbf{E}}^{new}_{t})^{T})\end{aligned}$$ where ${\mathbf{s}}_t$ is a goal select gate and ${\mathbf{q}}_t$ is a item select gate. With this modification, the generation process is controlled for the items to be generation in the recipe and the goal.

@sem_cond_lstm adapt the LSTM to control the dialogue act information in the generation process. The operation to compute the cell value ${\mathbf{c}}_t$ is given by: $${\mathbf{c}}_t = {\mathbf{f}}_t \odot {\mathbf{c}}_{t-1} + {\mathbf{i}}_t \odot {\mathbf{\tilde{c}}}_t + {\mathtt{tanh}}({\mathbf{W}}_d {\mathbf{d}}_t)$$ The dialogue act representation ${\mathbf{d}}_t$ is build using another LSTM cell.

RNNs, LSTMs and GRUs are commonly used to model controllable text generation tasks [@prabhumoye-etal-2019-towards; @rao2018dear; @see2017get; @zhou2018dataset; @fu:2017]. Most of these variants still have trouble remembering long sequences and are hence commonly used with attention mechanism ([§\[sec:bg-out-att\]]{}) on the source sequence.

Transformer
-----------

Transformers are proposed by [@vaswani2017attention] and they rely on attention mechanism to draw global dependencies between input and output. The Transformer uses stacked self-attention and point-wise, fully connected layers for both the encoder and decoder. The encoder stacks $N$ identical layers, each of which has two sub-layers. The first sub-layer is a multi-head self-attention mechanism ([§\[sec:bg-out-att\]]{}), and the second sub-layer is a positionwise fully connected feed-forward 