better at preserving the meaning of the generated story, the arithmetic operation provides a better signal of the personality for the generation process.

@hoang2016incorporating uses both the concatenation technique as well as performs a linear transform of ${\mathbf{s}}$ to obtain ${\mathbf{h}}_0$ for language modelling task. The control vectors in this case represents meta data such as key-words, topics etc. In case of the linear transform ${\mathbf{h}}_0 = {\mathtt{tanh}}({\mathbf{W}}_1 {\mathbf{h}}_e + {\mathbf{W}}_2 {\mathbf{s}} + {\mathbf{b}})$. The paper also explores adding the control vector to the encoder representation (${\mathbf{h}}_0 = {\mathbf{h}}_e + {\mathbf{s}}$).

In case of addition, the resulting ${\mathbf{h}}_0$ would be averaged representation of the input representation ${\mathbf{h}}_e$ and ${\mathbf{s}}$. Information could be lost in this case as control is not explicit. In case of concatenation, if the size of the control vector ${\mathbf{s}}$ is too small compared to the context vector ${\mathbf{h}}_e$, then ${\mathbf{s}}$ is over-shadowed by ${\mathbf{h}}_e$ and the generator may not be able to pay attention to ${\mathbf{s}}$. Hence it is important to choose comparable dimensions for these two vectors. But this increases the size of model considerably and could be quite costly. Linear transform avoids these issues and performs better than the other two techniques for @hoang2016incorporating.

