 Theoretically, this vector can encode information about the whole English sentence, giving the model all necessary knowledge, but in practice this information is often not well preserved.If we introduce an attention mechanism, the model can instead learn to attend to the latent states of early English tokens when producing the beginning of the French output, giving it a much more precise image of what it is translating.When added to RNNs, attention mechanisms led to large gains in performance.