 Both the encoders and decoders have a final feed-forward neural network for additional processing of the outputs, and also contain residual connections and layer normalization steps.The basic building blocks of the Transformer are scaled dot-product attention units.When a sentence is passed into a Transformer model, attention weights are calculated between every token simultaneously.