 The attention layer produces embeddings for every token in context that contain information not only about the token itself, but also a weighted combination of other relevant tokens weighted by the attention weights.Concretely, for each attention unit the Transformer model learns three weight matrices