<p>In my answer I refer to the original paper <a href="https://arxiv.org/pdf/1706.03762.pdf" rel="nofollow noreferrer">Attention Is All You Need</a> by Vaswani et al..</p>
<ol>
<li>The input is transformed into the matrix. For this purpose, a Word embedding layer is used, which can be thought of as a lookup table.</li>
<li>The encoder creates a representation matrix in one shot.
This is then the input for the decoder. So the matrix <strong>V</strong> and <strong>K</strong> which is used in the decoder is the representation of the input.</li>
<li>The encoder already knows the total input in advance. To avoid cheating, the words still to be processed are masked so that the encoder cannot &quot;cheat&quot;.</li>
</ol>
Inputs of a transformer model
<p>I am trying to understand the transformer model. Please consider my below example and help me to understand the concept.</p>
<p>Example: English to french conversion</p>
<p>My questions:</p>
<ol>
<li><p>Is the input word embedding is an English- french pretrained embedding?</p>
</li>
<li><p>In which step of the decoder the prediction of a french word is happening?</p>
</li>
<li><p>Is output embedding in decoder is just decoder's output till predicted, if so why should I mask the next word since that is unknown to me as I still yet not passed as output</p>
</li>
</ol>
<p>Please clarify me this doubt</p>
<p>I also referred to these links:</p>
<ul>
<li><a href="https://datascience.stackexchange.com/questions/81727/what-would-be-the-target-input-for-transformer-decoder-during-test-phase">https://datascience.stackexchange.com/questions/81727/what-would-be-the-target-input-for-transformer-decoder-during-test-phase</a></li>
<li><a href="https://datascience.stackexchange.com/questions/51785/what-is-the-first-input-to-the-decoder-in-a-transformer-model">https://datascience.stackexchange.com/questions/51785/what-is-the-first-input-to-the-decoder-in-a-transformer-model</a></li>
</ul>