title: Explanation of dimension of input in a transformer model
body: <p><a href="https://i.stack.imgur.com/phGpE.png" rel="nofollow noreferrer">Model Architechture</a></p>
  <p>Hey,
  I was trying to code the Transformer architecture from scratch as a part of my project. I wanted to ask that, what does the batch-size refer to in the dimension of the of the input matrix of the encoder.</p>
  <p>I have us the attached model architecture as my reference. Let me consider the example of an essay document. The <code>seq-len</code> refers to the maximum length of the sentence inside the document and the <code>embedding-size</code> refers to the size of the word embedding of each word of the sentence. In that case does <code>batch-size</code> refer to the number of sentences in the document ?</p>