<p>According to <a href="https://arxiv.org/pdf/2109.08668.pdf" rel="nofollow noreferrer">this</a> paper, the dff is the <strong>feed forward upwards projection size</strong>.</p>
What is the dff parameter in transformer model?
<p>In the paper, it describes the base model's network configuration are below:</p>
<p><a href="https://i.stack.imgur.com/0CfRQ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/0CfRQ.png" alt="enter image description here" /></a></p>
<pre><code>d_model:  embedding size
h: attention head count
d_k: key matrix dimension
d_v: value matrix dimension
dff: 2048?
</code></pre>
<p>What's the dff?</p>