title: How to implement time-distributed dense (TDD) layer in PyTorch
body: <p>In some deep learning models which analyse temporal data (e.g. audio, or video), we use a "time-distributed dense" (TDD) layer. What this creates is a fully-connected (dense) layer which is applied separately to every time-step.</p>
  
  <p><img src="https://i.stack.imgur.com/Rj4AM.png" width="300" /></p>
  
  <p>In Keras this can be done using the <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/TimeDistributed" rel="nofollow noreferrer">TimeDistributed wrapper</a>, which is actually slightly more general. In PyTorch it's been <a href="https://github.com/pytorch/pytorch/issues/1927" rel="nofollow noreferrer">an open feature request</a> for a couple of years.</p>
  
  <p>How can we implement <strong>time-distributed dense</strong> manually in PyTorch?</p>