title: Torch: the implementation of embedding layer
body: <p>I'm a beginner of torch and I'm reading the code of LSTM language models. I'm confused on the implementation of embedding layer (<a href="https://github.com/oxford-cs-ml-2015/practical6/blob/master/Embedding.lua" rel="nofollow">https://github.com/oxford-cs-ml-2015/practical6/blob/master/Embedding.lua</a>).</p>
  
  <p>This is the <code>updateGradInput</code> function in embedding layer:</p>
  
  <pre><code>function Embedding:updateGradInput(input, gradOutput)
    if self.gradInput then
      self.gradInput:resize(input:size())
      return self.gradInput
    end
  end
  </code></pre>
  
  <p>My first question is why there is a <code>if</code> statement in this function rather than just like below:</p>
  
  <pre><code>function Embedding:updateGradInput(input, gradOutput)
    self.gradInput:resize(input:size())
    return self.gradInput
  end
  </code></pre>
  
  <p>I think <code>self.gradInput</code> will always be true because <code>self.gradInput</code> was initialized in <code>__init()</code> of <code>nn.Module</code>.</p>
  
  <p>My second question is why <code>return self.gradInput</code> directly instead of calculates <code>self.gradInput</code> using <code>input</code> and <code>gradOutput</code>? I mean <code>self.gradInput = gradOutput * (deriv. of output wrt input)</code>.</p>
  
  <p>My last question is what the effect of this code in <a href="https://github.com/oxford-cs-ml-2015/practical6/blob/master/Embedding.lua" rel="nofollow">Embedding.lua</a>:</p>
  
  <pre><code>-- we do not need to accumulate parameters when sharing
  Embedding.sharedAccUpdateGradParameters = Embedding.accUpdateGradParameters
  </code></pre>
  
  <p>What does it mean of "we do not need to accumulate parameters when sharing"?</p>