<p>For loss layers, there is no next layer, and so the top diff blob is technically undefined and unused - but Caffe is using this preallocated space to store unrelated data:  Caffe supports multiplying loss layers with a user-defined weight (loss_weight in the prototxt), this information (a single scalar floating point number) is stored in the first element of the diff array of the top blob. That's why you'll see in every loss layer, that they multiply by that amount to support that functionality. This is explained in <a href="http://caffe.berkeleyvision.org/tutorial/loss.html">Caffe's tutorial about the loss layer</a>.</p>

<p>This weight is usually used to add auxiliary losses to the network. You can read more about it in Google's <a href="http://arxiv.org/abs/1409.4842">Going Deeper with Convoltions</a> or in <a href="http://arxiv.org/abs/1409.5185">Deeply-Supervised Nets</a>.</p>
Euclidean Loss Layer in Caffe
<p>I am currently trying to implement my own loss layer in caffe, and while attempting to do so, am using other layers as a reference. One thing that puzzles me, however, is the use of <code>top[0]-&gt;cpu_diff()</code> in <code>Backward_cpu</code>. I will be using the <code>EuclideanLossLayer</code> as a reference. Here are my questions</p>

<ul>
<li><p>It is my understanding that <code>top[0]-&gt;cpu_diff()</code> holds the error derivative from the next layer, but what if there is no other layer, how is it initialised? since it is used in <code>EuclideanLossLayer</code> without performing any checks:</p>

<pre><code>const Dtype alpha = sign * top[0]-&gt;cpu_diff()[0] / bottom[i]-&gt;num();
</code></pre></li>
<li><p>Again, in the <code>EuclideanLossLayer</code>, the derivative for the error with respect to the activations is calculated using the following code snippet:</p>

<pre><code>const Dtype alpha = sign * top[0]-&gt;cpu_diff()[0] / bottom[i]-&gt;num();
caffe_cpu_axpby(
  bottom[i]-&gt;count(),              // count
  alpha,                              // alpha
  diff_.cpu_data(),                   // a
  Dtype(0),                           // beta
  bottom[i]-&gt;mutable_cpu_diff());  // b
</code></pre>

<p>If my first assumption is correct, and <code>top[0]-&gt;cpu_diff()</code> does indeed hold the error derivative for the layer above, why do we only use the first element i.e. <code>top[0]-&gt;cpu_diff()[0]</code> as opposed to multiplying by the whole vector i.e. <code>top[0]-&gt;cpu_diff()</code>?</p></li>
</ul>