<p>The per-layer terminology in <a href="https://arxiv.org/pdf/1504.08083.pdf" rel="nofollow noreferrer">that paper</a> is slightly ambiguous. They aren't referring to the layer-specific learning rates.</p>
<blockquote>
<p>All layers use a per-layer learning rate of 1 for weights and 2 for biases and a global learning rate of 0.001.</p>
</blockquote>
<p>The concerned statement is w.r.t. Caffe framework in which Fast R-CNN was originally written (<a href="https://github.com/rbgirshick/fast-rcnn" rel="nofollow noreferrer">github link</a>).</p>
<p>They meant that they're setting the learning rate multiplier of weights and biases to be 1 and 2 respectively.</p>
<p>Check any <code>prototxt</code> file in the repo e.g. <a href="https://github.com/rbgirshick/fast-rcnn/blob/master/models/CaffeNet/train.prototxt" rel="nofollow noreferrer">CaffeNet/train.prototxt</a>.</p>
<pre><code>  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
</code></pre>
<p>Thus, the effective learning rate is going to be <code>base_lr*lr_mult</code>, and here, the base learning rate is 0.001, which is defined in <code>solver.prototxt</code>.</p>
what is the meaning of 'per-layer learning rate' in Fast R-CNN paper?
<p>I'm reading a paper about Fast-RCNN model.</p>
<p>In the paper section 2.3 part of 'SGD hyper-parameters', it said that <strong>All layers use a per-layer learning rate of 1 for weights and 2 for biases and a global learning rate of 0.001</strong></p>
<br>
<p>Is 'per-layer learning rate' same as 'layer-specific learning rate' that give different learning rate by layers? If so, I can't understand how they('per-layer learning rate' and 'global learning rate') can be apply at the same time?</p>
<br>
<p>I found the example of 'layer-specific learning rate' in pytorch.</p>
<pre><code>optim.SGD([
                {'params': model.some_layers.parameters()},
                {'params': model.some_layers.parameters(), 'lr': 1}
            ], lr=1e-3, momentum=0.9)
</code></pre>
<p>According to paper, Is this the correct approach?</p>
<br>
<p>Sorry for may English</p>