<p>From the original <a href="https://arxiv.org/abs/1502.03167" rel="nofollow noreferrer">batch normalization paper</a> by Ioffe &amp; Szegedy: "we make sure that the transformation inserted in the network can represent the identity transform." Without the Scale layer after the BatchNorm layer, that would not be the case because the Caffe BatchNorm layer has no learnable parameters.</p>

<p>I learned this from the <a href="https://github.com/KaimingHe/deep-residual-networks" rel="nofollow noreferrer">Deep Residual Networks git repo</a>; see item 6 under disclaimers and known issues there.</p>
Do I have to use a Scale-Layer after every BatchNorm Layer?
<p>I am using caffe , in detail pycaffe, to create my neuronal network. I noticed that I have to use BatchNormLayer to get a positive result. I am using the Kappa-Score as a result matrix. 
I now have seen several different locations for the BatchNorm-Layers in my network. But I came across the ScaleLayer, too which is not in the Layer Catalogue but gets often mentioned with the BatchNorm Layer</p>

<p>Do you always need to put a ScaleLayer after a BatchNorm - Layer and what does it do?</p>