Graphcore Research, UK
Antoine Labatie, Dominic Masters, Zach Eaton-Rosen, Carlo Luschi
Batch-Independent Normalization;Batch Normalization;Layer Normalization;Group Normalization;Instance Normalization;Deep Learning Theory;CNNs;ResNets;ResNeXts;EfficientNets
Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence