Title: Continual Learning with Group-wise Neuron Normalization. Abstract: Continual learning focuses on methods that accommodate the change in distribution and allow model adaptation and evolution while receiving data continuously. Importance and regularization -based weight update methods that rely on heuristics might not be effective. Recently, enhanced experience replay-based methods showed promising results but might add to the computational cost. 
In this paper, we propose simple parameter-free normalization over groups of distinct neurons at the penultimate layer of the used neural network and a straightforward experience replay algorithm. We argue that such normalization enables the network to balance its capacity for each task, reducing the chances of damaging interference between tasks and mitigating forgetting. Our evaluation shows that normalization over groups of neurons drastically impacts performance. We demonstrate improved retained accuracy and backward transfer with respect to related state-of-the-art methods while computationally efficient.