While self-organizing principles have motivated much of early learning models, such principles have rarely been included in deep learning architectures. Indeed, from a supervised learning perspective it seems that topographic constraints are rather decremental to optimal performance. Here we study a network model that incorporates self-organizing maps into a supervised network and show how gradient learning results in a form of a self-organizing learning rule. Moreover, we show that such a model is robust in the sense of its application to a variety of  areas, which is believed to be a hallmark of biological learning systems.
integration of self-organization and supervised learning in a hierarchical neural network The paper discusses learning in a neural network with three layers, where the middle layer is topographically organized and investigates interplay between unsupervised and hierarchical supervised learning in biological context. A supervised variant of Kohonen's self-organizing map (SOM), but where the linear output layer is replaced with squared error by a softmax layer with cross-entropy. Proposes a model using hidden neurons with self-organising activation function, whose outputs feed to classifier with softmax output function. 