Learning disentangled representation from any unlabelled data is a non-trivial problem. In this paper we propose Information Maximising Autoencoder (InfoAE) where the encoder learns powerful disentangled representation through maximizing the mutual information between the representation and given information in an unsupervised fashion. We have evaluated our model on MNIST dataset and achieved approximately 98.9 % test accuracy while using complete unsupervised training. Learning disentangled representation from any unlabelled data is an active area of research ). Self supervised learning BID3 ; BID15 ; BID11 ) is a way to learn representation from the unlabelled data but the supervised signal is needed to be developed manually, which usually varies depending on the problem and the dataset. Generative Adversarial Neural Networks (GANs) BID4 ) is a potential candidate for learning disentangled representation from unlabelled data BID12 ; BID7 ; BID2 ). In particular, InfoGAN BID1 ), which is a slight modification of the GAN, can learn interpretable and disentangled representation in an unsupervised fashion. The classifier from this model can be reused for any intermediate task such as feature extraction but the representation learned by the classifier of the model is fully dependent on the generation of the model which is a major shortcoming. Because if the generator of the InfoGAN fails to generate any data manifold, the classifier is unable to perform well on any sample from that manifold. Tricks from Mutual Information Neural Estimation paper BID0 ) might help to capture the training data distribution, yet learning all the training data manifold using GAN is a challenge for the research community ). Adversarial autoencoder (AAE) BID10 ) is another successful model for learning disentangled representation. The encoder of the AAE learns representation directly from the training data but it does not utilize the sample generation power of the decoder for learning the representations. In this paper, we aim to address this challenge. We aim to build a model that utilizes both training data and the generated samples and thereby learns more accurate disentangled representation maximizing the mutual information between the random condition/information and representation space. We have evaluated the model on MNIST dataset and received outstanding results. InfoAE is trained on MNIST training data without any labels. After trainning, We encoded the test data with Encoder, E and got classification label with the Classifier, C. Then we clustered the test data according to label and received classification accuracy of 98.9 (±.05), which is better than the popular methods as shown in TAB0 . In this paper we present and validate InfoAE, which learns the disentangled representation in a completely unsupervised fashion while utilizing both training and generated samples. We tested InfoAE on MNIST dataset and achieved test accuracy of 98.9 (±.1), which is a very competitive performance compared to the best reported results including InfoGAN. We observe that the encoder is able to disentangle the digit category and styles in the representation space, which results in the superior performance. InfoAE can be used to learn representation from unlabelled dataset and the learning can be utilized in a related problem where limited labeled data is available. Moreover, its power of representation learning can be exploited for data augmentation. This research is currently in progress. We are currently attempting to mathematically explain the results. We are also aiming to analyze the performance of InfoAE on large scale audio and image datasets.
Learn disentangle representation in an unsupervised manner. The authors present a framework in which an auto encoder (E, D) is regularized such that its latent representation to share mutual information with a generated latent space representation.