Recent advances in deep learning techniques has shown the usefulness of the deep neural networks in extracting features required to perform the task at hand.
 However, these features learnt are in particular helpful only for the initial task. This is due to the fact that the features learnt are very task specific and does not capture the most general and task agnostic features of the input.
 In fact the way humans are seen to learn is by disentangling features which task agnostic. This indicates that leaning task agnostic features by disentangling only the most informative features from the input data.
 Recently Variational Auto-Encoders (VAEs) have shown to be the de-facto models to capture the latent variables in a generative sense.
 As these latent features can be represented as continuous and/or discrete variables, this indicates us to use VAE with a mixture of continuous and discrete variables for the latent space.
 We achieve this by performing our experiments using a modified version of joint-vae to learn the disentangled features.
 Feature learning is one of the most fundamental task in machine learning and recently deep learning has made revolutionary advanced in this. What ever the machine learning task at hand, deep neural networks are excellent models for feature extraction from a raw data. But the features extracted or learned are very task specific as one use particular loss functions that are suited for task at hand. For example, cross entropy loss used for multiclass classification problems.This way of learning performs well only for the particular trained task leading to what is called as a narrow or weak artificial intelligence. However, to achieve the ultimate goal of true or general artificial intelligence, one needs to learn representations in a task agnostic manner. These task agnostic features should be enough to capture all the required information of the given entity.One such effort made in recent times is towards learning disentangled representations. As BID0 defines, disentangled representations are the representations where a change in a single unit of the representation corresponds to a change in a single factor of the BID2 .In this work, we experiment with JointVAE BID2 to explore the disentangled representation for the given dataset Gondal (2019). In the next sections, we discuss our experimental setup and results.
Mixture Model for Neural Disentanglement