Recently, a number of works have studied clustering strategies that combine classical clustering algorithms and deep learning methods. These approaches follow either a sequential way, where a deep representation is learned using a deep autoencoder before obtaining clusters with k-means, or a simultaneous way, where deep representation and clusters are learned jointly by optimizing a single objective function. Both strategies improve clustering performance, however the robustness of these approaches is impeded by several deep autoencoder setting issues, among which the weights initialization, the width and number of layers or the number of epochs. To alleviate the impact of such hyperparameters setting on the clustering performance, we propose a new model which combines the spectral clustering and deep autoencoder strengths in an ensemble learning framework. Extensive experiments on various benchmark datasets demonstrate the potential and robustness of our approach compared to state-of-the art deep clustering methods.
The deep autoencoders (DAE) have proven to be useful for dimensionality reduction @cite_16 and image denoising. In particular, the autoencoders (AE) can non-linearly transform data into a latent space. When this latent space has lower dimension than the original one @cite_16 , this can be viewed as a form of non-linear PCA. An autoencoder typically consists of an encoder stage, that can provide an encoding of the original data in lower dimension, and a decoder part, to define the data reconstruction cost. In clustering context, the general idea is to embed the data into a low dimensional latent space and then perform clustering in this new space. The goal of the embedding here is to learn new representations of the objects of interest (e.g., images) that encode only the most relevant information characterizing the original data, which would for example reduce noise and sparsity.
@cite_16
2100495367
High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such “autoencoder” networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.