BN means the Batch Normalization layer, and CE loss means the softmax cross-entropy loss.

\
\[-5pt\]

AE
--

Auto-encoder(AE) was first proposed in 2006 as a method for dimensionality reduction. It can reduce the dimensionality of the input data while retaining most of the information in the data. AE consists of an encoder and a decoder, which tries to reconstruct the input from the output of the encoder, and the reconstruction error is used as a loss function. The encoder and decoder are trained to generate the low-dimension representation of the input and reconstruct the input from low-dimension representation, respectively. Subsequently, various derivatives of AE were proposed by researchers, such as variational auto-encoder (VAE) [@kingma2013auto], denoising auto-encoder (DAE) [@vincent2008extracting], and sparse auto-encoder (SAE) [@ranzato2007efficient]. In this paper, we design the deep AE and its derivatives for 1D input data and two dimension (2D) input data, respectively. Considering different features of neural networks, the structures and hyper-parameters of them shown in [Fig. \[AE\]]{} change adaptively. Specifically, the network structures of DAE and SAE are the same with AE, and the differences are the loss function and inputs. During the training of AE and its derivatives, the encoder and decoder are trained 