<p>Regarding your first question (minus the anomaly detection part), Keras creator François Chollet gives some good hints in his (highly recommended) blog post <a href="https://blog.keras.io/building-autoencoders-in-keras.html" rel="nofollow noreferrer">Building Autoencoders in Keras</a>:</p>
<blockquote>
<p><strong>What are autoencoders good for?</strong></p>
<p>They are rarely used in practical applications. In 2012 they briefly found an application in greedy layer-wise pretraining for deep convolutional neural networks, but this quickly fell out of fashion as we started realizing that better random weight initialization schemes were sufficient for training deep networks from scratch. In 2014, batch normalization started allowing for even deeper networks, and from late 2015 we could train arbitrarily deep networks from scratch using residual learning.</p>
<p>[...]</p>
<p><strong>So what's the big deal with autoencoders?</strong></p>
<p>Their main claim to fame comes from being featured in many introductory machine learning classes available online. As a result, a lot of newcomers to the field absolutely love autoencoders and can't get enough of them. This is the reason why this tutorial exists!</p>
</blockquote>
<p><strong>UPDATE</strong></p>
<p>That said, there indeed seem to be some cases where autoencoders are used in practice for anomaly detection; here are some recent papers:</p>
<p><a href="https://arxiv.org/abs/1802.00187" rel="nofollow noreferrer">Clustering and Unsupervised Anomaly Detection with L2 Normalized Deep Auto-Encoder Representations</a></p>
<p><a href="https://arxiv.org/abs/1802.03903" rel="nofollow noreferrer">Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications</a></p>
<p><a href="http://www.kdd.org/kdd2017/papers/view/anomaly-detection-with-robust-deep-auto-encoders" rel="nofollow noreferrer">Anomaly Detection with Robust Deep Auto-encoders</a></p>
<p>and blog posts:</p>
<p><a href="https://medium.com/@curiousily/credit-card-fraud-detection-using-autoencoders-in-keras-tensorflow-for-hackers-part-vii-20e0c85301bd" rel="nofollow noreferrer">Credit Card Fraud Detection using Autoencoders in Keras</a></p>
<p><a href="https://www.kaggle.com/imrandude/h2o-autoencoders-and-anomaly-detection-python" rel="nofollow noreferrer">H2O - Autoencoders and anomaly detection (Python)</a></p>
<p><a href="http://How%20Deep%20Learning%20Analytics%20Can%20Keep%20Your%20Data%20and%20Decisions%20in%20Line" rel="nofollow noreferrer">How Deep Learning Analytics Can Keep Your Data and Decisions in Line</a></p>
Questions about AutoEncoders
<p>I have three questions about autoEncoders and i would really appreciate your help :</p>
<p>1-  I have noticed that there is a lack of research papers done on  deep auto encoders (AE) although the concept is explained in  plenty of tutorials and examples  and most of the tutorials claim that this model is powerful , is there a reason for the lack of research paper published using AE especially in Anomaly or novelty detection ?</p>
<p>2- in all the tutorials i have seen a threshold is manually  set ( hard set ) for AutoEncoder to be as a decision boundary for Anomaly detection  by testing several values and selecting the best one , is there another technique to select the Threshold value , in other words what are the different thresholding mechanisms that can automatically detect the threshold</p>