Quora post by Ian Goodfellow on why Autoencoders were a failure
https://www.quora.com/Why-are-autoencoders-considered-a-failure-What-are-their-alternatives/answer/Ian-Goodfellow?ch=2&oid=28474393&srid=6K26&target_type=answer
Why are autoencoders considered a failure? What are their alternatives?
2024-01-01
None
<div><div><div><div><p></p><div><p></p><a href="https://www.quora.com/Why-are-autoencoders-considered-a-failure-What-are-their-alternatives"><p></p></a></div></div><div><div><div><p></p></div><div><p></p></div></div><div><p></p></div><div><div><p></p></div><div><p></p></div></div><div><p></p></div><div><p></p></div></div><a href="https://www.quora.com/Why-are-autoencoders-considered-a-failure-What-are-their-alternatives"><div><p></p></div></a></div><div><div><div><p></p></div><div><p></p></div><div><p></p></div><div><p></p></div><div><p></p></div></div><div><p>More answers from </p><p></p></div></div></div></div>||||I|||| Skip to content
 Skip to search
 Sign In
 Why are autoencoders considered a failure? What are their alternatives?
 Ian Goodfellow
 Lead author of the Deep Learning textbook: http://www.deeplearningbook.org · Upvoted by
 Erdem Gökgezer
 , Ph.D Computer Vision & Machine Learning and
 Aaditya Prakash
 , Graduate student in Computer Vision and Deep Learning Author has 212 answers and 3.6M answer views · 7y ·
 Originally Answered: Why are Autoencoders considered a failure?

 Autoencoders are useful for some things, but turned out not to be nearly as necessary as we once thought. Around 10 years ago, we thought that deep nets would not learn correctly if trained with only backprop of the supervised cost. We thought that deep nets would also need an unsupervised cost, like the autoencoder cost, to regularize them. When Google Brain built their first very large neural network to recognize objects in images, it was an autoencoder (and it didn’t work very well at recognizing objects compared to later approaches). Today, we know we are able to recognize images just by using backprop on the supervised cost as long as there is enough labeled data. There are other tasks where we do still use autoencoders, but they’re not the fundamental solution to training deep nets that people once thought they were going to be.
 117.1K views ·
 View upvotes
 ·
 View 4 shares
 · Answer requested by
 Nafiz Hamid Rahi
 1 of 4 answers
 Comments
 Ian Goodfellow
 · 7y

 PS. just to be clear, I’m not endorsing the view that “autoencoders are a failure.” I’m explaining why autoencoders are not as prominent a part of the deep learning landscape as they were in 2006–2012. Autoencoders are successful at some things, just not as many as they were expected to be.
 Hendri Karisma
 · 4y

 is that same case for RBM in deep nets?
 Aashay Sachdeva
 · 6y

 But aren't autoencoders still used as feature extractor, for example in time series, and then passed through a feed neural network to make better predictions than hand crafted features?

 It makes sense intuitively as they capture a better representation of the underlying data.
 Wenfeng Pang
 · 7y

 Thank you very much. Because I haven’t seen it in paper for a long time, I curious that what kind of tasks still using the ad? Thanks again.
 Ian Goodfellow
 · 7y

 Generative modeling, where variational autoencoders are one of several currently popular approaches.
 Erdem Gökgezer
 · 3y

 It would have been nice to hear from Ian Goodfellow like what are those “some things” autoencoders are good for. Constructing high level feature encoders in an unsupervised manner maybe?
 Swarup Roy
 · 1y

 But not agree that there is a less use of autoencoder. Autoencoders are now extended to Graph data for representation learning and really successful in comparison to other graph embedding techniques. Future is promising.
 View more comments
 View 3 other answers to this question
 About the Author
 Ian Goodfellow
 AI Research Scientist
 Senior Staff Research Scientist at Google Brain 2017–present
 Studied at Stanford University
 Lives in Mountain View, CA
 3.6M content views 8.8K this month
 Top Writer 2018
 Published Writer HuffPost, Forbes, Apple News and 1 more
 More answers from Ian Goodfellow
 View more
 In what way are Adversarial Networks related or different to Adversarial Training?
 16,328 views
 How significant is the FractalNet paper?
 46,881 views
 What was your path toward learning ML? What books did you enjoy most while learning ML? What were the blind alleys?
 94,199 views
 What was your path toward learning DL? What books did you enjoy most while learning DL?
 8,094 views
 GANs are known for generating beautiful celeb faces and bedrooms. What keeps them from working as well on thousands of categories (ImageNet)?
 13,956 views
 About · Careers · Privacy · Terms · Contact · Languages · Your Ad Choices · Press ·
 © Quora, Inc. 2024
Please enable Javascript and refresh the page to continue
More answers from
Why are autoencoders considered a failure? What are their alternatives?
https://www.quora.com/Why-are-autoencoders-considered-a-failure-What-are-their-alternatives/answer/Ian-Goodfellow?ch=2&oid=28474393&srid=6K26&target_type=answer