title: Why should ReLU only be used in hidden layers?
body: <p>On <a href="https://www.analyticsvidhya.com/blog/2017/10/fundamentals-deep-learning-activation-functions-when-to-use-them/" rel="nofollow noreferrer">this post</a> I read that ReLU should only be used in hidden layers. Why is this like that?</p>
  
  <p>I have a neural network with a regression task. It outputs a number between 0 and 10. I thought ReLU would be a good choice here since it does'nt return numbers smaller than 0. What would be the best activation function for the output layer here?</p>