title: Activation function after pooling layer or convolutional layer?
body: <p>The theory from these links show that the order of Convolutional Network is: <code>Convolutional Layer - Non-linear Activation - Pooling Layer</code>.</p>
  
  <ol>
  <li><a href="http://neuralnetworksanddeeplearning.com/chap6.html" rel="noreferrer">Neural networks and deep learning (equation (125)</a></li>
  <li><a href="http://www.deeplearningbook.org/contents/convnets.html" rel="noreferrer">Deep learning book (page 304, 1st paragraph)</a></li>
  <li><a href="http://deeplearning.net/tutorial/lenet.html#details-and-notation" rel="noreferrer">Lenet (the equation)</a></li>
  <li><a href="http://deeplearning.net/tutorial/lenet.html#the-convolution-operator" rel="noreferrer">The source in this headline</a></li>
  </ol>
  
  <p>But, in the last implementation from those sites, it said that the order is: <code>Convolutional Layer - Pooling Layer - Non-linear Activation</code></p>
  
  <ol>
  <li><a href="http://neuralnetworksanddeeplearning.com/chap6.html#the_code_for_our_convolutional_networks" rel="noreferrer">network3.py</a></li>
  <li><a href="http://deeplearning.net/tutorial/lenet.html#putting-it-all-together" rel="noreferrer">The sourcecode, LeNetConvPoolLayer class</a></li>
  </ol>
  
  <p>I've tried too to explore a Conv2D operation syntax, but there is no activation function, it's only convolution with flipped kernel. Can someone help me to explain why is this happen?</p>