title: Why does TensorFlow use `None` as the default activation?
body: <p>In the TensorFlow Python API, the default value for the <code>activation</code> kwarg of <code>tf.layers.dense</code> is <code>None</code>, then in the documentation it says:</p>
  
  <blockquote>
    <p>activation: Activation function to use. If you don't specify anything, no activation is applied (ie. "linear" activation: a(x) = x).</p>
  </blockquote>
  
  <p>Why not just use the identity function as default value when defining the function? like this:</p>
  
  <pre><code>def dense(..., activation=lambda x: x, ...):
      pass
  </code></pre>
  
  <p>This way you don't have to worry about the inconsistency between the documentation and the code.</p>
  
  <p>Is this (using None to represent a default function) just a coding style, or is there some caveat for using function as the default value of a kw argument?</p>
  
  <p>It's not there to avoid unnecessary function calls, since an identity function is still created and called even <code>None</code> is passed to <code>activation</code>. Besides, since this happens at graph construction time, there is no point to do optimization like this - assuming this indeed is an optimization.</p>
  
  <p>Correction:</p>
  
  <p>As pointed out by @y-luo, the <code>tf</code> implementation doesn't actually create an identity function. But the <code>tf.keras</code> implementation does.</p>