title: RNN Regularization: Which Component to Regularize?
body: <p>I am building an RNN for classification (there is a softmax layer after the RNN). There are so many options for what to regularize and I am not sure if to just try all of them, would the effect be the same? which components do I regularize for what situation?</p>
  
  <p>The components being:</p>
  
  <ul>
  <li>Kernel weights (layer input)</li>
  <li>Recurrent weights</li>
  <li>Bias</li>
  <li>Activation function (layer output)</li>
  </ul>