title: How does Tensroflow initialize kernel weights in LSTM / GRU cells?
body: <p>Reading Tensorflow rnn cells <a href="https://github.com/tensorflow/tensorflow/blob/r1.7/tensorflow/python/ops/rnn_cell_impl.py" rel="nofollow noreferrer">source code</a> I've came across the following <code>GRUCell</code> code:</p>
  
  <pre><code>self._gate_kernel = self.add_variable(
      "gates/%s" % _WEIGHTS_VARIABLE_NAME,
      shape=[input_depth + self._num_units, 2 * self._num_units],
      initializer=self._kernel_initializer)
  self._gate_bias = self.add_variable(
      "gates/%s" % _BIAS_VARIABLE_NAME,
      shape=[2 * self._num_units],
      initializer=(
          self._bias_initializer
          if self._bias_initializer is not None
          else init_ops.constant_initializer(1.0, dtype=self.dtype)))
  self._candidate_kernel = self.add_variable(
      "candidate/%s" % _WEIGHTS_VARIABLE_NAME,
      shape=[input_depth + self._num_units, self._num_units],
      initializer=self._kernel_initializer)
  self._candidate_bias = self.add_variable(
      "candidate/%s" % _BIAS_VARIABLE_NAME,
      shape=[self._num_units],
      initializer=(
          self._bias_initializer
          if self._bias_initializer is not None
          else init_ops.zeros_initializer(dtype=self.dtype)))
  </code></pre>
  
  <p>You can see that gate bias tensor is initialized with <code>1</code> and candidate bias with <code>0</code> while gate kernel and candidate kernel tensors do not have default initializers (<code>self._kernel_initializer=None</code>) if we don't explicitly provide them. So does Tensorflow populate these tensors with random numbers?</p>