N-Step RNN (recurrent neural networks).

    N-Step RNN function implements Elman RNN with nonlineraity to input sequence.
    N-Step RNN function is defined as following:

    .. math::
        h_t = \\tanh(w_{ih}x_t+b_{ih}+w_{hh}h_{(t-1)}).

    We use the following notations to describe the inputs and outputs below.
    :math:`T`: sequcne length, :math:`B`: batch size, :math:`I`: input size, :math:`L`: number of layers, :math:`D`: number of directions, can be either 1 or 2, :math:`H`: hidden size.

    References:

        Jeffrey L. Elman. "Finding Structure in Time."
        Cognitive Science. 1990.

    Args:
        x (~nnabla.Variable): Input N-D array with shape :math:`(T, B, I)`.
        h (~nnabla.Variable): Input N-D array with shape :math:`(L, D, B, H)`.
        w0_init (:obj:`nnabla.initializer.BaseInitializer` or :obj:`numpy.ndarray`, optional): Initializer for weight at the first layer. Shape is :math:`(D, H, I + H)`.
        w_init (:obj:`nnabla.initializer.BaseInitializer` or :obj:`numpy.ndarray`, optional): Initializer for weights at the second layer and up. Shape is :math:`(L-1, D, H, D*H + H)`.
        b_init (:obj:`nnabla.initializer.BaseInitializer` or :obj:`numpy.ndarray`, optional): Initializer for bias. Shape is :math:`(L, D, H)`.
        num_layers (int, optional): Number of layers in the network. If set to 1, only the weights for the first layer will be invoked. Default is 1.
        nonlinearity (str, optional): Type of nonlinearity applied to input sequcne. Must be either tanh or relu. Default is tanh.
        dropout (float, optional): Dropout ratio applied to parameters. Default is 0.0.
        bidirectional (bool, optional): If True, bidirectional computation will be performed in each layer. Default is False.
        training (bool, optional): Backpropagation will be performed only when it is true. Default is True.
        with_bias (bool, optional): Specify whether to include the bias term.

    Returns:
        ~nnabla.Variable: Output :math:`y` with shape :math:`(T, B, D * H)`
        ~nnabla.Variable: Output :math:`h_n` with shape :math:`(L, D, B, H)`

    Example:
        .. code-block:: python

            x = nn.Variable((seq_len, batch_size, input_size))
            h = nn.Variable((num_layers, num_directions, batch_size, hidden_size))
            y, hn = PF.rnn(x, h)

def rnn(x, h, w0_init=None, w_init=None, b_init=None, num_layers=1, nonlinearity='tanh', dropout=0.0, bidirectional=False, training=True, rng=None, with_bias=True, fix_parameters=False):
    """N-Step RNN (recurrent neural networks).

    N-Step RNN function implements Elman RNN with nonlineraity to input sequence.
    N-Step RNN function is defined as following:

    .. math::
        h_t = \\tanh(w_{ih}x_t+b_{ih}+w_{hh}h_{(t-1)}).

    We use the following notations to describe the inputs and outputs below.
    :math:`T`: sequcne length, :math:`B`: batch size, :math:`I`: input size, :math:`L`: number of layers, :math:`D`: number of directions, can be either 1 or 2, :math:`H`: hidden size.

    References:

        Jeffrey L. Elman. "Finding Structure in Time."
        Cognitive Science. 1990.

    Args:
        x (~nnabla.Variable): Input N-D array with shape :math:`(T, B, I)`.
        h (~nnabla.Variable): Input N-D array with shape :math:`(L, D, B, H)`.
        w0_init (:obj:`nnabla.initializer.BaseInitializer` or :obj:`numpy.ndarray`, optional): Initializer for weight at the first layer. Shape is :math:`(D, H, I + H)`.
        w_init (:obj:`nnabla.initializer.BaseInitializer` or :obj:`numpy.ndarray`, optional): Initializer for weights at the second layer and up. Shape is :math:`(L-1, D, H, D*H + H)`.
        b_init (:obj:`nnabla.initializer.BaseInitializer` or :obj:`numpy.ndarray`, optional): Initializer for bias. Shape is :math:`(L, D, H)`.
        num_layers (int, optional): Number of layers in the network. If set to 1, only the weights for the first layer will be invoked. Default is 1.
        nonlinearity (str, optional): Type of nonlinearity applied to input sequcne. Must be either tanh or relu. Default is tanh.
        dropout (float, optional): Dropout ratio applied to parameters. Default is 0.0.
        bidirectional (bool, optional): If True, bidirectional computation will be performed in each layer. Default is False.
        training (bool, optional): Backpropagation will be performed only when it is true. Default is True.
        with_bias (bool, optional): Specify whether to include the bias term.

    Returns:
        ~nnabla.Variable: Output :math:`y` with shape :math:`(T, B, D * H)`
        ~nnabla.Variable: Output :math:`h_n` with shape :math:`(L, D, B, H)`

    Example:
        .. code-block:: python

            x = nn.Variable((seq_len, batch_size, input_size))
            h = nn.Variable((num_layers, num_directions, batch_size, hidden_size))
            y, hn = PF.rnn(x, h)

    """
    input_size = x.shape[2]
    hidden_size = h.shape[3]
    num_layers = h.shape[0]
    num_directions = 2 if bidirectional else 1

    if w0_init is None:
        w0_init_ih = UniformInitializer(
            calc_uniform_lim_glorot(input_size, hidden_size), rng)
        w0_init_ih = w0_init_ih((num_directions, hidden_size, input_size))
        w0_init_hh = UniformInitializer(
            calc_uniform_lim_glorot(hidden_size, hidden_size), rng)
        w0_init_hh = w0_init_hh((num_directions, hidden_size, hidden_size))
        w0_init = np.concatenate((w0_init_ih, w0_init_hh), axis=2)
    if w_init is None:
        w_init_ih = UniformInitializer(calc_uniform_lim_glorot(
            num_directions*hidden_size, hidden_size), rng)
        w_init_ih = w_init_ih(
            (num_layers - 1, num_directions, hidden_size, num_directions*hidden_size))
        w_init_hh = UniformInitializer(
            calc_uniform_lim_glorot(hidden_size, hidden_size), rng)
        w_init_hh = w_init_hh(
            (num_layers - 1, num_directions, hidden_size, hidden_size))
        w_init = np.concatenate((w_init_ih, w_init_hh), axis=3)
    if with_bias and b_init is None:
        b_init = ConstantInitializer()
    w0 = get_parameter_or_create(
        "weight_l0", w0_init.shape,
        w0_init, True, not fix_parameters)
    w = None
    if num_layers > 1:
        w = get_parameter_or_create(
            "weight", w_init.shape,
            w_init, True, not fix_parameters)
    b = None
    n_outmaps = (num_layers, num_directions, hidden_size)
    if with_bias:
        b = get_parameter_or_create(
            "bias", n_outmaps, b_init, True, not fix_parameters)

    return F.rnn(x, h, weight_l0=w0, weight=w, bias=b, num_layers=num_layers, nonlinearity=nonlinearity, dropout=dropout, bidirectional=bidirectional, training=training)