original_docstring: Returns a list of layers that implements a Transformer decoder block.
  
    The input is an activation tensor.
  
    Args:
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      layer_idx: which layer are we at (for bookkeeping)
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A list of layers that maps an activation tensor to an activation tensor.
    
docstring: Returns a list of layers that implements a Transformer decoder block.
  The input is an activation tensor.
docstring_tokens: Returns a list of layers that implements a Transformer decoder block . The input is an activation tensor .
code: def _DecoderBlock(d_model, d_ff, n_heads,
                    dropout, layer_idx, mode, ff_activation):
    """Returns a list of layers that implements a Transformer decoder block.
  
    The input is an activation tensor.
  
    Args:
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      layer_idx: which layer are we at (for bookkeeping)
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A list of layers that maps an activation tensor to an activation tensor.
    """
    causal_attention = tl.CausalAttention(
        d_model, n_heads=n_heads, dropout=dropout, mode=mode),
  
    dropout_ = tl.Dropout(
        rate=dropout, name='attention_%d' % layer_idx, mode=mode)
  
    feed_forward = _FeedForwardBlock(
        d_model, d_ff, dropout, layer_idx, mode, ff_activation)
  
    return [
        tl.Residual(
            tl.LayerNorm(),
            causal_attention,
            dropout_,
        ),
        tl.Residual(
            feed_forward
        ),
    ]
code_tokens: def _DecoderBlock ( d_model , d_ff , n_heads , dropout , layer_idx , mode , ff_activation ) : causal_attention = tl . CausalAttention ( d_model , n_heads = n_heads , dropout = dropout , mode = mode ) , dropout_ = tl . Dropout ( rate = dropout , name = 'attention_%d' % layer_idx , mode = mode ) feed_forward = _FeedForwardBlock ( d_model , d_ff , dropout , layer_idx , mode , ff_activation ) return [ tl . Residual ( tl . LayerNorm ( ) , causal_attention , dropout_ , ) , tl . Residual ( feed_forward ) , ]
short_docstring: Returns a list of layers that implements a Transformer decoder block.
short_docstring_tokens: Returns a list of layers that implements a Transformer decoder block .
comment: """Returns a list of layers that implements a Transformer decoder block.
  
    The input is an activation tensor.
  
    Args:
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      layer_idx: which layer are we at (for bookkeeping)
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A list of layers that maps an activation tensor to an activation tensor.
    """