original_docstring: Returns a list of layers implementing a Transformer encoder-decoder block.
  
    The input is a triple (decoder_input, mask, encoder) where the mask is
    created from the original source to prevent attending to the padding part
    of the encoder.
  
    Args:
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      layer_idx: which layer are we at (for bookkeeping)
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A list of layers which maps triples (decoder_activations, mask,
      encoder_activations) to triples of the same sort.
    
docstring: Returns a list of layers implementing a Transformer encoder-decoder block.
  The input is a triple (decoder_input, mask, encoder) where the mask is
  created from the original source to prevent attending to the padding part
  of the encoder.
docstring_tokens: Returns a list of layers implementing a Transformer encoder - decoder block . The input is a triple ( decoder_input mask encoder ) where the mask is created from the original source to prevent attending to the padding part of the encoder .
code: def _EncoderDecoderBlock(d_model, d_ff, n_heads, dropout, layer_idx, mode,
                           ff_activation):
    """Returns a list of layers implementing a Transformer encoder-decoder block.
  
    The input is a triple (decoder_input, mask, encoder) where the mask is
    created from the original source to prevent attending to the padding part
    of the encoder.
  
    Args:
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      layer_idx: which layer are we at (for bookkeeping)
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A list of layers which maps triples (decoder_activations, mask,
      encoder_activations) to triples of the same sort.
    """
    def _Dropout():
      return tl.Dropout(rate=dropout, mode=mode)
  
    attention_qkv = tl.AttentionQKV(
        d_model, n_heads=n_heads, dropout=dropout, mode=mode)
  
    causal_attention = tl.CausalAttention(
        d_model, n_heads=n_heads, mode=mode)
  
    feed_forward = _FeedForwardBlock(
        d_model, d_ff, dropout, layer_idx, mode, ff_activation)
  
    return [                             # vec_d masks vec_e
        tl.Residual(
            tl.LayerNorm(),              # vec_d ..... .....
            causal_attention,            # vec_d ..... .....
            _Dropout(),                  # vec_d ..... .....
        ),
        tl.Residual(
            tl.LayerNorm(),              # vec_d ..... .....
            tl.Select([0, 2, 2, 1, 2]),  # vec_d vec_e vec_e masks vec_e
            attention_qkv,               # vec_d masks vec_e
            _Dropout(),                  # vec_d masks vec_e
        ),
        tl.Residual(
            feed_forward                 # vec_d masks vec_e
        ),
    ]
code_tokens: def _EncoderDecoderBlock ( d_model , d_ff , n_heads , dropout , layer_idx , mode , ff_activation ) : def _Dropout ( ) : return tl . Dropout ( rate = dropout , mode = mode ) attention_qkv = tl . AttentionQKV ( d_model , n_heads = n_heads , dropout = dropout , mode = mode ) causal_attention = tl . CausalAttention ( d_model , n_heads = n_heads , mode = mode ) feed_forward = _FeedForwardBlock ( d_model , d_ff , dropout , layer_idx , mode , ff_activation ) return [ tl . Residual ( tl . LayerNorm ( ) , causal_attention , _Dropout ( ) , ) , tl . Residual ( tl . LayerNorm ( ) , tl . Select ( [ 0 , 2 , 2 , 1 , 2 ] ) , attention_qkv , _Dropout ( ) , ) , tl . Residual ( feed_forward ) , ]
short_docstring: Returns a list of layers implementing a Transformer encoder-decoder block.
short_docstring_tokens: Returns a list of layers implementing a Transformer encoder - decoder block .
comment: """Returns a list of layers implementing a Transformer encoder-decoder block.
  
    The input is a triple (decoder_input, mask, encoder) where the mask is
    created from the original source to prevent attending to the padding part
    of the encoder.
  
    Args:
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      layer_idx: which layer are we at (for bookkeeping)
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A list of layers which maps triples (decoder_activations, mask,
      encoder_activations) to triples of the same sort.
    """$SEP$# vec_d masks vec_e$SEP$# vec_d ..... .....$SEP$# vec_d ..... .....$SEP$# vec_d ..... .....$SEP$# vec_d ..... .....$SEP$# vec_d vec_e vec_e masks vec_e$SEP$# vec_d masks vec_e$SEP$# vec_d masks vec_e$SEP$# vec_d masks vec_e