original_docstring: Returns a Transformer decoder model.
  
    The input to the model is either continuous or discrete - controlled by
    vocab_size. Does not shift the input to the right, i.e. the output for
    timestep t is based on inputs up to timestep t inclusively.
  
    Args:
      vocab_size: int or None: vocab size if running on discrete input, None
        otherwise.
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_layers: int: number of encoder/decoder layers
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      max_len: int: maximum symbol length for positional encoding
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A Transformer decoder as a layer that maps from a continuous or discrete
      tensor to a continuous tensor.
    
docstring: Returns a Transformer decoder model.
  The input to the model is either continuous or discrete - controlled by
  vocab_size. Does not shift the input to the right, i.e. the output for
  timestep t is based on inputs up to timestep t inclusively.
docstring_tokens: Returns a Transformer decoder model . The input to the model is either continuous or discrete - controlled by vocab_size . Does not shift the input to the right i . e . the output for timestep t is based on inputs up to timestep t inclusively .
code: def TransformerDecoder(vocab_size=None,
                         d_model=512,
                         d_ff=2048,
                         n_layers=6,
                         n_heads=8,
                         dropout=0.1,
                         max_len=2048,
                         mode='train',
                         ff_activation=tl.Relu):
    """Returns a Transformer decoder model.
  
    The input to the model is either continuous or discrete - controlled by
    vocab_size. Does not shift the input to the right, i.e. the output for
    timestep t is based on inputs up to timestep t inclusively.
  
    Args:
      vocab_size: int or None: vocab size if running on discrete input, None
        otherwise.
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_layers: int: number of encoder/decoder layers
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      max_len: int: maximum symbol length for positional encoding
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A Transformer decoder as a layer that maps from a continuous or discrete
      tensor to a continuous tensor.
    """
    positional_encoder = [
        (tl.Embedding(d_model, vocab_size) if vocab_size is not None
         else tl.Dense(d_model)),
        tl.Dropout(rate=dropout, mode=mode),
        tl.PositionalEncoding(max_len=max_len)]
  
    decoder_blocks = [
        # pylint: disable=g-complex-comprehension
        _DecoderBlock(d_model, d_ff, n_heads,
                      dropout, i, mode, ff_activation)
        for i in range(n_layers)]
  
    # Assemble and return the model.
    return tl.Serial(        # toks
        positional_encoder,  # vecs
        decoder_blocks,      # vecs
        tl.LayerNorm(),      # vecs
    )
code_tokens: def TransformerDecoder ( vocab_size = None , d_model = 512 , d_ff = 2048 , n_layers = 6 , n_heads = 8 , dropout = 0.1 , max_len = 2048 , mode = 'train' , ff_activation = tl . Relu ) : positional_encoder = [ ( tl . Embedding ( d_model , vocab_size ) if vocab_size is not None else tl . Dense ( d_model ) ) , tl . Dropout ( rate = dropout , mode = mode ) , tl . PositionalEncoding ( max_len = max_len ) ] decoder_blocks = [ _DecoderBlock ( d_model , d_ff , n_heads , dropout , i , mode , ff_activation ) for i in range ( n_layers ) ] return tl . Serial ( positional_encoder , decoder_blocks , tl . LayerNorm ( ) , )
short_docstring: Returns a Transformer decoder model.
short_docstring_tokens: Returns a Transformer decoder model .
comment: """Returns a Transformer decoder model.
  
    The input to the model is either continuous or discrete - controlled by
    vocab_size. Does not shift the input to the right, i.e. the output for
    timestep t is based on inputs up to timestep t inclusively.
  
    Args:
      vocab_size: int or None: vocab size if running on discrete input, None
        otherwise.
      d_model: int:  depth of embedding
      d_ff: int: depth of feed-forward layer
      n_layers: int: number of encoder/decoder layers
      n_heads: int: number of attention heads
      dropout: float: dropout rate (how much to drop out)
      max_len: int: maximum symbol length for positional encoding
      mode: str: 'train' or 'eval'
      ff_activation: the non-linearity in feed-forward layer
  
    Returns:
      A Transformer decoder as a layer that maps from a continuous or discrete
      tensor to a continuous tensor.
    """$SEP$# pylint: disable=g-complex-comprehension$SEP$# Assemble and return the model.$SEP$# toks$SEP$# vecs$SEP$# vecs$SEP$# vecs