<SYSTEM_TASK:>
Solve the following problem using Python, implementing the functions described below, one line at a time
<END_TASK>
<USER_TASK:>
Description:
 def attention_lm_moe_large():
 """Large model for distributed training. Over 1B parameters, so requires multi-gpu training due to memory requirements. on lm1b_32k: After 45K steps on 8 GPUs (synchronous):
 eval_log_ppl_per_token = 3.18 eval_ppl_per_word = exp(1.107893 * eval_log_ppl_per_token) = 33.9 Returns: an hparams object. """

  hparams = attention_lm_moe_base()
  hparams.num_hidden_layers = 5
  hparams.moe_layers = "3"
  hparams.hidden_size = 1024
  hparams.num_heads = 16
  hparams.filter_size = 4096
  hparams.moe_hidden_sizes = "4096"
  hparams.moe_num_experts = 128
  hparams.layer_prepostprocess_dropout = 0.2
  return hparams