<SYSTEM_TASK:>
Solve the following problem using Python, implementing the functions described below, one line at a time
<END_TASK>
<USER_TASK:>
Description:
 def attention_lm_moe_small():
 """Cheap model for single-gpu training. on lm1b_32k: ~312M params 1.6 steps/sec on [GeForce GTX TITAN X] After 50K steps on 8 GPUs (synchronous):
 eval_log_ppl_per_token = 3.31 Returns: an hparams object. """

  hparams = attention_lm_moe_base()
  hparams.num_hidden_layers = 4
  hparams.hidden_size = 512
  hparams.filter_size = 2048
  hparams.moe_num_experts = 128
  hparams.moe_layers = "2"
  return hparams