of the size of the one reported in on the same corpus. [<span style="font-variant:small-caps;">ELMo*-A*$_{\small \textsc{}}$</span>]{} achieves a perplexity of 35.8, 3.9 points lower than [<span style="font-variant:small-caps;">ELMo</span>]{}’s 39.7. This shows that it is an efficient yet strong baseline.

We also list the performance of the following models for reference. [<span style="font-variant:small-caps;">ELMo$_{\small \textsc{Org}}$</span>]{} and [<span style="font-variant:small-caps;">Base</span>]{} are results listed in but they are tested using different configurations. [<span style="font-variant:small-caps;">FastText$_{\small \textsc{cc}}$</span>]{} is the non-contextual word embedding trained on the Common Crawl corpus with 600 billion tokens, which serves as a baseline non-contextual model.

  Model                                                                             Time      SpeedUp   Batch   Params
  --------------------------------------------------------------------------------- --------- --------- ------- --------
  [<span style="font-variant:small-caps;">ELMo</span>]{}                            14 x 3    1x        128     499m
  [<span style="font-variant:small-caps;">ELMo*-A*$_{\small \textsc{}}$</span>]{}   5.7 x 4   1.8x      256     196m
  [<span style="font-variant:small-caps;">ELMo</span>*-S*$_{\small \textsc{}}$]{}   2.5 x 4   4.2x      768     76m

  : Model Efficiency. Time is in Days x Cards format. Batch is the maximal batch size per card. Params is the total trainable parameters in millions.[]{data-label="table:elmo_model_time"}

[l@l@l@l@l@l@l@l@l@l@l@l]{} & & [<span style="font-variant:small-caps;">ELMo$_{\small \textsc{Org}}$</span>]{}& [<span style="font-variant:small-caps;">Base</span>]{}& [<span style="font-variant:small-caps;">FastText$_{\small \textsc{cc}}$</span>]{}& & [<span style="font-variant:small-caps;">ELMo</span>]{}& [<span style="font-variant:small-caps;">ELMo*-A*$_{\small \textsc{}}$</span>]{}& [<span style="font-variant:small-caps;">ELMo</span>*-S*$_{\small \textsc{}}$]{}& & [<span style="font-variant:small-caps;">ELMo*-S*$_{\small \textsc{OneB}}$</span>]{}& [<span style="font-variant:small-caps;">ELMo*-S*$_{\small \textsc{CNN}}$</span>]{}\

SNLI & & 88.7 & 88.0 & 87.7 & & 88.5 & 88.9 & 88.8 & & 88.4 & 88.2\
Coref 