Tel Aviv University; IBM Research
Ido Amos, Jonathan Berant, Ankit Gupta
Pre Training;Transformers;State Space Models;Long Range Models;Fair Evaluation
Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors