This checkpoint of the 1.3B GLA model used in the paper Gated Linear Attention. The model is trained with 100B tokens from the SlimPajama dataset tokenized with Llama2 tokenizer.
See the model and loading script in this repo.
- Downloads last month
- 77
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.