File size: 972 Bytes
e4f5830 da371cc 86f0121 da371cc |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
---
language:
- en
license: apache-2.0
datasets:
- delphi-suite/stories
library_name: delphi
---
This is a part of `stories-llama2-*` model family:
name | params | layers | hidden_size | query heads | key & value heads
-|-|-|-|-|-
stories-llama2-50k | 49,554 | 1 | 6 | 3 | 1
stories-llama2-100k | 99,924 | 1 | 12 | 2 | 1
stories-llama2-250k | 246,820 | 2 | 28 | 2 | 1
stories-llama2-500k | 527,912 | 2 | 56 | 4 | 2
stories-llama2-1m | 1,019,508 | 4 | 84 | 6 | 3
stories-llama2-2.5m | 2,437,280 | 4 | 160 | 8 | 4
stories-llama2-5m | 5,136,720 | 5 | 240 | 10 | 5
stories-llama2-10m | 10,421,340 | 6 | 340 | 10 | 5
stories-llama2-25m | 24,215,520 | 8 | 480 | 16 | 8
stories-llama2-50m | 49,387,712 | 8 | 704 | 16 | 8
You can access W&B logs [here](https://wandb.ai/delphi-suite/delphi).
This model was trained using [delphi](https://github.com/delphi-suite/delphi). See `training_config.json` and `run_context.json` for details.
|