Update README.md
Browse files
README.md
CHANGED
@@ -7,4 +7,24 @@ datasets:
|
|
7 |
library_name: delphi
|
8 |
---
|
9 |
|
10 |
-
This
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
library_name: delphi
|
8 |
---
|
9 |
|
10 |
+
This is a part of `stories-llama2-*` model family:
|
11 |
+
|
12 |
+
name | params | layers | hidden_size | query heads | key & value heads
|
13 |
+
-|-|-|-|-|-
|
14 |
+
stories-llama2-50k | 49,554 | 1 | 6 | 3 | 1
|
15 |
+
stories-llama2-100k | 99,924 | 1 | 12 | 2 | 1
|
16 |
+
stories-llama2-250k | 246,820 | 2 | 28 | 2 | 1
|
17 |
+
stories-llama2-500k | 527,912 | 2 | 56 | 4 | 2
|
18 |
+
stories-llama2-1m | 1,019,508 | 4 | 84 | 6 | 3
|
19 |
+
stories-llama2-2.5m | 2,437,280 | 4 | 160 | 8 | 4
|
20 |
+
stories-llama2-5m | 5,136,720 | 5 | 240 | 10 | 5
|
21 |
+
stories-llama2-10m | 10,421,340 | 6 | 340 | 10 | 5
|
22 |
+
stories-llama2-25m | 24,215,520 | 8 | 480 | 16 | 8
|
23 |
+
stories-llama2-50m | 49,387,712 | 8 | 704 | 16 | 8
|
24 |
+
|
25 |
+
This model was trained using [delphi](https://github.com/delphi-suite/delphi). See `training_config.json` and `run_context.json` for details.
|
26 |
+
|
27 |
+
|
28 |
+
|
29 |
+
|
30 |
+
|