Update README.md
Browse files
README.md
CHANGED
|
@@ -13,6 +13,7 @@ datasets:
|
|
| 13 |
pipeline_tag: text-generation
|
| 14 |
---
|
| 15 |
|
|
|
|
| 16 |
|
| 17 |
# beomi/KoAlpaca-KoRWKV-1.5B (v1.0)
|
| 18 |
|
|
@@ -22,6 +23,11 @@ Dataset available at [KoAlpaca Github Repository](https://github.com/Beomi/KoAlp
|
|
| 22 |
|
| 23 |
## Training procedure
|
| 24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
### Training hyperparameters
|
| 26 |
|
| 27 |
The following hyperparameters were used during training:
|
|
|
|
| 13 |
pipeline_tag: text-generation
|
| 14 |
---
|
| 15 |
|
| 16 |
+
> 🚧 Note: this repo is under construction, current uploaded version is finetuned version of KoRWKV which is ~20% trained ckpt (with ~31Billion tokens) 🚧
|
| 17 |
|
| 18 |
# beomi/KoAlpaca-KoRWKV-1.5B (v1.0)
|
| 19 |
|
|
|
|
| 23 |
|
| 24 |
## Training procedure
|
| 25 |
|
| 26 |
+
### Train Device
|
| 27 |
+
|
| 28 |
+
- A100 80G x2
|
| 29 |
+
- ~2hrs
|
| 30 |
+
|
| 31 |
### Training hyperparameters
|
| 32 |
|
| 33 |
The following hyperparameters were used during training:
|