Update README.md
Browse files
README.md
CHANGED
|
@@ -12,7 +12,7 @@ pipeline_tag: text-generation
|
|
| 12 |
---
|
| 13 |
|
| 14 |
|
| 15 |
-
# MiniLLM
|
| 16 |
|
| 17 |
[paper](https://arxiv.org/abs/2306.08543) | [code](https://github.com/microsoft/LMOps/tree/main/minillm)
|
| 18 |
|
|
@@ -22,7 +22,7 @@ pipeline_tag: text-generation
|
|
| 22 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/624ac662102fcdff87be51b9/7hBWGZzYMJihCRQ70XoiQ.png" width="1000">
|
| 23 |
</p>
|
| 24 |
|
| 25 |
-
**Note**: MiniLLM requires a [SFT model]() for initilization to perform the PPO optimization.
|
| 26 |
|
| 27 |
## Evaluation
|
| 28 |
|
|
@@ -33,9 +33,9 @@ We ask GPT-4 to give scores for the generated responses of MiniLLM. The prompts
|
|
| 33 |
</p>
|
| 34 |
|
| 35 |
## Baseline Models
|
| 36 |
-
+ [SFT w/o KD]()
|
| 37 |
-
+ [KD]()
|
| 38 |
-
+ [SeqKD]()
|
| 39 |
|
| 40 |
## Citation
|
| 41 |
```
|
|
|
|
| 12 |
---
|
| 13 |
|
| 14 |
|
| 15 |
+
# MiniLLM-gpt2-760M
|
| 16 |
|
| 17 |
[paper](https://arxiv.org/abs/2306.08543) | [code](https://github.com/microsoft/LMOps/tree/main/minillm)
|
| 18 |
|
|
|
|
| 22 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/624ac662102fcdff87be51b9/7hBWGZzYMJihCRQ70XoiQ.png" width="1000">
|
| 23 |
</p>
|
| 24 |
|
| 25 |
+
**Note**: MiniLLM requires a [SFT model](https://huggingface.co/MiniLLM/init-gpt2-760M) for initilization to perform the PPO optimization.
|
| 26 |
|
| 27 |
## Evaluation
|
| 28 |
|
|
|
|
| 33 |
</p>
|
| 34 |
|
| 35 |
## Baseline Models
|
| 36 |
+
+ [SFT w/o KD](https://huggingface.co/MiniLLM/SFT-gpt2-760M)
|
| 37 |
+
+ [KD](https://huggingface.co/MiniLLM/KD-gpt2-760M)
|
| 38 |
+
+ [SeqKD](https://huggingface.co/MiniLLM/SeqKD-gpt2-760M)
|
| 39 |
|
| 40 |
## Citation
|
| 41 |
```
|