Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,56 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
+
tags:
|
4 |
+
- text-generation
|
5 |
+
base_model: EleutherAI/pythia-31m
|
6 |
+
datasets:
|
7 |
+
- totally-not-an-llm/EverythingLM-data-V3
|
8 |
+
- databricks/databricks-dolly-15k
|
9 |
+
- THUDM/webglm-qa
|
10 |
+
- starfishmedical/webGPT_x_dolly
|
11 |
+
- Amod/mental_health_counseling_conversations
|
12 |
+
- sablo/oasst2_curated
|
13 |
+
- cognitivecomputations/wizard_vicuna_70k_unfiltered
|
14 |
+
- mlabonne/chatml_dpo_pairs
|
15 |
+
inference:
|
16 |
+
parameters:
|
17 |
+
max_new_tokens: 250
|
18 |
+
do_sample: true
|
19 |
+
temperature: 0.4
|
20 |
+
top_p: 0.25
|
21 |
+
top_k: 7
|
22 |
+
repetition_penalty: 1.0016
|
23 |
---
|
24 |
+
|
25 |
+
# A Pythia Chat Model of 31M Parameters
|
26 |
+
|
27 |
+
- Base model: [EleutherAI/pythia-31m](https://huggingface.co/EleutherAI/pythia-31m)
|
28 |
+
- Datasets:
|
29 |
+
- [totally-not-an-llm/EverythingLM-data-V3](https://huggingface.co/datasets/totally-not-an-llm/EverythingLM-data-V3)
|
30 |
+
- [databricks/databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k)
|
31 |
+
- [THUDM/webglm-qa](https://huggingface.co/datasets/THUDM/webglm-qa)
|
32 |
+
- [starfishmedical/webGPT_x_dolly](https://huggingface.co/datasets/starfishmedical/webGPT_x_dolly)
|
33 |
+
- [Amod/mental_health_counseling_conversations](https://huggingface.co/datasets/Amod/mental_health_counseling_conversations)
|
34 |
+
- [sablo/oasst2_curated](https://huggingface.co/datasets/sablo/oasst2_curated)
|
35 |
+
- [cognitivecomputations/wizard_vicuna_70k_unfiltered](https://huggingface.co/datasets/cognitivecomputations/wizard_vicuna_70k_unfiltered)
|
36 |
+
- [mlabonne/chatml_dpo_pairs](https://huggingface.co/datasets/mlabonne/chatml_dpo_pairs)
|
37 |
+
|
38 |
+
## Recommended Prompt Format
|
39 |
+
|
40 |
+
```
|
41 |
+
<|im_start|>system
|
42 |
+
{system_message}<|im_end|>
|
43 |
+
<|im_start|>user
|
44 |
+
{user_message}<|im_end|>
|
45 |
+
<|im_start|>assistant
|
46 |
+
```
|
47 |
+
|
48 |
+
## Recommended Inference Parameters
|
49 |
+
|
50 |
+
```yml
|
51 |
+
do_sample: true
|
52 |
+
temperature: 0.4
|
53 |
+
top_p: 0.25
|
54 |
+
top_k: 7
|
55 |
+
repetition_penalty: 1.0016
|
56 |
+
```
|