HirCoir commited on
Commit
036ac56
·
verified ·
1 Parent(s): 2cf6314

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -142
README.md CHANGED
@@ -7,115 +7,7 @@ license: apache-2.0
7
  library_name: transformers
8
  widget:
9
  - text: <s> [|User|] Hola </s>[|Assistant|]
10
- model-index:
11
- - name: MiniChat-2-3B
12
- results:
13
- - task:
14
- type: text-generation
15
- name: Text Generation
16
- dataset:
17
- name: AI2 Reasoning Challenge (25-Shot)
18
- type: ai2_arc
19
- config: ARC-Challenge
20
- split: test
21
- args:
22
- num_few_shot: 25
23
- metrics:
24
- - type: acc_norm
25
- value: 44.88
26
- name: normalized accuracy
27
- source:
28
- url: >-
29
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-2-3B
30
- name: Open LLM Leaderboard
31
- - task:
32
- type: text-generation
33
- name: Text Generation
34
- dataset:
35
- name: HellaSwag (10-Shot)
36
- type: hellaswag
37
- split: validation
38
- args:
39
- num_few_shot: 10
40
- metrics:
41
- - type: acc_norm
42
- value: 67.69
43
- name: normalized accuracy
44
- source:
45
- url: >-
46
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-2-3B
47
- name: Open LLM Leaderboard
48
- - task:
49
- type: text-generation
50
- name: Text Generation
51
- dataset:
52
- name: MMLU (5-Shot)
53
- type: cais/mmlu
54
- config: all
55
- split: test
56
- args:
57
- num_few_shot: 5
58
- metrics:
59
- - type: acc
60
- value: 47.59
61
- name: accuracy
62
- source:
63
- url: >-
64
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-2-3B
65
- name: Open LLM Leaderboard
66
- - task:
67
- type: text-generation
68
- name: Text Generation
69
- dataset:
70
- name: TruthfulQA (0-shot)
71
- type: truthful_qa
72
- config: multiple_choice
73
- split: validation
74
- args:
75
- num_few_shot: 0
76
- metrics:
77
- - type: mc2
78
- value: 49.64
79
- source:
80
- url: >-
81
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-2-3B
82
- name: Open LLM Leaderboard
83
- - task:
84
- type: text-generation
85
- name: Text Generation
86
- dataset:
87
- name: Winogrande (5-shot)
88
- type: winogrande
89
- config: winogrande_xl
90
- split: validation
91
- args:
92
- num_few_shot: 5
93
- metrics:
94
- - type: acc
95
- value: 66.46
96
- name: accuracy
97
- source:
98
- url: >-
99
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-2-3B
100
- name: Open LLM Leaderboard
101
- - task:
102
- type: text-generation
103
- name: Text Generation
104
- dataset:
105
- name: GSM8k (5-shot)
106
- type: gsm8k
107
- config: main
108
- split: test
109
- args:
110
- num_few_shot: 5
111
- metrics:
112
- - type: acc
113
- value: 32.68
114
- name: accuracy
115
- source:
116
- url: >-
117
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=GeneZC/MiniChat-2-3B
118
- name: Open LLM Leaderboard
119
  tags:
120
  - unsloth
121
  - Sorah
@@ -124,16 +16,6 @@ tags:
124
 
125
  ## MiniChat-2-3B
126
 
127
- 📑 [arXiv](https://arxiv.org/abs/2311.07052) | 👻 [GitHub](https://github.com/GeneZC/MiniMA) | 🤗 [HuggingFace-MiniMA](https://huggingface.co/GeneZC/MiniMA-3B) | 🤗 [HuggingFace-MiniChat](https://huggingface.co/GeneZC/MiniChat-3B) | 🤖 [ModelScope-MiniMA](https://modelscope.cn/models/GeneZC/MiniMA-3B) | 🤖 [ModelScope-MiniChat](https://modelscope.cn/models/GeneZC/MiniChat-3B) | 🤗 [HuggingFace-MiniChat-1.5](https://huggingface.co/GeneZC/MiniChat-1.5-3B) | 🤗 [HuggingFace-MiniMA-2](https://huggingface.co/GeneZC/MiniMA-2-3B) | 🤗 [HuggingFace-MiniChat-2](https://huggingface.co/GeneZC/MiniChat-2-3B)
128
-
129
- 🆕 **Updates from MiniChat-3B**:
130
- - better base model MiniMA-2-3B;
131
- - better data mixture;
132
- - use of [NEFTune](https://arxiv.org/abs/2310.05914);
133
- - use of [DPO](https://arxiv.org/abs/2305.18290).
134
-
135
- ❗ Must comply with LICENSE of LLaMA2 since it is derived from LLaMA2.
136
-
137
  A language model continued from MiniMA-3B and finetuned on both instruction and preference data.
138
 
139
  Surpassing Vicuna-7B and approximating LLaMA-2-Chat-7B on MT-Bench.
@@ -173,26 +55,3 @@ output = tokenizer.decode(output_ids, skip_special_tokens=True).strip()
173
  # output: "def common_elements(arr1, arr2):\n if len(arr1) == 0:\n return []\n if len(arr2) == 0:\n return arr1\n\n common_elements = []\n for element in arr1:\n if element in arr2:\n common_elements.append(element)\n\n return common_elements"
174
  # Multiturn conversation could be realized by continuously appending questions to `conv`.
175
  ```
176
-
177
- ## Bibtex
178
-
179
- ```bibtex
180
- @article{zhang2023law,
181
- title={Towards the Law of Capacity Gap in Distilling Language Models},
182
- author={Zhang, Chen and Song, Dawei and Ye, Zheyu and Gao, Yan},
183
- year={2023},
184
- url={https://arxiv.org/abs/2311.07052}
185
- }
186
- ```
187
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
188
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_GeneZC__MiniChat-2-3B)
189
-
190
- | Metric |Value|
191
- |---------------------------------|----:|
192
- |Avg. |51.49|
193
- |AI2 Reasoning Challenge (25-Shot)|44.88|
194
- |HellaSwag (10-Shot) |67.69|
195
- |MMLU (5-Shot) |47.59|
196
- |TruthfulQA (0-shot) |49.64|
197
- |Winogrande (5-shot) |66.46|
198
- |GSM8k (5-shot) |32.68|
 
7
  library_name: transformers
8
  widget:
9
  - text: <s> [|User|] Hola </s>[|Assistant|]
10
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  tags:
12
  - unsloth
13
  - Sorah
 
16
 
17
  ## MiniChat-2-3B
18
 
 
 
 
 
 
 
 
 
 
 
19
  A language model continued from MiniMA-3B and finetuned on both instruction and preference data.
20
 
21
  Surpassing Vicuna-7B and approximating LLaMA-2-Chat-7B on MT-Bench.
 
55
  # output: "def common_elements(arr1, arr2):\n if len(arr1) == 0:\n return []\n if len(arr2) == 0:\n return arr1\n\n common_elements = []\n for element in arr1:\n if element in arr2:\n common_elements.append(element)\n\n return common_elements"
56
  # Multiturn conversation could be realized by continuously appending questions to `conv`.
57
  ```