Hugging Face's logo

YOYO-AI
/
Qwen2.5-14B-1M-YOYO-V3

Text Generation
Safetensors
English
Chinese
qwen2
Merge
conversational
Eval Results
Model card Files Community
YOYO-AI commited on
Commit
8e675bf
·
verified ·
1 Parent(s): bb54e01

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +107 -1
README.md CHANGED
@@ -13,6 +13,101 @@ base_model:
13
  pipeline_tag: text-generation
14
  tags:
15
  - merge
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  ---
17
 
18
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64e174e202fa032de4143324/CfIE4_oZgpNsNZyurjO7D.png)
@@ -46,7 +141,18 @@ name: Qwen2.5-14B-YOYO-latest-V2
46
  Although the uncontrollable output issue has been addressed, the model still lacks stability.
47
 
48
  Through practical experimentation, I found that first merging **"high-divergence"** models (significantly different from the base) into **"low-divergence"** models (closer to the base) using the [DELLA](https://arxiv.org/abs/2406.11617) method, then applying the [Model Stock](https://arxiv.org/abs/2403.19522) method, ultimately produces a model that is not only more stable but also achieves better performance.
 
 
49
 
 
 
 
 
 
 
 
 
 
50
  ## Key models used:
51
  *1. Low-divergence, high-performance models:*
52
 
@@ -191,4 +297,4 @@ int8_mask: true
191
  normalize: true
192
  name: Qwen2.5-14B-1M-YOYO-V3
193
  ```
194
- I hope this helps!
 
13
  pipeline_tag: text-generation
14
  tags:
15
  - merge
16
+ model-index:
17
+ - name: Qwen2.5-14B-1M-YOYO-V3
18
+ results:
19
+ - task:
20
+ type: text-generation
21
+ name: Text Generation
22
+ dataset:
23
+ name: IFEval (0-Shot)
24
+ type: HuggingFaceH4/ifeval
25
+ args:
26
+ num_few_shot: 0
27
+ metrics:
28
+ - type: inst_level_strict_acc and prompt_level_strict_acc
29
+ value: 83.98
30
+ name: strict accuracy
31
+ source:
32
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=YOYO-AI/Qwen2.5-14B-1M-YOYO-V3
33
+ name: Open LLM Leaderboard
34
+ - task:
35
+ type: text-generation
36
+ name: Text Generation
37
+ dataset:
38
+ name: BBH (3-Shot)
39
+ type: BBH
40
+ args:
41
+ num_few_shot: 3
42
+ metrics:
43
+ - type: acc_norm
44
+ value: 49.47
45
+ name: normalized accuracy
46
+ source:
47
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=YOYO-AI/Qwen2.5-14B-1M-YOYO-V3
48
+ name: Open LLM Leaderboard
49
+ - task:
50
+ type: text-generation
51
+ name: Text Generation
52
+ dataset:
53
+ name: MATH Lvl 5 (4-Shot)
54
+ type: hendrycks/competition_math
55
+ args:
56
+ num_few_shot: 4
57
+ metrics:
58
+ - type: exact_match
59
+ value: 53.55
60
+ name: exact match
61
+ source:
62
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=YOYO-AI/Qwen2.5-14B-1M-YOYO-V3
63
+ name: Open LLM Leaderboard
64
+ - task:
65
+ type: text-generation
66
+ name: Text Generation
67
+ dataset:
68
+ name: GPQA (0-shot)
69
+ type: Idavidrein/gpqa
70
+ args:
71
+ num_few_shot: 0
72
+ metrics:
73
+ - type: acc_norm
74
+ value: 10.51
75
+ name: acc_norm
76
+ source:
77
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=YOYO-AI/Qwen2.5-14B-1M-YOYO-V3
78
+ name: Open LLM Leaderboard
79
+ - task:
80
+ type: text-generation
81
+ name: Text Generation
82
+ dataset:
83
+ name: MuSR (0-shot)
84
+ type: TAUR-Lab/MuSR
85
+ args:
86
+ num_few_shot: 0
87
+ metrics:
88
+ - type: acc_norm
89
+ value: 11.10
90
+ name: acc_norm
91
+ source:
92
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=YOYO-AI/Qwen2.5-14B-1M-YOYO-V3
93
+ name: Open LLM Leaderboard
94
+ - task:
95
+ type: text-generation
96
+ name: Text Generation
97
+ dataset:
98
+ name: MMLU-PRO (5-shot)
99
+ type: TIGER-Lab/MMLU-Pro
100
+ config: main
101
+ split: test
102
+ args:
103
+ num_few_shot: 5
104
+ metrics:
105
+ - type: acc
106
+ value: 46.74
107
+ name: accuracy
108
+ source:
109
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=YOYO-AI/Qwen2.5-14B-1M-YOYO-V3
110
+ name: Open LLM Leaderboard
111
  ---
112
 
113
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64e174e202fa032de4143324/CfIE4_oZgpNsNZyurjO7D.png)
 
141
  Although the uncontrollable output issue has been addressed, the model still lacks stability.
142
 
143
  Through practical experimentation, I found that first merging **"high-divergence"** models (significantly different from the base) into **"low-divergence"** models (closer to the base) using the [DELLA](https://arxiv.org/abs/2406.11617) method, then applying the [Model Stock](https://arxiv.org/abs/2403.19522) method, ultimately produces a model that is not only more stable but also achieves better performance.
144
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
145
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Replete-AI__Replete-LLM-V2.5-Qwen-14b)
146
 
147
+ | Metric |Value|
148
+ |-------------------|----:|
149
+ |Avg. |42.56|
150
+ |IFEval (0-Shot) |83.98|
151
+ |BBH (3-Shot) |49.47|
152
+ |MATH Lvl 5 (4-Shot)|53.55|
153
+ |GPQA (0-shot) |10.51|
154
+ |MuSR (0-shot) |11.10|
155
+ |MMLU-PRO (5-shot) |46.74|
156
  ## Key models used:
157
  *1. Low-divergence, high-performance models:*
158
 
 
297
  normalize: true
298
  name: Qwen2.5-14B-1M-YOYO-V3
299
  ```
300
+ I hope this helps!