Hugging Face's logo

ValiantLabs
/
Llama3.1-8B-Cobalt

Text Generation
Safetensors
English
llama
cobalt
valiant
valiant-labs
llama-3.1
llama-3.1-instruct
llama-3.1-instruct-8b
llama-3
llama-3-instruct
llama-3-instruct-8b
8b
math
math-instruct
conversational
chat
instruct
Eval Results
Model card Files
xet
Community
2

Adding Evaluation Results

#2
by leaderboard-pr-bot - opened
base: refs/heads/main
←
from: refs/pr/2
Discussion Files changed
+113
-3
Files changed (1) hide show
  1. README.md +113 -3
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  language:
3
  - en
4
- pipeline_tag: text-generation
5
  tags:
6
  - cobalt
7
  - valiant
@@ -19,12 +19,107 @@ tags:
19
  - conversational
20
  - chat
21
  - instruct
22
- model_type: llama
23
  base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
24
  datasets:
25
  - sequelbox/Polytope
26
  - LDJnr/Pure-Dove
27
- license: llama3.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  ---
29
 
30
 
@@ -77,6 +172,21 @@ Cobalt is built on top of Llama 3.1 8b Instruct, using math-instruct data to sup
77
  Our current version of the Cobalt math-instruct dataset is [sequelbox/Polytope](https://huggingface.co/datasets/sequelbox/Polytope), supplemented with a small selection of data from [LDJnr/Pure-Dove](https://huggingface.co/datasets/LDJnr/Pure-Dove) for general chat consistency.
78
 
79
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
80
  ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/63444f2687964b331809eb55/VCJ8Fmefd8cdVhXSSxJiD.jpeg)
81
 
82
 
 
1
  ---
2
  language:
3
  - en
4
+ license: llama3.1
5
  tags:
6
  - cobalt
7
  - valiant
 
19
  - conversational
20
  - chat
21
  - instruct
 
22
  base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
23
  datasets:
24
  - sequelbox/Polytope
25
  - LDJnr/Pure-Dove
26
+ pipeline_tag: text-generation
27
+ model_type: llama
28
+ model-index:
29
+ - name: Llama3.1-8B-Cobalt
30
+ results:
31
+ - task:
32
+ type: text-generation
33
+ name: Text Generation
34
+ dataset:
35
+ name: IFEval (0-Shot)
36
+ type: HuggingFaceH4/ifeval
37
+ args:
38
+ num_few_shot: 0
39
+ metrics:
40
+ - type: inst_level_strict_acc and prompt_level_strict_acc
41
+ value: 73.94
42
+ name: strict accuracy
43
+ source:
44
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ValiantLabs/Llama3.1-8B-Cobalt
45
+ name: Open LLM Leaderboard
46
+ - task:
47
+ type: text-generation
48
+ name: Text Generation
49
+ dataset:
50
+ name: BBH (3-Shot)
51
+ type: BBH
52
+ args:
53
+ num_few_shot: 3
54
+ metrics:
55
+ - type: acc_norm
56
+ value: 27.53
57
+ name: normalized accuracy
58
+ source:
59
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ValiantLabs/Llama3.1-8B-Cobalt
60
+ name: Open LLM Leaderboard
61
+ - task:
62
+ type: text-generation
63
+ name: Text Generation
64
+ dataset:
65
+ name: MATH Lvl 5 (4-Shot)
66
+ type: hendrycks/competition_math
67
+ args:
68
+ num_few_shot: 4
69
+ metrics:
70
+ - type: exact_match
71
+ value: 15.48
72
+ name: exact match
73
+ source:
74
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ValiantLabs/Llama3.1-8B-Cobalt
75
+ name: Open LLM Leaderboard
76
+ - task:
77
+ type: text-generation
78
+ name: Text Generation
79
+ dataset:
80
+ name: GPQA (0-shot)
81
+ type: Idavidrein/gpqa
82
+ args:
83
+ num_few_shot: 0
84
+ metrics:
85
+ - type: acc_norm
86
+ value: 4.47
87
+ name: acc_norm
88
+ source:
89
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ValiantLabs/Llama3.1-8B-Cobalt
90
+ name: Open LLM Leaderboard
91
+ - task:
92
+ type: text-generation
93
+ name: Text Generation
94
+ dataset:
95
+ name: MuSR (0-shot)
96
+ type: TAUR-Lab/MuSR
97
+ args:
98
+ num_few_shot: 0
99
+ metrics:
100
+ - type: acc_norm
101
+ value: 5.08
102
+ name: acc_norm
103
+ source:
104
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ValiantLabs/Llama3.1-8B-Cobalt
105
+ name: Open LLM Leaderboard
106
+ - task:
107
+ type: text-generation
108
+ name: Text Generation
109
+ dataset:
110
+ name: MMLU-PRO (5-shot)
111
+ type: TIGER-Lab/MMLU-Pro
112
+ config: main
113
+ split: test
114
+ args:
115
+ num_few_shot: 5
116
+ metrics:
117
+ - type: acc
118
+ value: 29.56
119
+ name: accuracy
120
+ source:
121
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ValiantLabs/Llama3.1-8B-Cobalt
122
+ name: Open LLM Leaderboard
123
  ---
124
 
125
 
 
172
  Our current version of the Cobalt math-instruct dataset is [sequelbox/Polytope](https://huggingface.co/datasets/sequelbox/Polytope), supplemented with a small selection of data from [LDJnr/Pure-Dove](https://huggingface.co/datasets/LDJnr/Pure-Dove) for general chat consistency.
173
 
174
 
175
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
176
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ValiantLabs__Llama3.1-8B-Cobalt)
177
+
178
+ | Metric |Value|
179
+ |-------------------|----:|
180
+ |Avg. |26.01|
181
+ |IFEval (0-Shot) |73.94|
182
+ |BBH (3-Shot) |27.53|
183
+ |MATH Lvl 5 (4-Shot)|15.48|
184
+ |GPQA (0-shot) | 4.47|
185
+ |MuSR (0-shot) | 5.08|
186
+ |MMLU-PRO (5-shot) |29.56|
187
+
188
+
189
+
190
  ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/63444f2687964b331809eb55/VCJ8Fmefd8cdVhXSSxJiD.jpeg)
191
 
192