Hugging Face's logo

eaddario
/
Hammer2.1-7b-GGUF

Text Generation
GGUF
English
quant
experimental
Inference Endpoints
conversational
Model card Files Community
eaddario commited on
Commit
ed91853
·
verified ·
1 Parent(s): 13ee655

Generate Perplexity, KLD, ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. scores/Hammer2.1-7b-F16.arc +763 -0
  2. scores/Hammer2.1-7b-F16.hsw +760 -0
  3. scores/Hammer2.1-7b-F16.mmlu +763 -0
  4. scores/Hammer2.1-7b-F16.tqa +763 -0
  5. scores/Hammer2.1-7b-F16.wng +760 -0
  6. scores/Hammer2.1-7b-iq3_m.arc +763 -0
  7. scores/Hammer2.1-7b-iq3_m.hsw +760 -0
  8. scores/Hammer2.1-7b-iq3_m.mmlu +763 -0
  9. scores/Hammer2.1-7b-iq3_m.ppx +37 -0
  10. scores/Hammer2.1-7b-iq3_m.tqa +763 -0
  11. scores/Hammer2.1-7b-iq3_m.wng +760 -0
  12. scores/Hammer2.1-7b-iq3_s.arc +763 -0
  13. scores/Hammer2.1-7b-iq3_s.hsw +760 -0
  14. scores/Hammer2.1-7b-iq3_s.mmlu +763 -0
  15. scores/Hammer2.1-7b-iq3_s.ppx +37 -0
  16. scores/Hammer2.1-7b-iq3_s.tqa +763 -0
  17. scores/Hammer2.1-7b-iq3_s.wng +760 -0
  18. scores/Hammer2.1-7b-iq4_nl.arc +763 -0
  19. scores/Hammer2.1-7b-iq4_nl.hsw +760 -0
  20. scores/Hammer2.1-7b-iq4_nl.mmlu +763 -0
  21. scores/Hammer2.1-7b-iq4_nl.ppx +37 -0
  22. scores/Hammer2.1-7b-iq4_nl.tqa +763 -0
  23. scores/Hammer2.1-7b-iq4_nl.wng +760 -0
  24. scores/Hammer2.1-7b-q3_k_l.arc +763 -0
  25. scores/Hammer2.1-7b-q3_k_l.hsw +760 -0
  26. scores/Hammer2.1-7b-q3_k_l.mmlu +763 -0
  27. scores/Hammer2.1-7b-q3_k_l.ppx +37 -0
  28. scores/Hammer2.1-7b-q3_k_l.tqa +763 -0
  29. scores/Hammer2.1-7b-q3_k_l.wng +760 -0
  30. scores/Hammer2.1-7b-q3_k_m.arc +763 -0
  31. scores/Hammer2.1-7b-q3_k_m.hsw +760 -0
  32. scores/Hammer2.1-7b-q3_k_m.mmlu +763 -0
  33. scores/Hammer2.1-7b-q3_k_m.ppx +37 -0
  34. scores/Hammer2.1-7b-q3_k_m.tqa +763 -0
  35. scores/Hammer2.1-7b-q3_k_m.wng +760 -0
  36. scores/Hammer2.1-7b-q3_k_s.arc +763 -0
  37. scores/Hammer2.1-7b-q3_k_s.hsw +760 -0
  38. scores/Hammer2.1-7b-q3_k_s.mmlu +763 -0
  39. scores/Hammer2.1-7b-q3_k_s.ppx +37 -0
  40. scores/Hammer2.1-7b-q3_k_s.tqa +763 -0
  41. scores/Hammer2.1-7b-q3_k_s.wng +760 -0
  42. scores/Hammer2.1-7b-q4_k_m-naive.arc +763 -0
  43. scores/Hammer2.1-7b-q4_k_m-naive.hsw +760 -0
  44. scores/Hammer2.1-7b-q4_k_m-naive.mmlu +763 -0
  45. scores/Hammer2.1-7b-q4_k_m-naive.tqa +763 -0
  46. scores/Hammer2.1-7b-q4_k_m-naive.wng +760 -0
  47. scores/Hammer2.1-7b-q4_k_m.arc +763 -0
  48. scores/Hammer2.1-7b-q4_k_m.hsw +760 -0
  49. scores/Hammer2.1-7b-q4_k_m.mmlu +763 -0
  50. scores/Hammer2.1-7b-q4_k_m.ppx +37 -0
scores/Hammer2.1-7b-F16.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 57.14285714
18
+ 8 62.50000000
19
+ 9 55.55555556
20
+ 10 60.00000000
21
+ 11 54.54545455
22
+ 12 58.33333333
23
+ 13 61.53846154
24
+ 14 64.28571429
25
+ 15 66.66666667
26
+ 16 68.75000000
27
+ 17 70.58823529
28
+ 18 66.66666667
29
+ 19 63.15789474
30
+ 20 60.00000000
31
+ 21 61.90476190
32
+ 22 63.63636364
33
+ 23 60.86956522
34
+ 24 62.50000000
35
+ 25 60.00000000
36
+ 26 57.69230769
37
+ 27 59.25925926
38
+ 28 57.14285714
39
+ 29 58.62068966
40
+ 30 60.00000000
41
+ 31 58.06451613
42
+ 32 56.25000000
43
+ 33 57.57575758
44
+ 34 55.88235294
45
+ 35 57.14285714
46
+ 36 55.55555556
47
+ 37 56.75675676
48
+ 38 57.89473684
49
+ 39 58.97435897
50
+ 40 60.00000000
51
+ 41 60.97560976
52
+ 42 61.90476190
53
+ 43 62.79069767
54
+ 44 61.36363636
55
+ 45 62.22222222
56
+ 46 63.04347826
57
+ 47 63.82978723
58
+ 48 64.58333333
59
+ 49 63.26530612
60
+ 50 64.00000000
61
+ 51 62.74509804
62
+ 52 63.46153846
63
+ 53 64.15094340
64
+ 54 62.96296296
65
+ 55 61.81818182
66
+ 56 62.50000000
67
+ 57 63.15789474
68
+ 58 63.79310345
69
+ 59 64.40677966
70
+ 60 65.00000000
71
+ 61 63.93442623
72
+ 62 62.90322581
73
+ 63 63.49206349
74
+ 64 64.06250000
75
+ 65 63.07692308
76
+ 66 63.63636364
77
+ 67 64.17910448
78
+ 68 64.70588235
79
+ 69 63.76811594
80
+ 70 62.85714286
81
+ 71 61.97183099
82
+ 72 61.11111111
83
+ 73 61.64383562
84
+ 74 62.16216216
85
+ 75 62.66666667
86
+ 76 63.15789474
87
+ 77 63.63636364
88
+ 78 64.10256410
89
+ 79 63.29113924
90
+ 80 63.75000000
91
+ 81 62.96296296
92
+ 82 63.41463415
93
+ 83 62.65060241
94
+ 84 61.90476190
95
+ 85 62.35294118
96
+ 86 62.79069767
97
+ 87 63.21839080
98
+ 88 63.63636364
99
+ 89 64.04494382
100
+ 90 64.44444444
101
+ 91 64.83516484
102
+ 92 65.21739130
103
+ 93 64.51612903
104
+ 94 63.82978723
105
+ 95 64.21052632
106
+ 96 64.58333333
107
+ 97 63.91752577
108
+ 98 63.26530612
109
+ 99 63.63636364
110
+ 100 64.00000000
111
+ 101 63.36633663
112
+ 102 63.72549020
113
+ 103 63.10679612
114
+ 104 62.50000000
115
+ 105 62.85714286
116
+ 106 62.26415094
117
+ 107 62.61682243
118
+ 108 62.03703704
119
+ 109 61.46788991
120
+ 110 61.81818182
121
+ 111 62.16216216
122
+ 112 62.50000000
123
+ 113 61.94690265
124
+ 114 62.28070175
125
+ 115 62.60869565
126
+ 116 62.06896552
127
+ 117 61.53846154
128
+ 118 61.86440678
129
+ 119 61.34453782
130
+ 120 61.66666667
131
+ 121 61.15702479
132
+ 122 61.47540984
133
+ 123 61.78861789
134
+ 124 61.29032258
135
+ 125 61.60000000
136
+ 126 61.90476190
137
+ 127 61.41732283
138
+ 128 61.71875000
139
+ 129 61.24031008
140
+ 130 60.76923077
141
+ 131 61.06870229
142
+ 132 61.36363636
143
+ 133 61.65413534
144
+ 134 61.19402985
145
+ 135 61.48148148
146
+ 136 61.02941176
147
+ 137 61.31386861
148
+ 138 61.59420290
149
+ 139 61.87050360
150
+ 140 62.14285714
151
+ 141 62.41134752
152
+ 142 61.97183099
153
+ 143 62.23776224
154
+ 144 61.80555556
155
+ 145 62.06896552
156
+ 146 61.64383562
157
+ 147 61.90476190
158
+ 148 62.16216216
159
+ 149 62.41610738
160
+ 150 62.66666667
161
+ 151 62.91390728
162
+ 152 63.15789474
163
+ 153 63.39869281
164
+ 154 62.98701299
165
+ 155 63.22580645
166
+ 156 63.46153846
167
+ 157 63.69426752
168
+ 158 63.92405063
169
+ 159 64.15094340
170
+ 160 63.75000000
171
+ 161 63.35403727
172
+ 162 63.58024691
173
+ 163 63.19018405
174
+ 164 62.80487805
175
+ 165 63.03030303
176
+ 166 63.25301205
177
+ 167 62.87425150
178
+ 168 63.09523810
179
+ 169 63.31360947
180
+ 170 62.94117647
181
+ 171 62.57309942
182
+ 172 62.79069767
183
+ 173 63.00578035
184
+ 174 63.21839080
185
+ 175 62.85714286
186
+ 176 62.50000000
187
+ 177 62.71186441
188
+ 178 62.35955056
189
+ 179 62.56983240
190
+ 180 62.22222222
191
+ 181 62.43093923
192
+ 182 62.63736264
193
+ 183 62.29508197
194
+ 184 61.95652174
195
+ 185 62.16216216
196
+ 186 62.36559140
197
+ 187 62.56684492
198
+ 188 62.23404255
199
+ 189 61.90476190
200
+ 190 61.57894737
201
+ 191 61.78010471
202
+ 192 61.45833333
203
+ 193 61.13989637
204
+ 194 61.34020619
205
+ 195 61.53846154
206
+ 196 61.22448980
207
+ 197 61.42131980
208
+ 198 61.11111111
209
+ 199 60.80402010
210
+ 200 61.00000000
211
+ 201 61.19402985
212
+ 202 61.38613861
213
+ 203 61.57635468
214
+ 204 61.76470588
215
+ 205 61.46341463
216
+ 206 61.16504854
217
+ 207 61.35265700
218
+ 208 61.53846154
219
+ 209 61.72248804
220
+ 210 61.90476190
221
+ 211 62.08530806
222
+ 212 61.79245283
223
+ 213 61.50234742
224
+ 214 61.68224299
225
+ 215 61.86046512
226
+ 216 61.57407407
227
+ 217 61.29032258
228
+ 218 61.00917431
229
+ 219 61.18721461
230
+ 220 60.90909091
231
+ 221 60.63348416
232
+ 222 60.81081081
233
+ 223 60.53811659
234
+ 224 60.71428571
235
+ 225 60.44444444
236
+ 226 60.17699115
237
+ 227 60.35242291
238
+ 228 60.52631579
239
+ 229 60.69868996
240
+ 230 60.86956522
241
+ 231 60.60606061
242
+ 232 60.34482759
243
+ 233 60.51502146
244
+ 234 60.68376068
245
+ 235 60.42553191
246
+ 236 60.59322034
247
+ 237 60.75949367
248
+ 238 60.92436975
249
+ 239 61.08786611
250
+ 240 61.25000000
251
+ 241 61.41078838
252
+ 242 61.15702479
253
+ 243 61.31687243
254
+ 244 61.06557377
255
+ 245 61.22448980
256
+ 246 61.38211382
257
+ 247 61.53846154
258
+ 248 61.69354839
259
+ 249 61.44578313
260
+ 250 61.60000000
261
+ 251 61.75298805
262
+ 252 61.90476190
263
+ 253 62.05533597
264
+ 254 61.81102362
265
+ 255 61.96078431
266
+ 256 62.10937500
267
+ 257 62.25680934
268
+ 258 62.40310078
269
+ 259 62.16216216
270
+ 260 62.30769231
271
+ 261 62.45210728
272
+ 262 62.59541985
273
+ 263 62.73764259
274
+ 264 62.87878788
275
+ 265 63.01886792
276
+ 266 62.78195489
277
+ 267 62.92134831
278
+ 268 63.05970149
279
+ 269 63.19702602
280
+ 270 63.33333333
281
+ 271 63.46863469
282
+ 272 63.23529412
283
+ 273 63.00366300
284
+ 274 62.77372263
285
+ 275 62.54545455
286
+ 276 62.68115942
287
+ 277 62.81588448
288
+ 278 62.58992806
289
+ 279 62.72401434
290
+ 280 62.85714286
291
+ 281 62.98932384
292
+ 282 63.12056738
293
+ 283 62.89752650
294
+ 284 63.02816901
295
+ 285 63.15789474
296
+ 286 63.28671329
297
+ 287 63.06620209
298
+ 288 63.19444444
299
+ 289 62.97577855
300
+ 290 63.10344828
301
+ 291 63.23024055
302
+ 292 63.01369863
303
+ 293 63.13993174
304
+ 294 62.92517007
305
+ 295 63.05084746
306
+ 296 63.17567568
307
+ 297 63.29966330
308
+ 298 63.42281879
309
+ 299 63.54515050
310
+ 300 63.33333333
311
+ 301 63.12292359
312
+ 302 63.24503311
313
+ 303 63.36633663
314
+ 304 63.48684211
315
+ 305 63.60655738
316
+ 306 63.39869281
317
+ 307 63.51791531
318
+ 308 63.63636364
319
+ 309 63.75404531
320
+ 310 63.54838710
321
+ 311 63.66559486
322
+ 312 63.46153846
323
+ 313 63.57827476
324
+ 314 63.37579618
325
+ 315 63.17460317
326
+ 316 63.29113924
327
+ 317 63.40694006
328
+ 318 63.20754717
329
+ 319 63.00940439
330
+ 320 62.81250000
331
+ 321 62.61682243
332
+ 322 62.73291925
333
+ 323 62.84829721
334
+ 324 62.65432099
335
+ 325 62.46153846
336
+ 326 62.57668712
337
+ 327 62.38532110
338
+ 328 62.50000000
339
+ 329 62.61398176
340
+ 330 62.72727273
341
+ 331 62.83987915
342
+ 332 62.95180723
343
+ 333 63.06306306
344
+ 334 62.87425150
345
+ 335 62.68656716
346
+ 336 62.79761905
347
+ 337 62.61127596
348
+ 338 62.42603550
349
+ 339 62.53687316
350
+ 340 62.64705882
351
+ 341 62.75659824
352
+ 342 62.86549708
353
+ 343 62.97376093
354
+ 344 63.08139535
355
+ 345 62.89855072
356
+ 346 63.00578035
357
+ 347 62.82420749
358
+ 348 62.64367816
359
+ 349 62.75071633
360
+ 350 62.57142857
361
+ 351 62.67806268
362
+ 352 62.78409091
363
+ 353 62.88951841
364
+ 354 62.71186441
365
+ 355 62.53521127
366
+ 356 62.35955056
367
+ 357 62.18487395
368
+ 358 62.29050279
369
+ 359 62.11699164
370
+ 360 62.22222222
371
+ 361 62.32686981
372
+ 362 62.43093923
373
+ 363 62.53443526
374
+ 364 62.36263736
375
+ 365 62.46575342
376
+ 366 62.56830601
377
+ 367 62.67029973
378
+ 368 62.77173913
379
+ 369 62.60162602
380
+ 370 62.43243243
381
+ 371 62.26415094
382
+ 372 62.09677419
383
+ 373 61.93029491
384
+ 374 62.03208556
385
+ 375 62.13333333
386
+ 376 62.23404255
387
+ 377 62.06896552
388
+ 378 62.16931217
389
+ 379 62.26912929
390
+ 380 62.36842105
391
+ 381 62.20472441
392
+ 382 62.04188482
393
+ 383 61.87989556
394
+ 384 61.97916667
395
+ 385 62.07792208
396
+ 386 62.17616580
397
+ 387 62.01550388
398
+ 388 61.85567010
399
+ 389 61.69665810
400
+ 390 61.53846154
401
+ 391 61.38107417
402
+ 392 61.47959184
403
+ 393 61.32315522
404
+ 394 61.42131980
405
+ 395 61.51898734
406
+ 396 61.36363636
407
+ 397 61.46095718
408
+ 398 61.30653266
409
+ 399 61.40350877
410
+ 400 61.50000000
411
+ 401 61.59600998
412
+ 402 61.44278607
413
+ 403 61.53846154
414
+ 404 61.63366337
415
+ 405 61.72839506
416
+ 406 61.82266010
417
+ 407 61.91646192
418
+ 408 62.00980392
419
+ 409 61.85819071
420
+ 410 61.95121951
421
+ 411 62.04379562
422
+ 412 62.13592233
423
+ 413 61.98547215
424
+ 414 62.07729469
425
+ 415 61.92771084
426
+ 416 61.77884615
427
+ 417 61.63069544
428
+ 418 61.72248804
429
+ 419 61.57517900
430
+ 420 61.42857143
431
+ 421 61.28266033
432
+ 422 61.37440758
433
+ 423 61.46572104
434
+ 424 61.32075472
435
+ 425 61.41176471
436
+ 426 61.26760563
437
+ 427 61.35831382
438
+ 428 61.21495327
439
+ 429 61.30536131
440
+ 430 61.16279070
441
+ 431 61.25290023
442
+ 432 61.34259259
443
+ 433 61.20092379
444
+ 434 61.29032258
445
+ 435 61.37931034
446
+ 436 61.46788991
447
+ 437 61.32723112
448
+ 438 61.18721461
449
+ 439 61.27562642
450
+ 440 61.13636364
451
+ 441 61.22448980
452
+ 442 61.08597285
453
+ 443 60.94808126
454
+ 444 61.03603604
455
+ 445 61.12359551
456
+ 446 61.21076233
457
+ 447 61.29753915
458
+ 448 61.38392857
459
+ 449 61.46993318
460
+ 450 61.55555556
461
+ 451 61.64079823
462
+ 452 61.72566372
463
+ 453 61.58940397
464
+ 454 61.45374449
465
+ 455 61.53846154
466
+ 456 61.40350877
467
+ 457 61.48796499
468
+ 458 61.57205240
469
+ 459 61.65577342
470
+ 460 61.52173913
471
+ 461 61.38828633
472
+ 462 61.47186147
473
+ 463 61.33909287
474
+ 464 61.20689655
475
+ 465 61.07526882
476
+ 466 61.15879828
477
+ 467 61.24197002
478
+ 468 61.32478632
479
+ 469 61.19402985
480
+ 470 61.27659574
481
+ 471 61.35881104
482
+ 472 61.44067797
483
+ 473 61.31078224
484
+ 474 61.39240506
485
+ 475 61.26315789
486
+ 476 61.13445378
487
+ 477 61.21593291
488
+ 478 61.29707113
489
+ 479 61.37787056
490
+ 480 61.45833333
491
+ 481 61.33056133
492
+ 482 61.41078838
493
+ 483 61.49068323
494
+ 484 61.57024793
495
+ 485 61.64948454
496
+ 486 61.72839506
497
+ 487 61.60164271
498
+ 488 61.68032787
499
+ 489 61.55419223
500
+ 490 61.42857143
501
+ 491 61.30346232
502
+ 492 61.38211382
503
+ 493 61.25760649
504
+ 494 61.33603239
505
+ 495 61.21212121
506
+ 496 61.29032258
507
+ 497 61.36820926
508
+ 498 61.44578313
509
+ 499 61.52304609
510
+ 500 61.60000000
511
+ 501 61.47704591
512
+ 502 61.55378486
513
+ 503 61.63021869
514
+ 504 61.50793651
515
+ 505 61.58415842
516
+ 506 61.46245059
517
+ 507 61.34122288
518
+ 508 61.22047244
519
+ 509 61.10019646
520
+ 510 61.17647059
521
+ 511 61.05675147
522
+ 512 61.13281250
523
+ 513 61.01364522
524
+ 514 61.08949416
525
+ 515 61.16504854
526
+ 516 61.04651163
527
+ 517 61.12185687
528
+ 518 61.19691120
529
+ 519 61.07899807
530
+ 520 60.96153846
531
+ 521 61.03646833
532
+ 522 60.91954023
533
+ 523 60.80305927
534
+ 524 60.68702290
535
+ 525 60.57142857
536
+ 526 60.45627376
537
+ 527 60.34155598
538
+ 528 60.22727273
539
+ 529 60.11342155
540
+ 530 60.18867925
541
+ 531 60.26365348
542
+ 532 60.33834586
543
+ 533 60.41275797
544
+ 534 60.29962547
545
+ 535 60.37383178
546
+ 536 60.44776119
547
+ 537 60.52141527
548
+ 538 60.40892193
549
+ 539 60.48237477
550
+ 540 60.37037037
551
+ 541 60.44362292
552
+ 542 60.33210332
553
+ 543 60.40515654
554
+ 544 60.29411765
555
+ 545 60.36697248
556
+ 546 60.25641026
557
+ 547 60.14625229
558
+ 548 60.21897810
559
+ 549 60.29143898
560
+ 550 60.36363636
561
+ 551 60.25408348
562
+ 552 60.32608696
563
+ 553 60.39783002
564
+ 554 60.46931408
565
+ 555 60.54054054
566
+ 556 60.43165468
567
+ 557 60.50269300
568
+ 558 60.39426523
569
+ 559 60.46511628
570
+ 560 60.53571429
571
+ 561 60.60606061
572
+ 562 60.67615658
573
+ 563 60.74600355
574
+ 564 60.63829787
575
+ 565 60.53097345
576
+ 566 60.60070671
577
+ 567 60.49382716
578
+ 568 60.56338028
579
+ 569 60.45694200
580
+ 570 60.35087719
581
+ 571 60.42031524
582
+ 572 60.31468531
583
+ 573 60.20942408
584
+ 574 60.27874564
585
+ 575 60.34782609
586
+ 576 60.41666667
587
+ 577 60.48526863
588
+ 578 60.55363322
589
+ 579 60.62176166
590
+ 580 60.51724138
591
+ 581 60.41308090
592
+ 582 60.48109966
593
+ 583 60.54888508
594
+ 584 60.61643836
595
+ 585 60.68376068
596
+ 586 60.75085324
597
+ 587 60.64735945
598
+ 588 60.54421769
599
+ 589 60.61120543
600
+ 590 60.67796610
601
+ 591 60.74450085
602
+ 592 60.64189189
603
+ 593 60.70826307
604
+ 594 60.77441077
605
+ 595 60.67226891
606
+ 596 60.57046980
607
+ 597 60.46901173
608
+ 598 60.36789298
609
+ 599 60.43405676
610
+ 600 60.50000000
611
+ 601 60.56572379
612
+ 602 60.46511628
613
+ 603 60.36484245
614
+ 604 60.26490066
615
+ 605 60.16528926
616
+ 606 60.23102310
617
+ 607 60.29654036
618
+ 608 60.36184211
619
+ 609 60.26272578
620
+ 610 60.32786885
621
+ 611 60.22913257
622
+ 612 60.13071895
623
+ 613 60.03262643
624
+ 614 59.93485342
625
+ 615 59.83739837
626
+ 616 59.74025974
627
+ 617 59.80551053
628
+ 618 59.70873786
629
+ 619 59.61227787
630
+ 620 59.67741935
631
+ 621 59.74235105
632
+ 622 59.64630225
633
+ 623 59.71107544
634
+ 624 59.61538462
635
+ 625 59.52000000
636
+ 626 59.42492013
637
+ 627 59.48963317
638
+ 628 59.39490446
639
+ 629 59.30047695
640
+ 630 59.36507937
641
+ 631 59.27099842
642
+ 632 59.17721519
643
+ 633 59.08372828
644
+ 634 59.14826498
645
+ 635 59.21259843
646
+ 636 59.27672956
647
+ 637 59.18367347
648
+ 638 59.24764890
649
+ 639 59.15492958
650
+ 640 59.06250000
651
+ 641 58.97035881
652
+ 642 58.87850467
653
+ 643 58.78693624
654
+ 644 58.85093168
655
+ 645 58.75968992
656
+ 646 58.82352941
657
+ 647 58.88717156
658
+ 648 58.79629630
659
+ 649 58.70570108
660
+ 650 58.61538462
661
+ 651 58.67895545
662
+ 652 58.58895706
663
+ 653 58.65237366
664
+ 654 58.71559633
665
+ 655 58.62595420
666
+ 656 58.68902439
667
+ 657 58.59969559
668
+ 658 58.51063830
669
+ 659 58.57359636
670
+ 660 58.63636364
671
+ 661 58.54765507
672
+ 662 58.61027190
673
+ 663 58.67269985
674
+ 664 58.73493976
675
+ 665 58.79699248
676
+ 666 58.70870871
677
+ 667 58.77061469
678
+ 668 58.68263473
679
+ 669 58.59491779
680
+ 670 58.65671642
681
+ 671 58.71833085
682
+ 672 58.63095238
683
+ 673 58.54383358
684
+ 674 58.60534125
685
+ 675 58.66666667
686
+ 676 58.57988166
687
+ 677 58.64106352
688
+ 678 58.70206490
689
+ 679 58.76288660
690
+ 680 58.82352941
691
+ 681 58.73715125
692
+ 682 58.65102639
693
+ 683 58.56515373
694
+ 684 58.47953216
695
+ 685 58.54014599
696
+ 686 58.45481050
697
+ 687 58.51528384
698
+ 688 58.57558140
699
+ 689 58.63570392
700
+ 690 58.69565217
701
+ 691 58.75542692
702
+ 692 58.81502890
703
+ 693 58.87445887
704
+ 694 58.93371758
705
+ 695 58.84892086
706
+ 696 58.76436782
707
+ 697 58.68005739
708
+ 698 58.59598854
709
+ 699 58.51216023
710
+ 700 58.57142857
711
+ 701 58.48787447
712
+ 702 58.54700855
713
+ 703 58.60597440
714
+ 704 58.66477273
715
+ 705 58.72340426
716
+ 706 58.78186969
717
+ 707 58.84016973
718
+ 708 58.89830508
719
+ 709 58.95627645
720
+ 710 59.01408451
721
+ 711 59.07172996
722
+ 712 59.12921348
723
+ 713 59.18653576
724
+ 714 59.10364146
725
+ 715 59.02097902
726
+ 716 59.07821229
727
+ 717 58.99581590
728
+ 718 58.91364903
729
+ 719 58.83171071
730
+ 720 58.88888889
731
+ 721 58.80721221
732
+ 722 58.86426593
733
+ 723 58.92116183
734
+ 724 58.97790055
735
+ 725 59.03448276
736
+ 726 58.95316804
737
+ 727 58.87207703
738
+ 728 58.92857143
739
+ 729 58.98491084
740
+ 730 59.04109589
741
+ 731 59.09712722
742
+ 732 59.15300546
743
+ 733 59.20873124
744
+ 734 59.26430518
745
+ 735 59.18367347
746
+ 736 59.23913043
747
+ 737 59.15875170
748
+ 738 59.07859079
749
+ 739 58.99864682
750
+ 740 58.91891892
751
+ 741 58.97435897
752
+ 742 59.02964960
753
+ 743 59.08479139
754
+ 744 59.00537634
755
+ 745 59.06040268
756
+ 746 58.98123324
757
+ 747 58.90227577
758
+ 748 58.82352941
759
+ 749 58.87850467
760
+ 750 58.93333333
761
+
762
+ Final result: 58.9333 +/- 1.7976
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-F16.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 77.77777778
20
+ 10 80.00000000
21
+ 11 81.81818182
22
+ 12 75.00000000
23
+ 13 76.92307692
24
+ 14 78.57142857
25
+ 15 80.00000000
26
+ 16 81.25000000
27
+ 17 76.47058824
28
+ 18 72.22222222
29
+ 19 68.42105263
30
+ 20 70.00000000
31
+ 21 71.42857143
32
+ 22 72.72727273
33
+ 23 73.91304348
34
+ 24 70.83333333
35
+ 25 68.00000000
36
+ 26 65.38461538
37
+ 27 62.96296296
38
+ 28 64.28571429
39
+ 29 65.51724138
40
+ 30 63.33333333
41
+ 31 64.51612903
42
+ 32 62.50000000
43
+ 33 63.63636364
44
+ 34 64.70588235
45
+ 35 65.71428571
46
+ 36 63.88888889
47
+ 37 62.16216216
48
+ 38 63.15789474
49
+ 39 64.10256410
50
+ 40 65.00000000
51
+ 41 65.85365854
52
+ 42 64.28571429
53
+ 43 62.79069767
54
+ 44 63.63636364
55
+ 45 64.44444444
56
+ 46 63.04347826
57
+ 47 61.70212766
58
+ 48 60.41666667
59
+ 49 61.22448980
60
+ 50 60.00000000
61
+ 51 60.78431373
62
+ 52 59.61538462
63
+ 53 58.49056604
64
+ 54 57.40740741
65
+ 55 58.18181818
66
+ 56 57.14285714
67
+ 57 57.89473684
68
+ 58 56.89655172
69
+ 59 57.62711864
70
+ 60 58.33333333
71
+ 61 57.37704918
72
+ 62 56.45161290
73
+ 63 57.14285714
74
+ 64 57.81250000
75
+ 65 56.92307692
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 57.97101449
80
+ 70 58.57142857
81
+ 71 57.74647887
82
+ 72 58.33333333
83
+ 73 58.90410959
84
+ 74 59.45945946
85
+ 75 58.66666667
86
+ 76 59.21052632
87
+ 77 59.74025974
88
+ 78 60.25641026
89
+ 79 60.75949367
90
+ 80 61.25000000
91
+ 81 61.72839506
92
+ 82 62.19512195
93
+ 83 62.65060241
94
+ 84 63.09523810
95
+ 85 62.35294118
96
+ 86 62.79069767
97
+ 87 63.21839080
98
+ 88 63.63636364
99
+ 89 62.92134831
100
+ 90 63.33333333
101
+ 91 63.73626374
102
+ 92 63.04347826
103
+ 93 63.44086022
104
+ 94 63.82978723
105
+ 95 64.21052632
106
+ 96 64.58333333
107
+ 97 64.94845361
108
+ 98 65.30612245
109
+ 99 65.65656566
110
+ 100 65.00000000
111
+ 101 65.34653465
112
+ 102 64.70588235
113
+ 103 64.07766990
114
+ 104 64.42307692
115
+ 105 64.76190476
116
+ 106 65.09433962
117
+ 107 65.42056075
118
+ 108 65.74074074
119
+ 109 65.13761468
120
+ 110 65.45454545
121
+ 111 64.86486486
122
+ 112 65.17857143
123
+ 113 65.48672566
124
+ 114 65.78947368
125
+ 115 66.08695652
126
+ 116 65.51724138
127
+ 117 64.95726496
128
+ 118 65.25423729
129
+ 119 65.54621849
130
+ 120 65.00000000
131
+ 121 64.46280992
132
+ 122 64.75409836
133
+ 123 65.04065041
134
+ 124 64.51612903
135
+ 125 64.80000000
136
+ 126 65.07936508
137
+ 127 65.35433071
138
+ 128 64.84375000
139
+ 129 65.11627907
140
+ 130 64.61538462
141
+ 131 64.88549618
142
+ 132 64.39393939
143
+ 133 64.66165414
144
+ 134 64.92537313
145
+ 135 65.18518519
146
+ 136 65.44117647
147
+ 137 65.69343066
148
+ 138 65.94202899
149
+ 139 66.18705036
150
+ 140 66.42857143
151
+ 141 66.66666667
152
+ 142 66.90140845
153
+ 143 67.13286713
154
+ 144 67.36111111
155
+ 145 67.58620690
156
+ 146 67.80821918
157
+ 147 68.02721088
158
+ 148 68.24324324
159
+ 149 68.45637584
160
+ 150 68.66666667
161
+ 151 68.21192053
162
+ 152 67.76315789
163
+ 153 67.97385621
164
+ 154 68.18181818
165
+ 155 68.38709677
166
+ 156 67.94871795
167
+ 157 67.51592357
168
+ 158 67.72151899
169
+ 159 67.29559748
170
+ 160 67.50000000
171
+ 161 67.70186335
172
+ 162 67.90123457
173
+ 163 68.09815951
174
+ 164 68.29268293
175
+ 165 68.48484848
176
+ 166 68.67469880
177
+ 167 68.86227545
178
+ 168 69.04761905
179
+ 169 68.63905325
180
+ 170 68.82352941
181
+ 171 69.00584795
182
+ 172 68.60465116
183
+ 173 68.20809249
184
+ 174 67.81609195
185
+ 175 68.00000000
186
+ 176 68.18181818
187
+ 177 68.36158192
188
+ 178 68.53932584
189
+ 179 68.15642458
190
+ 180 68.33333333
191
+ 181 67.95580110
192
+ 182 68.13186813
193
+ 183 68.30601093
194
+ 184 67.93478261
195
+ 185 68.10810811
196
+ 186 67.74193548
197
+ 187 67.37967914
198
+ 188 67.02127660
199
+ 189 66.66666667
200
+ 190 66.84210526
201
+ 191 67.01570681
202
+ 192 67.18750000
203
+ 193 66.83937824
204
+ 194 67.01030928
205
+ 195 66.66666667
206
+ 196 66.83673469
207
+ 197 66.49746193
208
+ 198 66.16161616
209
+ 199 66.33165829
210
+ 200 66.50000000
211
+ 201 66.66666667
212
+ 202 66.83168317
213
+ 203 66.50246305
214
+ 204 66.66666667
215
+ 205 66.34146341
216
+ 206 66.50485437
217
+ 207 66.66666667
218
+ 208 66.34615385
219
+ 209 66.50717703
220
+ 210 66.66666667
221
+ 211 66.82464455
222
+ 212 66.98113208
223
+ 213 67.13615023
224
+ 214 67.28971963
225
+ 215 66.97674419
226
+ 216 67.12962963
227
+ 217 66.82027650
228
+ 218 66.97247706
229
+ 219 67.12328767
230
+ 220 67.27272727
231
+ 221 66.96832579
232
+ 222 67.11711712
233
+ 223 67.26457399
234
+ 224 67.41071429
235
+ 225 67.55555556
236
+ 226 67.69911504
237
+ 227 67.40088106
238
+ 228 67.54385965
239
+ 229 67.68558952
240
+ 230 67.39130435
241
+ 231 67.09956710
242
+ 232 66.81034483
243
+ 233 66.52360515
244
+ 234 66.66666667
245
+ 235 66.38297872
246
+ 236 66.10169492
247
+ 237 66.24472574
248
+ 238 66.38655462
249
+ 239 66.52719665
250
+ 240 66.66666667
251
+ 241 66.80497925
252
+ 242 66.94214876
253
+ 243 67.07818930
254
+ 244 66.80327869
255
+ 245 66.53061224
256
+ 246 66.66666667
257
+ 247 66.80161943
258
+ 248 66.93548387
259
+ 249 67.06827309
260
+ 250 67.20000000
261
+ 251 67.33067729
262
+ 252 67.46031746
263
+ 253 67.19367589
264
+ 254 67.32283465
265
+ 255 67.45098039
266
+ 256 67.57812500
267
+ 257 67.31517510
268
+ 258 67.44186047
269
+ 259 67.56756757
270
+ 260 67.69230769
271
+ 261 67.81609195
272
+ 262 67.55725191
273
+ 263 67.68060837
274
+ 264 67.80303030
275
+ 265 67.92452830
276
+ 266 67.66917293
277
+ 267 67.79026217
278
+ 268 67.91044776
279
+ 269 68.02973978
280
+ 270 68.14814815
281
+ 271 67.89667897
282
+ 272 68.01470588
283
+ 273 67.76556777
284
+ 274 67.88321168
285
+ 275 67.63636364
286
+ 276 67.75362319
287
+ 277 67.87003610
288
+ 278 67.98561151
289
+ 279 68.10035842
290
+ 280 68.21428571
291
+ 281 68.32740214
292
+ 282 68.43971631
293
+ 283 68.19787986
294
+ 284 68.30985915
295
+ 285 68.07017544
296
+ 286 68.18181818
297
+ 287 68.29268293
298
+ 288 68.40277778
299
+ 289 68.51211073
300
+ 290 68.62068966
301
+ 291 68.72852234
302
+ 292 68.49315068
303
+ 293 68.60068259
304
+ 294 68.70748299
305
+ 295 68.81355932
306
+ 296 68.91891892
307
+ 297 69.02356902
308
+ 298 69.12751678
309
+ 299 68.89632107
310
+ 300 68.66666667
311
+ 301 68.43853821
312
+ 302 68.54304636
313
+ 303 68.31683168
314
+ 304 68.09210526
315
+ 305 68.19672131
316
+ 306 68.30065359
317
+ 307 68.07817590
318
+ 308 68.18181818
319
+ 309 68.28478964
320
+ 310 68.38709677
321
+ 311 68.48874598
322
+ 312 68.58974359
323
+ 313 68.69009585
324
+ 314 68.47133758
325
+ 315 68.57142857
326
+ 316 68.67088608
327
+ 317 68.76971609
328
+ 318 68.55345912
329
+ 319 68.33855799
330
+ 320 68.43750000
331
+ 321 68.22429907
332
+ 322 68.01242236
333
+ 323 68.11145511
334
+ 324 68.20987654
335
+ 325 68.30769231
336
+ 326 68.09815951
337
+ 327 67.88990826
338
+ 328 67.98780488
339
+ 329 68.08510638
340
+ 330 68.18181818
341
+ 331 68.27794562
342
+ 332 68.07228916
343
+ 333 68.16816817
344
+ 334 67.96407186
345
+ 335 68.05970149
346
+ 336 68.15476190
347
+ 337 67.95252226
348
+ 338 67.75147929
349
+ 339 67.84660767
350
+ 340 67.94117647
351
+ 341 68.03519062
352
+ 342 68.12865497
353
+ 343 68.22157434
354
+ 344 68.02325581
355
+ 345 68.11594203
356
+ 346 67.91907514
357
+ 347 68.01152738
358
+ 348 67.81609195
359
+ 349 67.62177650
360
+ 350 67.71428571
361
+ 351 67.80626781
362
+ 352 67.89772727
363
+ 353 67.98866856
364
+ 354 68.07909605
365
+ 355 68.16901408
366
+ 356 68.25842697
367
+ 357 68.34733894
368
+ 358 68.43575419
369
+ 359 68.52367688
370
+ 360 68.33333333
371
+ 361 68.42105263
372
+ 362 68.50828729
373
+ 363 68.59504132
374
+ 364 68.68131868
375
+ 365 68.76712329
376
+ 366 68.85245902
377
+ 367 68.93732970
378
+ 368 69.02173913
379
+ 369 69.10569106
380
+ 370 69.18918919
381
+ 371 69.00269542
382
+ 372 69.08602151
383
+ 373 68.90080429
384
+ 374 68.98395722
385
+ 375 69.06666667
386
+ 376 69.14893617
387
+ 377 68.96551724
388
+ 378 69.04761905
389
+ 379 69.12928760
390
+ 380 69.21052632
391
+ 381 69.29133858
392
+ 382 69.10994764
393
+ 383 69.19060052
394
+ 384 69.27083333
395
+ 385 69.35064935
396
+ 386 69.43005181
397
+ 387 69.50904393
398
+ 388 69.32989691
399
+ 389 69.40874036
400
+ 390 69.48717949
401
+ 391 69.30946292
402
+ 392 69.38775510
403
+ 393 69.46564885
404
+ 394 69.54314721
405
+ 395 69.62025316
406
+ 396 69.69696970
407
+ 397 69.77329975
408
+ 398 69.84924623
409
+ 399 69.92481203
410
+ 400 70.00000000
411
+ 401 70.07481297
412
+ 402 70.14925373
413
+ 403 69.97518610
414
+ 404 70.04950495
415
+ 405 69.87654321
416
+ 406 69.95073892
417
+ 407 70.02457002
418
+ 408 70.09803922
419
+ 409 70.17114914
420
+ 410 70.00000000
421
+ 411 69.82968370
422
+ 412 69.66019417
423
+ 413 69.73365617
424
+ 414 69.80676329
425
+ 415 69.63855422
426
+ 416 69.71153846
427
+ 417 69.78417266
428
+ 418 69.85645933
429
+ 419 69.92840095
430
+ 420 70.00000000
431
+ 421 69.83372922
432
+ 422 69.90521327
433
+ 423 69.97635934
434
+ 424 70.04716981
435
+ 425 70.11764706
436
+ 426 70.18779343
437
+ 427 70.02341920
438
+ 428 70.09345794
439
+ 429 69.93006993
440
+ 430 70.00000000
441
+ 431 70.06960557
442
+ 432 70.13888889
443
+ 433 70.20785219
444
+ 434 70.27649770
445
+ 435 70.11494253
446
+ 436 70.18348624
447
+ 437 70.02288330
448
+ 438 69.86301370
449
+ 439 69.93166287
450
+ 440 70.00000000
451
+ 441 70.06802721
452
+ 442 70.13574661
453
+ 443 70.20316027
454
+ 444 70.27027027
455
+ 445 70.33707865
456
+ 446 70.40358744
457
+ 447 70.46979866
458
+ 448 70.53571429
459
+ 449 70.60133630
460
+ 450 70.66666667
461
+ 451 70.50997783
462
+ 452 70.57522124
463
+ 453 70.41942605
464
+ 454 70.48458150
465
+ 455 70.32967033
466
+ 456 70.39473684
467
+ 457 70.45951860
468
+ 458 70.52401747
469
+ 459 70.58823529
470
+ 460 70.65217391
471
+ 461 70.71583514
472
+ 462 70.77922078
473
+ 463 70.84233261
474
+ 464 70.90517241
475
+ 465 70.75268817
476
+ 466 70.81545064
477
+ 467 70.87794433
478
+ 468 70.72649573
479
+ 469 70.57569296
480
+ 470 70.42553191
481
+ 471 70.27600849
482
+ 472 70.33898305
483
+ 473 70.40169133
484
+ 474 70.46413502
485
+ 475 70.52631579
486
+ 476 70.37815126
487
+ 477 70.23060797
488
+ 478 70.29288703
489
+ 479 70.35490605
490
+ 480 70.41666667
491
+ 481 70.47817048
492
+ 482 70.33195021
493
+ 483 70.39337474
494
+ 484 70.24793388
495
+ 485 70.10309278
496
+ 486 70.16460905
497
+ 487 70.22587269
498
+ 488 70.08196721
499
+ 489 70.14314928
500
+ 490 70.20408163
501
+ 491 70.26476578
502
+ 492 70.32520325
503
+ 493 70.38539554
504
+ 494 70.44534413
505
+ 495 70.50505051
506
+ 496 70.56451613
507
+ 497 70.62374245
508
+ 498 70.68273092
509
+ 499 70.54108216
510
+ 500 70.60000000
511
+ 501 70.65868263
512
+ 502 70.71713147
513
+ 503 70.77534791
514
+ 504 70.83333333
515
+ 505 70.89108911
516
+ 506 70.94861660
517
+ 507 71.00591716
518
+ 508 71.06299213
519
+ 509 71.11984283
520
+ 510 71.17647059
521
+ 511 71.23287671
522
+ 512 71.09375000
523
+ 513 70.95516569
524
+ 514 71.01167315
525
+ 515 71.06796117
526
+ 516 70.93023256
527
+ 517 70.79303675
528
+ 518 70.84942085
529
+ 519 70.71290944
530
+ 520 70.76923077
531
+ 521 70.63339731
532
+ 522 70.49808429
533
+ 523 70.55449331
534
+ 524 70.61068702
535
+ 525 70.66666667
536
+ 526 70.53231939
537
+ 527 70.39848197
538
+ 528 70.45454545
539
+ 529 70.51039698
540
+ 530 70.56603774
541
+ 531 70.62146893
542
+ 532 70.67669173
543
+ 533 70.54409006
544
+ 534 70.59925094
545
+ 535 70.65420561
546
+ 536 70.70895522
547
+ 537 70.76350093
548
+ 538 70.81784387
549
+ 539 70.68645640
550
+ 540 70.74074074
551
+ 541 70.79482440
552
+ 542 70.84870849
553
+ 543 70.90239411
554
+ 544 70.95588235
555
+ 545 71.00917431
556
+ 546 71.06227106
557
+ 547 71.11517367
558
+ 548 71.16788321
559
+ 549 71.22040073
560
+ 550 71.09090909
561
+ 551 71.14337568
562
+ 552 71.01449275
563
+ 553 71.06690778
564
+ 554 71.11913357
565
+ 555 70.99099099
566
+ 556 71.04316547
567
+ 557 70.91561939
568
+ 558 70.78853047
569
+ 559 70.66189624
570
+ 560 70.71428571
571
+ 561 70.76648841
572
+ 562 70.64056940
573
+ 563 70.69271758
574
+ 564 70.74468085
575
+ 565 70.61946903
576
+ 566 70.49469965
577
+ 567 70.54673721
578
+ 568 70.59859155
579
+ 569 70.47451670
580
+ 570 70.52631579
581
+ 571 70.57793345
582
+ 572 70.62937063
583
+ 573 70.68062827
584
+ 574 70.73170732
585
+ 575 70.78260870
586
+ 576 70.83333333
587
+ 577 70.71057192
588
+ 578 70.76124567
589
+ 579 70.81174439
590
+ 580 70.68965517
591
+ 581 70.56798623
592
+ 582 70.61855670
593
+ 583 70.66895369
594
+ 584 70.71917808
595
+ 585 70.76923077
596
+ 586 70.81911263
597
+ 587 70.86882453
598
+ 588 70.74829932
599
+ 589 70.62818336
600
+ 590 70.50847458
601
+ 591 70.55837563
602
+ 592 70.60810811
603
+ 593 70.48903879
604
+ 594 70.37037037
605
+ 595 70.42016807
606
+ 596 70.46979866
607
+ 597 70.51926298
608
+ 598 70.56856187
609
+ 599 70.45075125
610
+ 600 70.50000000
611
+ 601 70.54908486
612
+ 602 70.59800664
613
+ 603 70.48092869
614
+ 604 70.52980132
615
+ 605 70.57851240
616
+ 606 70.46204620
617
+ 607 70.51070840
618
+ 608 70.39473684
619
+ 609 70.44334975
620
+ 610 70.49180328
621
+ 611 70.54009820
622
+ 612 70.42483660
623
+ 613 70.47308320
624
+ 614 70.35830619
625
+ 615 70.40650407
626
+ 616 70.29220779
627
+ 617 70.34035656
628
+ 618 70.38834951
629
+ 619 70.27463651
630
+ 620 70.32258065
631
+ 621 70.20933977
632
+ 622 70.25723473
633
+ 623 70.30497592
634
+ 624 70.35256410
635
+ 625 70.40000000
636
+ 626 70.28753994
637
+ 627 70.33492823
638
+ 628 70.38216561
639
+ 629 70.27027027
640
+ 630 70.15873016
641
+ 631 70.20602219
642
+ 632 70.09493671
643
+ 633 70.14218009
644
+ 634 70.18927445
645
+ 635 70.23622047
646
+ 636 70.28301887
647
+ 637 70.32967033
648
+ 638 70.37617555
649
+ 639 70.42253521
650
+ 640 70.31250000
651
+ 641 70.35881435
652
+ 642 70.24922118
653
+ 643 70.29548989
654
+ 644 70.34161491
655
+ 645 70.23255814
656
+ 646 70.27863777
657
+ 647 70.32457496
658
+ 648 70.37037037
659
+ 649 70.26194145
660
+ 650 70.30769231
661
+ 651 70.35330261
662
+ 652 70.39877301
663
+ 653 70.44410413
664
+ 654 70.48929664
665
+ 655 70.53435115
666
+ 656 70.57926829
667
+ 657 70.62404871
668
+ 658 70.51671733
669
+ 659 70.40971168
670
+ 660 70.45454545
671
+ 661 70.49924357
672
+ 662 70.54380665
673
+ 663 70.58823529
674
+ 664 70.63253012
675
+ 665 70.67669173
676
+ 666 70.72072072
677
+ 667 70.76461769
678
+ 668 70.80838323
679
+ 669 70.85201794
680
+ 670 70.89552239
681
+ 671 70.93889717
682
+ 672 70.98214286
683
+ 673 71.02526003
684
+ 674 71.06824926
685
+ 675 71.11111111
686
+ 676 71.15384615
687
+ 677 71.19645495
688
+ 678 71.23893805
689
+ 679 71.28129602
690
+ 680 71.32352941
691
+ 681 71.21879589
692
+ 682 71.26099707
693
+ 683 71.30307467
694
+ 684 71.34502924
695
+ 685 71.24087591
696
+ 686 71.28279883
697
+ 687 71.32459971
698
+ 688 71.36627907
699
+ 689 71.40783745
700
+ 690 71.44927536
701
+ 691 71.49059334
702
+ 692 71.53179191
703
+ 693 71.42857143
704
+ 694 71.46974063
705
+ 695 71.51079137
706
+ 696 71.55172414
707
+ 697 71.59253945
708
+ 698 71.63323782
709
+ 699 71.67381974
710
+ 700 71.71428571
711
+ 701 71.75463623
712
+ 702 71.79487179
713
+ 703 71.83499289
714
+ 704 71.87500000
715
+ 705 71.91489362
716
+ 706 71.95467422
717
+ 707 71.99434229
718
+ 708 72.03389831
719
+ 709 72.07334274
720
+ 710 72.11267606
721
+ 711 72.01125176
722
+ 712 72.05056180
723
+ 713 72.08976157
724
+ 714 72.12885154
725
+ 715 72.02797203
726
+ 716 72.06703911
727
+ 717 72.10599721
728
+ 718 72.14484680
729
+ 719 72.04450626
730
+ 720 72.08333333
731
+ 721 71.98335645
732
+ 722 72.02216066
733
+ 723 72.06085754
734
+ 724 72.09944751
735
+ 725 72.13793103
736
+ 726 72.17630854
737
+ 727 72.21458047
738
+ 728 72.25274725
739
+ 729 72.29080933
740
+ 730 72.19178082
741
+ 731 72.22982216
742
+ 732 72.13114754
743
+ 733 72.16916780
744
+ 734 72.20708447
745
+ 735 72.24489796
746
+ 736 72.28260870
747
+ 737 72.18453189
748
+ 738 72.08672087
749
+ 739 71.98917456
750
+ 740 72.02702703
751
+ 741 72.06477733
752
+ 742 72.10242588
753
+ 743 72.13997308
754
+ 744 72.17741935
755
+ 745 72.08053691
756
+ 746 72.11796247
757
+ 747 72.02141901
758
+ 748 72.05882353
759
+ 749 72.09612817
760
+ 750 72.13333333
scores/Hammer2.1-7b-F16.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 0.00000000
13
+ 3 0.00000000
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 28.57142857
18
+ 8 37.50000000
19
+ 9 33.33333333
20
+ 10 30.00000000
21
+ 11 27.27272727
22
+ 12 33.33333333
23
+ 13 38.46153846
24
+ 14 35.71428571
25
+ 15 33.33333333
26
+ 16 37.50000000
27
+ 17 41.17647059
28
+ 18 38.88888889
29
+ 19 42.10526316
30
+ 20 45.00000000
31
+ 21 47.61904762
32
+ 22 45.45454545
33
+ 23 43.47826087
34
+ 24 41.66666667
35
+ 25 44.00000000
36
+ 26 42.30769231
37
+ 27 44.44444444
38
+ 28 46.42857143
39
+ 29 48.27586207
40
+ 30 50.00000000
41
+ 31 51.61290323
42
+ 32 53.12500000
43
+ 33 51.51515152
44
+ 34 52.94117647
45
+ 35 54.28571429
46
+ 36 52.77777778
47
+ 37 54.05405405
48
+ 38 55.26315789
49
+ 39 53.84615385
50
+ 40 52.50000000
51
+ 41 53.65853659
52
+ 42 52.38095238
53
+ 43 53.48837209
54
+ 44 52.27272727
55
+ 45 53.33333333
56
+ 46 52.17391304
57
+ 47 51.06382979
58
+ 48 52.08333333
59
+ 49 53.06122449
60
+ 50 52.00000000
61
+ 51 50.98039216
62
+ 52 50.00000000
63
+ 53 49.05660377
64
+ 54 48.14814815
65
+ 55 49.09090909
66
+ 56 50.00000000
67
+ 57 49.12280702
68
+ 58 50.00000000
69
+ 59 50.84745763
70
+ 60 50.00000000
71
+ 61 49.18032787
72
+ 62 48.38709677
73
+ 63 47.61904762
74
+ 64 46.87500000
75
+ 65 46.15384615
76
+ 66 46.96969697
77
+ 67 46.26865672
78
+ 68 45.58823529
79
+ 69 44.92753623
80
+ 70 45.71428571
81
+ 71 46.47887324
82
+ 72 45.83333333
83
+ 73 45.20547945
84
+ 74 45.94594595
85
+ 75 46.66666667
86
+ 76 47.36842105
87
+ 77 48.05194805
88
+ 78 48.71794872
89
+ 79 48.10126582
90
+ 80 48.75000000
91
+ 81 49.38271605
92
+ 82 48.78048780
93
+ 83 49.39759036
94
+ 84 50.00000000
95
+ 85 50.58823529
96
+ 86 50.00000000
97
+ 87 50.57471264
98
+ 88 51.13636364
99
+ 89 50.56179775
100
+ 90 50.00000000
101
+ 91 49.45054945
102
+ 92 48.91304348
103
+ 93 48.38709677
104
+ 94 48.93617021
105
+ 95 48.42105263
106
+ 96 48.95833333
107
+ 97 48.45360825
108
+ 98 47.95918367
109
+ 99 47.47474747
110
+ 100 47.00000000
111
+ 101 47.52475248
112
+ 102 47.05882353
113
+ 103 46.60194175
114
+ 104 47.11538462
115
+ 105 47.61904762
116
+ 106 48.11320755
117
+ 107 47.66355140
118
+ 108 48.14814815
119
+ 109 47.70642202
120
+ 110 47.27272727
121
+ 111 46.84684685
122
+ 112 47.32142857
123
+ 113 46.90265487
124
+ 114 47.36842105
125
+ 115 47.82608696
126
+ 116 48.27586207
127
+ 117 47.86324786
128
+ 118 47.45762712
129
+ 119 47.89915966
130
+ 120 47.50000000
131
+ 121 47.93388430
132
+ 122 48.36065574
133
+ 123 47.96747967
134
+ 124 47.58064516
135
+ 125 48.00000000
136
+ 126 47.61904762
137
+ 127 47.24409449
138
+ 128 46.87500000
139
+ 129 46.51162791
140
+ 130 46.15384615
141
+ 131 46.56488550
142
+ 132 46.21212121
143
+ 133 45.86466165
144
+ 134 45.52238806
145
+ 135 45.18518519
146
+ 136 45.58823529
147
+ 137 45.98540146
148
+ 138 45.65217391
149
+ 139 46.04316547
150
+ 140 46.42857143
151
+ 141 46.09929078
152
+ 142 46.47887324
153
+ 143 46.85314685
154
+ 144 46.52777778
155
+ 145 46.89655172
156
+ 146 47.26027397
157
+ 147 46.93877551
158
+ 148 47.29729730
159
+ 149 47.65100671
160
+ 150 47.33333333
161
+ 151 47.01986755
162
+ 152 47.36842105
163
+ 153 47.05882353
164
+ 154 46.75324675
165
+ 155 46.45161290
166
+ 156 46.15384615
167
+ 157 46.49681529
168
+ 158 46.83544304
169
+ 159 46.54088050
170
+ 160 46.87500000
171
+ 161 46.58385093
172
+ 162 46.29629630
173
+ 163 46.01226994
174
+ 164 45.73170732
175
+ 165 46.06060606
176
+ 166 45.78313253
177
+ 167 46.10778443
178
+ 168 45.83333333
179
+ 169 46.15384615
180
+ 170 45.88235294
181
+ 171 45.61403509
182
+ 172 45.34883721
183
+ 173 45.08670520
184
+ 174 45.40229885
185
+ 175 45.14285714
186
+ 176 44.88636364
187
+ 177 44.63276836
188
+ 178 44.38202247
189
+ 179 44.69273743
190
+ 180 44.44444444
191
+ 181 44.19889503
192
+ 182 43.95604396
193
+ 183 44.26229508
194
+ 184 44.02173913
195
+ 185 44.32432432
196
+ 186 44.62365591
197
+ 187 44.38502674
198
+ 188 44.14893617
199
+ 189 44.44444444
200
+ 190 44.21052632
201
+ 191 43.97905759
202
+ 192 43.75000000
203
+ 193 43.52331606
204
+ 194 43.29896907
205
+ 195 43.58974359
206
+ 196 43.36734694
207
+ 197 43.14720812
208
+ 198 42.92929293
209
+ 199 42.71356784
210
+ 200 42.50000000
211
+ 201 42.28855721
212
+ 202 42.07920792
213
+ 203 41.87192118
214
+ 204 41.66666667
215
+ 205 41.95121951
216
+ 206 41.74757282
217
+ 207 42.02898551
218
+ 208 41.82692308
219
+ 209 41.62679426
220
+ 210 41.42857143
221
+ 211 41.70616114
222
+ 212 41.50943396
223
+ 213 41.31455399
224
+ 214 41.12149533
225
+ 215 41.39534884
226
+ 216 41.66666667
227
+ 217 41.47465438
228
+ 218 41.74311927
229
+ 219 42.00913242
230
+ 220 42.27272727
231
+ 221 42.53393665
232
+ 222 42.34234234
233
+ 223 42.60089686
234
+ 224 42.85714286
235
+ 225 43.11111111
236
+ 226 43.36283186
237
+ 227 43.17180617
238
+ 228 43.42105263
239
+ 229 43.23144105
240
+ 230 43.04347826
241
+ 231 42.85714286
242
+ 232 43.10344828
243
+ 233 42.91845494
244
+ 234 43.16239316
245
+ 235 42.97872340
246
+ 236 42.79661017
247
+ 237 42.61603376
248
+ 238 42.43697479
249
+ 239 42.25941423
250
+ 240 42.50000000
251
+ 241 42.32365145
252
+ 242 42.56198347
253
+ 243 42.79835391
254
+ 244 42.62295082
255
+ 245 42.85714286
256
+ 246 42.68292683
257
+ 247 42.91497976
258
+ 248 43.14516129
259
+ 249 42.97188755
260
+ 250 42.80000000
261
+ 251 42.62948207
262
+ 252 42.46031746
263
+ 253 42.29249012
264
+ 254 42.12598425
265
+ 255 41.96078431
266
+ 256 41.79687500
267
+ 257 42.02334630
268
+ 258 41.86046512
269
+ 259 41.69884170
270
+ 260 41.53846154
271
+ 261 41.37931034
272
+ 262 41.22137405
273
+ 263 41.06463878
274
+ 264 41.28787879
275
+ 265 41.13207547
276
+ 266 41.35338346
277
+ 267 41.19850187
278
+ 268 41.04477612
279
+ 269 41.26394052
280
+ 270 41.48148148
281
+ 271 41.32841328
282
+ 272 41.17647059
283
+ 273 41.02564103
284
+ 274 40.87591241
285
+ 275 40.72727273
286
+ 276 40.94202899
287
+ 277 41.15523466
288
+ 278 41.36690647
289
+ 279 41.21863799
290
+ 280 41.42857143
291
+ 281 41.28113879
292
+ 282 41.13475177
293
+ 283 40.98939929
294
+ 284 40.84507042
295
+ 285 40.70175439
296
+ 286 40.55944056
297
+ 287 40.76655052
298
+ 288 40.62500000
299
+ 289 40.48442907
300
+ 290 40.68965517
301
+ 291 40.89347079
302
+ 292 41.09589041
303
+ 293 41.29692833
304
+ 294 41.15646259
305
+ 295 41.01694915
306
+ 296 40.87837838
307
+ 297 40.74074074
308
+ 298 40.93959732
309
+ 299 41.13712375
310
+ 300 41.33333333
311
+ 301 41.19601329
312
+ 302 41.39072848
313
+ 303 41.25412541
314
+ 304 41.11842105
315
+ 305 41.31147541
316
+ 306 41.17647059
317
+ 307 41.36807818
318
+ 308 41.23376623
319
+ 309 41.10032362
320
+ 310 41.29032258
321
+ 311 41.47909968
322
+ 312 41.66666667
323
+ 313 41.85303514
324
+ 314 42.03821656
325
+ 315 42.22222222
326
+ 316 42.08860759
327
+ 317 42.27129338
328
+ 318 42.13836478
329
+ 319 42.31974922
330
+ 320 42.18750000
331
+ 321 42.05607477
332
+ 322 41.92546584
333
+ 323 41.79566563
334
+ 324 41.66666667
335
+ 325 41.84615385
336
+ 326 41.71779141
337
+ 327 41.89602446
338
+ 328 42.07317073
339
+ 329 41.94528875
340
+ 330 41.81818182
341
+ 331 41.69184290
342
+ 332 41.86746988
343
+ 333 41.74174174
344
+ 334 41.61676647
345
+ 335 41.79104478
346
+ 336 41.96428571
347
+ 337 41.83976261
348
+ 338 41.71597633
349
+ 339 41.59292035
350
+ 340 41.76470588
351
+ 341 41.93548387
352
+ 342 42.10526316
353
+ 343 41.98250729
354
+ 344 41.86046512
355
+ 345 41.73913043
356
+ 346 41.90751445
357
+ 347 42.07492795
358
+ 348 41.95402299
359
+ 349 42.12034384
360
+ 350 42.28571429
361
+ 351 42.16524217
362
+ 352 42.04545455
363
+ 353 42.20963173
364
+ 354 42.37288136
365
+ 355 42.25352113
366
+ 356 42.41573034
367
+ 357 42.29691877
368
+ 358 42.45810056
369
+ 359 42.33983287
370
+ 360 42.50000000
371
+ 361 42.38227147
372
+ 362 42.54143646
373
+ 363 42.42424242
374
+ 364 42.30769231
375
+ 365 42.19178082
376
+ 366 42.34972678
377
+ 367 42.23433243
378
+ 368 42.11956522
379
+ 369 42.00542005
380
+ 370 42.16216216
381
+ 371 42.31805930
382
+ 372 42.20430108
383
+ 373 42.09115282
384
+ 374 42.24598930
385
+ 375 42.40000000
386
+ 376 42.28723404
387
+ 377 42.17506631
388
+ 378 42.32804233
389
+ 379 42.21635884
390
+ 380 42.10526316
391
+ 381 42.25721785
392
+ 382 42.40837696
393
+ 383 42.55874674
394
+ 384 42.44791667
395
+ 385 42.59740260
396
+ 386 42.48704663
397
+ 387 42.37726098
398
+ 388 42.26804124
399
+ 389 42.41645244
400
+ 390 42.56410256
401
+ 391 42.71099744
402
+ 392 42.85714286
403
+ 393 42.74809160
404
+ 394 42.63959391
405
+ 395 42.78481013
406
+ 396 42.92929293
407
+ 397 43.07304786
408
+ 398 42.96482412
409
+ 399 43.10776942
410
+ 400 43.00000000
411
+ 401 43.14214464
412
+ 402 43.03482587
413
+ 403 42.92803970
414
+ 404 42.82178218
415
+ 405 42.71604938
416
+ 406 42.61083744
417
+ 407 42.50614251
418
+ 408 42.40196078
419
+ 409 42.54278729
420
+ 410 42.43902439
421
+ 411 42.33576642
422
+ 412 42.23300971
423
+ 413 42.13075061
424
+ 414 42.02898551
425
+ 415 42.16867470
426
+ 416 42.06730769
427
+ 417 41.96642686
428
+ 418 41.86602871
429
+ 419 42.00477327
430
+ 420 42.14285714
431
+ 421 42.04275534
432
+ 422 41.94312796
433
+ 423 41.84397163
434
+ 424 41.98113208
435
+ 425 42.11764706
436
+ 426 42.25352113
437
+ 427 42.15456674
438
+ 428 42.05607477
439
+ 429 42.19114219
440
+ 430 42.09302326
441
+ 431 42.22737819
442
+ 432 42.36111111
443
+ 433 42.26327945
444
+ 434 42.16589862
445
+ 435 42.06896552
446
+ 436 41.97247706
447
+ 437 41.87643021
448
+ 438 42.00913242
449
+ 439 42.14123007
450
+ 440 42.04545455
451
+ 441 41.95011338
452
+ 442 41.85520362
453
+ 443 41.76072235
454
+ 444 41.66666667
455
+ 445 41.57303371
456
+ 446 41.47982063
457
+ 447 41.61073826
458
+ 448 41.51785714
459
+ 449 41.42538976
460
+ 450 41.55555556
461
+ 451 41.46341463
462
+ 452 41.59292035
463
+ 453 41.72185430
464
+ 454 41.62995595
465
+ 455 41.53846154
466
+ 456 41.66666667
467
+ 457 41.79431072
468
+ 458 41.70305677
469
+ 459 41.61220044
470
+ 460 41.73913043
471
+ 461 41.64859002
472
+ 462 41.55844156
473
+ 463 41.46868251
474
+ 464 41.59482759
475
+ 465 41.50537634
476
+ 466 41.41630901
477
+ 467 41.54175589
478
+ 468 41.45299145
479
+ 469 41.57782516
480
+ 470 41.48936170
481
+ 471 41.40127389
482
+ 472 41.31355932
483
+ 473 41.22621564
484
+ 474 41.13924051
485
+ 475 41.05263158
486
+ 476 41.17647059
487
+ 477 41.09014675
488
+ 478 41.21338912
489
+ 479 41.33611691
490
+ 480 41.45833333
491
+ 481 41.37214137
492
+ 482 41.28630705
493
+ 483 41.20082816
494
+ 484 41.11570248
495
+ 485 41.03092784
496
+ 486 40.94650206
497
+ 487 41.06776181
498
+ 488 40.98360656
499
+ 489 40.89979550
500
+ 490 41.02040816
501
+ 491 40.93686354
502
+ 492 40.85365854
503
+ 493 40.77079108
504
+ 494 40.89068826
505
+ 495 41.01010101
506
+ 496 41.12903226
507
+ 497 41.04627767
508
+ 498 40.96385542
509
+ 499 40.88176353
510
+ 500 40.80000000
511
+ 501 40.71856287
512
+ 502 40.63745020
513
+ 503 40.55666004
514
+ 504 40.67460317
515
+ 505 40.59405941
516
+ 506 40.71146245
517
+ 507 40.63116371
518
+ 508 40.55118110
519
+ 509 40.47151277
520
+ 510 40.39215686
521
+ 511 40.31311155
522
+ 512 40.23437500
523
+ 513 40.15594542
524
+ 514 40.07782101
525
+ 515 40.00000000
526
+ 516 40.11627907
527
+ 517 40.23210832
528
+ 518 40.34749035
529
+ 519 40.46242775
530
+ 520 40.57692308
531
+ 521 40.49904031
532
+ 522 40.42145594
533
+ 523 40.34416826
534
+ 524 40.45801527
535
+ 525 40.38095238
536
+ 526 40.30418251
537
+ 527 40.41745731
538
+ 528 40.34090909
539
+ 529 40.26465028
540
+ 530 40.37735849
541
+ 531 40.30131827
542
+ 532 40.22556391
543
+ 533 40.15009381
544
+ 534 40.07490637
545
+ 535 40.18691589
546
+ 536 40.11194030
547
+ 537 40.22346369
548
+ 538 40.33457249
549
+ 539 40.25974026
550
+ 540 40.18518519
551
+ 541 40.11090573
552
+ 542 40.22140221
553
+ 543 40.14732965
554
+ 544 40.07352941
555
+ 545 40.00000000
556
+ 546 39.92673993
557
+ 547 40.03656307
558
+ 548 40.14598540
559
+ 549 40.07285974
560
+ 550 40.00000000
561
+ 551 40.10889292
562
+ 552 40.03623188
563
+ 553 39.96383363
564
+ 554 40.07220217
565
+ 555 40.00000000
566
+ 556 40.10791367
567
+ 557 40.03590664
568
+ 558 40.14336918
569
+ 559 40.07155635
570
+ 560 40.00000000
571
+ 561 39.92869875
572
+ 562 40.03558719
573
+ 563 40.14209591
574
+ 564 40.07092199
575
+ 565 40.17699115
576
+ 566 40.10600707
577
+ 567 40.03527337
578
+ 568 40.14084507
579
+ 569 40.07029877
580
+ 570 40.17543860
581
+ 571 40.10507881
582
+ 572 40.20979021
583
+ 573 40.13961606
584
+ 574 40.06968641
585
+ 575 40.00000000
586
+ 576 40.10416667
587
+ 577 40.20797227
588
+ 578 40.13840830
589
+ 579 40.06908463
590
+ 580 40.17241379
591
+ 581 40.10327022
592
+ 582 40.03436426
593
+ 583 40.13722127
594
+ 584 40.23972603
595
+ 585 40.17094017
596
+ 586 40.27303754
597
+ 587 40.20442930
598
+ 588 40.13605442
599
+ 589 40.06791171
600
+ 590 40.00000000
601
+ 591 39.93231810
602
+ 592 39.86486486
603
+ 593 39.79763912
604
+ 594 39.89898990
605
+ 595 39.83193277
606
+ 596 39.93288591
607
+ 597 39.86599665
608
+ 598 39.79933110
609
+ 599 39.89983306
610
+ 600 39.83333333
611
+ 601 39.93344426
612
+ 602 39.86710963
613
+ 603 39.96683250
614
+ 604 39.90066225
615
+ 605 40.00000000
616
+ 606 39.93399340
617
+ 607 40.03294893
618
+ 608 39.96710526
619
+ 609 39.90147783
620
+ 610 39.83606557
621
+ 611 39.93453355
622
+ 612 39.86928105
623
+ 613 39.80424144
624
+ 614 39.90228013
625
+ 615 39.83739837
626
+ 616 39.93506494
627
+ 617 39.87034036
628
+ 618 39.96763754
629
+ 619 39.90306947
630
+ 620 39.83870968
631
+ 621 39.77455717
632
+ 622 39.87138264
633
+ 623 39.80738363
634
+ 624 39.74358974
635
+ 625 39.68000000
636
+ 626 39.77635783
637
+ 627 39.71291866
638
+ 628 39.64968153
639
+ 629 39.74562798
640
+ 630 39.68253968
641
+ 631 39.77812995
642
+ 632 39.87341772
643
+ 633 39.81042654
644
+ 634 39.74763407
645
+ 635 39.68503937
646
+ 636 39.62264151
647
+ 637 39.71742543
648
+ 638 39.65517241
649
+ 639 39.74960876
650
+ 640 39.84375000
651
+ 641 39.78159126
652
+ 642 39.87538941
653
+ 643 39.81337481
654
+ 644 39.90683230
655
+ 645 39.84496124
656
+ 646 39.78328173
657
+ 647 39.72179289
658
+ 648 39.66049383
659
+ 649 39.59938367
660
+ 650 39.69230769
661
+ 651 39.63133641
662
+ 652 39.72392638
663
+ 653 39.66309342
664
+ 654 39.75535168
665
+ 655 39.69465649
666
+ 656 39.78658537
667
+ 657 39.72602740
668
+ 658 39.81762918
669
+ 659 39.75720789
670
+ 660 39.69696970
671
+ 661 39.63691377
672
+ 662 39.57703927
673
+ 663 39.51734540
674
+ 664 39.60843373
675
+ 665 39.69924812
676
+ 666 39.63963964
677
+ 667 39.73013493
678
+ 668 39.82035928
679
+ 669 39.91031390
680
+ 670 39.85074627
681
+ 671 39.79135618
682
+ 672 39.73214286
683
+ 673 39.67310550
684
+ 674 39.61424332
685
+ 675 39.55555556
686
+ 676 39.49704142
687
+ 677 39.43870015
688
+ 678 39.38053097
689
+ 679 39.32253314
690
+ 680 39.41176471
691
+ 681 39.50073421
692
+ 682 39.44281525
693
+ 683 39.38506589
694
+ 684 39.32748538
695
+ 685 39.27007299
696
+ 686 39.21282799
697
+ 687 39.15574964
698
+ 688 39.09883721
699
+ 689 39.04208999
700
+ 690 38.98550725
701
+ 691 39.07380608
702
+ 692 39.01734104
703
+ 693 38.96103896
704
+ 694 38.90489914
705
+ 695 38.84892086
706
+ 696 38.93678161
707
+ 697 39.02439024
708
+ 698 38.96848138
709
+ 699 39.05579399
710
+ 700 39.00000000
711
+ 701 39.08701854
712
+ 702 39.17378917
713
+ 703 39.26031294
714
+ 704 39.20454545
715
+ 705 39.14893617
716
+ 706 39.09348442
717
+ 707 39.03818953
718
+ 708 38.98305085
719
+ 709 39.06911142
720
+ 710 39.15492958
721
+ 711 39.24050633
722
+ 712 39.18539326
723
+ 713 39.27068724
724
+ 714 39.35574230
725
+ 715 39.30069930
726
+ 716 39.38547486
727
+ 717 39.33054393
728
+ 718 39.27576602
729
+ 719 39.22114047
730
+ 720 39.16666667
731
+ 721 39.11234397
732
+ 722 39.19667590
733
+ 723 39.28077455
734
+ 724 39.22651934
735
+ 725 39.31034483
736
+ 726 39.25619835
737
+ 727 39.20220083
738
+ 728 39.14835165
739
+ 729 39.09465021
740
+ 730 39.04109589
741
+ 731 38.98768810
742
+ 732 39.07103825
743
+ 733 39.15416098
744
+ 734 39.23705722
745
+ 735 39.18367347
746
+ 736 39.13043478
747
+ 737 39.21302578
748
+ 738 39.15989160
749
+ 739 39.24221922
750
+ 740 39.18918919
751
+ 741 39.13630229
752
+ 742 39.21832884
753
+ 743 39.16554509
754
+ 744 39.24731183
755
+ 745 39.19463087
756
+ 746 39.14209115
757
+ 747 39.08969210
758
+ 748 39.03743316
759
+ 749 38.98531375
760
+ 750 39.06666667
761
+
762
+ Final result: 39.0667 +/- 1.7827
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-F16.tqa ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 817 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 817 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 14.28571429
18
+ 8 12.50000000
19
+ 9 11.11111111
20
+ 10 10.00000000
21
+ 11 9.09090909
22
+ 12 16.66666667
23
+ 13 23.07692308
24
+ 14 21.42857143
25
+ 15 20.00000000
26
+ 16 18.75000000
27
+ 17 17.64705882
28
+ 18 16.66666667
29
+ 19 21.05263158
30
+ 20 20.00000000
31
+ 21 19.04761905
32
+ 22 18.18181818
33
+ 23 17.39130435
34
+ 24 20.83333333
35
+ 25 20.00000000
36
+ 26 19.23076923
37
+ 27 18.51851852
38
+ 28 17.85714286
39
+ 29 20.68965517
40
+ 30 20.00000000
41
+ 31 19.35483871
42
+ 32 21.87500000
43
+ 33 24.24242424
44
+ 34 23.52941176
45
+ 35 22.85714286
46
+ 36 22.22222222
47
+ 37 24.32432432
48
+ 38 23.68421053
49
+ 39 23.07692308
50
+ 40 22.50000000
51
+ 41 21.95121951
52
+ 42 21.42857143
53
+ 43 23.25581395
54
+ 44 22.72727273
55
+ 45 24.44444444
56
+ 46 26.08695652
57
+ 47 25.53191489
58
+ 48 25.00000000
59
+ 49 24.48979592
60
+ 50 26.00000000
61
+ 51 27.45098039
62
+ 52 26.92307692
63
+ 53 26.41509434
64
+ 54 27.77777778
65
+ 55 29.09090909
66
+ 56 28.57142857
67
+ 57 28.07017544
68
+ 58 27.58620690
69
+ 59 27.11864407
70
+ 60 26.66666667
71
+ 61 26.22950820
72
+ 62 27.41935484
73
+ 63 26.98412698
74
+ 64 26.56250000
75
+ 65 26.15384615
76
+ 66 25.75757576
77
+ 67 25.37313433
78
+ 68 25.00000000
79
+ 69 26.08695652
80
+ 70 27.14285714
81
+ 71 26.76056338
82
+ 72 27.77777778
83
+ 73 27.39726027
84
+ 74 27.02702703
85
+ 75 26.66666667
86
+ 76 26.31578947
87
+ 77 25.97402597
88
+ 78 25.64102564
89
+ 79 25.31645570
90
+ 80 26.25000000
91
+ 81 25.92592593
92
+ 82 26.82926829
93
+ 83 26.50602410
94
+ 84 26.19047619
95
+ 85 25.88235294
96
+ 86 25.58139535
97
+ 87 25.28735632
98
+ 88 25.00000000
99
+ 89 25.84269663
100
+ 90 25.55555556
101
+ 91 26.37362637
102
+ 92 26.08695652
103
+ 93 25.80645161
104
+ 94 25.53191489
105
+ 95 26.31578947
106
+ 96 27.08333333
107
+ 97 26.80412371
108
+ 98 26.53061224
109
+ 99 27.27272727
110
+ 100 28.00000000
111
+ 101 28.71287129
112
+ 102 28.43137255
113
+ 103 28.15533981
114
+ 104 27.88461538
115
+ 105 27.61904762
116
+ 106 27.35849057
117
+ 107 28.03738318
118
+ 108 28.70370370
119
+ 109 28.44036697
120
+ 110 29.09090909
121
+ 111 28.82882883
122
+ 112 28.57142857
123
+ 113 28.31858407
124
+ 114 28.94736842
125
+ 115 28.69565217
126
+ 116 28.44827586
127
+ 117 28.20512821
128
+ 118 27.96610169
129
+ 119 27.73109244
130
+ 120 28.33333333
131
+ 121 28.92561983
132
+ 122 28.68852459
133
+ 123 29.26829268
134
+ 124 29.03225806
135
+ 125 29.60000000
136
+ 126 29.36507937
137
+ 127 29.13385827
138
+ 128 28.90625000
139
+ 129 28.68217054
140
+ 130 28.46153846
141
+ 131 28.24427481
142
+ 132 28.03030303
143
+ 133 27.81954887
144
+ 134 28.35820896
145
+ 135 28.88888889
146
+ 136 28.67647059
147
+ 137 29.19708029
148
+ 138 29.71014493
149
+ 139 30.21582734
150
+ 140 30.00000000
151
+ 141 29.78723404
152
+ 142 29.57746479
153
+ 143 29.37062937
154
+ 144 29.86111111
155
+ 145 29.65517241
156
+ 146 29.45205479
157
+ 147 29.25170068
158
+ 148 29.05405405
159
+ 149 28.85906040
160
+ 150 28.66666667
161
+ 151 29.13907285
162
+ 152 28.94736842
163
+ 153 28.75816993
164
+ 154 28.57142857
165
+ 155 29.03225806
166
+ 156 29.48717949
167
+ 157 29.29936306
168
+ 158 29.74683544
169
+ 159 29.55974843
170
+ 160 29.37500000
171
+ 161 29.19254658
172
+ 162 29.01234568
173
+ 163 28.83435583
174
+ 164 28.65853659
175
+ 165 29.09090909
176
+ 166 29.51807229
177
+ 167 29.34131737
178
+ 168 29.16666667
179
+ 169 28.99408284
180
+ 170 28.82352941
181
+ 171 28.65497076
182
+ 172 28.48837209
183
+ 173 28.32369942
184
+ 174 28.73563218
185
+ 175 28.57142857
186
+ 176 28.40909091
187
+ 177 28.24858757
188
+ 178 28.08988764
189
+ 179 28.49162011
190
+ 180 28.33333333
191
+ 181 28.72928177
192
+ 182 28.57142857
193
+ 183 28.41530055
194
+ 184 28.26086957
195
+ 185 28.10810811
196
+ 186 28.49462366
197
+ 187 28.34224599
198
+ 188 28.72340426
199
+ 189 29.10052910
200
+ 190 28.94736842
201
+ 191 28.79581152
202
+ 192 29.16666667
203
+ 193 29.01554404
204
+ 194 29.38144330
205
+ 195 29.23076923
206
+ 196 29.08163265
207
+ 197 29.44162437
208
+ 198 29.79797980
209
+ 199 29.64824121
210
+ 200 29.50000000
211
+ 201 29.35323383
212
+ 202 29.70297030
213
+ 203 29.55665025
214
+ 204 29.41176471
215
+ 205 29.26829268
216
+ 206 29.12621359
217
+ 207 29.46859903
218
+ 208 29.32692308
219
+ 209 29.18660287
220
+ 210 29.04761905
221
+ 211 28.90995261
222
+ 212 28.77358491
223
+ 213 28.63849765
224
+ 214 28.50467290
225
+ 215 28.37209302
226
+ 216 28.70370370
227
+ 217 28.57142857
228
+ 218 28.89908257
229
+ 219 28.76712329
230
+ 220 28.63636364
231
+ 221 28.50678733
232
+ 222 28.37837838
233
+ 223 28.25112108
234
+ 224 28.57142857
235
+ 225 28.44444444
236
+ 226 28.31858407
237
+ 227 28.63436123
238
+ 228 28.50877193
239
+ 229 28.38427948
240
+ 230 28.26086957
241
+ 231 28.13852814
242
+ 232 28.44827586
243
+ 233 28.75536481
244
+ 234 29.05982906
245
+ 235 28.93617021
246
+ 236 29.23728814
247
+ 237 29.53586498
248
+ 238 29.83193277
249
+ 239 29.70711297
250
+ 240 29.58333333
251
+ 241 29.46058091
252
+ 242 29.33884298
253
+ 243 29.21810700
254
+ 244 29.09836066
255
+ 245 28.97959184
256
+ 246 28.86178862
257
+ 247 28.74493927
258
+ 248 28.62903226
259
+ 249 28.51405622
260
+ 250 28.40000000
261
+ 251 28.68525896
262
+ 252 28.57142857
263
+ 253 28.45849802
264
+ 254 28.34645669
265
+ 255 28.62745098
266
+ 256 28.51562500
267
+ 257 28.40466926
268
+ 258 28.29457364
269
+ 259 28.18532819
270
+ 260 28.07692308
271
+ 261 28.35249042
272
+ 262 28.24427481
273
+ 263 28.13688213
274
+ 264 28.40909091
275
+ 265 28.67924528
276
+ 266 28.57142857
277
+ 267 28.83895131
278
+ 268 28.73134328
279
+ 269 28.62453532
280
+ 270 28.51851852
281
+ 271 28.41328413
282
+ 272 28.30882353
283
+ 273 28.20512821
284
+ 274 28.46715328
285
+ 275 28.36363636
286
+ 276 28.26086957
287
+ 277 28.15884477
288
+ 278 28.05755396
289
+ 279 27.95698925
290
+ 280 28.21428571
291
+ 281 28.46975089
292
+ 282 28.36879433
293
+ 283 28.26855124
294
+ 284 28.52112676
295
+ 285 28.42105263
296
+ 286 28.67132867
297
+ 287 28.91986063
298
+ 288 28.81944444
299
+ 289 28.71972318
300
+ 290 28.62068966
301
+ 291 28.86597938
302
+ 292 28.76712329
303
+ 293 29.01023891
304
+ 294 28.91156463
305
+ 295 29.15254237
306
+ 296 29.39189189
307
+ 297 29.62962963
308
+ 298 29.53020134
309
+ 299 29.43143813
310
+ 300 29.33333333
311
+ 301 29.23588040
312
+ 302 29.13907285
313
+ 303 29.04290429
314
+ 304 28.94736842
315
+ 305 28.85245902
316
+ 306 28.75816993
317
+ 307 28.66449511
318
+ 308 28.89610390
319
+ 309 28.80258900
320
+ 310 28.70967742
321
+ 311 28.61736334
322
+ 312 28.52564103
323
+ 313 28.43450479
324
+ 314 28.66242038
325
+ 315 28.57142857
326
+ 316 28.48101266
327
+ 317 28.39116719
328
+ 318 28.30188679
329
+ 319 28.21316614
330
+ 320 28.12500000
331
+ 321 28.03738318
332
+ 322 28.26086957
333
+ 323 28.17337461
334
+ 324 28.08641975
335
+ 325 28.00000000
336
+ 326 27.91411043
337
+ 327 28.13455657
338
+ 328 28.35365854
339
+ 329 28.26747720
340
+ 330 28.18181818
341
+ 331 28.09667674
342
+ 332 28.31325301
343
+ 333 28.22822823
344
+ 334 28.14371257
345
+ 335 28.35820896
346
+ 336 28.57142857
347
+ 337 28.48664688
348
+ 338 28.40236686
349
+ 339 28.61356932
350
+ 340 28.52941176
351
+ 341 28.73900293
352
+ 342 28.65497076
353
+ 343 28.57142857
354
+ 344 28.77906977
355
+ 345 28.98550725
356
+ 346 29.19075145
357
+ 347 29.39481268
358
+ 348 29.31034483
359
+ 349 29.22636103
360
+ 350 29.14285714
361
+ 351 29.05982906
362
+ 352 28.97727273
363
+ 353 28.89518414
364
+ 354 28.81355932
365
+ 355 29.01408451
366
+ 356 28.93258427
367
+ 357 29.13165266
368
+ 358 29.32960894
369
+ 359 29.24791086
370
+ 360 29.16666667
371
+ 361 29.36288089
372
+ 362 29.28176796
373
+ 363 29.47658402
374
+ 364 29.39560440
375
+ 365 29.58904110
376
+ 366 29.50819672
377
+ 367 29.70027248
378
+ 368 29.61956522
379
+ 369 29.53929539
380
+ 370 29.45945946
381
+ 371 29.38005391
382
+ 372 29.56989247
383
+ 373 29.49061662
384
+ 374 29.67914439
385
+ 375 29.60000000
386
+ 376 29.52127660
387
+ 377 29.44297082
388
+ 378 29.36507937
389
+ 379 29.28759894
390
+ 380 29.21052632
391
+ 381 29.13385827
392
+ 382 29.31937173
393
+ 383 29.24281984
394
+ 384 29.16666667
395
+ 385 29.09090909
396
+ 386 29.01554404
397
+ 387 28.94056848
398
+ 388 29.12371134
399
+ 389 29.04884319
400
+ 390 29.23076923
401
+ 391 29.41176471
402
+ 392 29.33673469
403
+ 393 29.51653944
404
+ 394 29.69543147
405
+ 395 29.62025316
406
+ 396 29.54545455
407
+ 397 29.72292191
408
+ 398 29.64824121
409
+ 399 29.57393484
410
+ 400 29.50000000
411
+ 401 29.42643392
412
+ 402 29.60199005
413
+ 403 29.77667494
414
+ 404 29.70297030
415
+ 405 29.62962963
416
+ 406 29.80295567
417
+ 407 29.72972973
418
+ 408 29.65686275
419
+ 409 29.58435208
420
+ 410 29.51219512
421
+ 411 29.44038929
422
+ 412 29.61165049
423
+ 413 29.53995157
424
+ 414 29.71014493
425
+ 415 29.87951807
426
+ 416 29.80769231
427
+ 417 29.73621103
428
+ 418 29.66507177
429
+ 419 29.59427208
430
+ 420 29.76190476
431
+ 421 29.92874109
432
+ 422 29.85781991
433
+ 423 29.78723404
434
+ 424 29.71698113
435
+ 425 29.64705882
436
+ 426 29.57746479
437
+ 427 29.50819672
438
+ 428 29.43925234
439
+ 429 29.60372960
440
+ 430 29.53488372
441
+ 431 29.46635731
442
+ 432 29.62962963
443
+ 433 29.79214781
444
+ 434 29.95391705
445
+ 435 29.88505747
446
+ 436 29.81651376
447
+ 437 29.74828375
448
+ 438 29.68036530
449
+ 439 29.61275626
450
+ 440 29.54545455
451
+ 441 29.47845805
452
+ 442 29.41176471
453
+ 443 29.34537246
454
+ 444 29.50450450
455
+ 445 29.43820225
456
+ 446 29.37219731
457
+ 447 29.30648770
458
+ 448 29.24107143
459
+ 449 29.39866370
460
+ 450 29.55555556
461
+ 451 29.49002217
462
+ 452 29.42477876
463
+ 453 29.35982340
464
+ 454 29.51541850
465
+ 455 29.45054945
466
+ 456 29.38596491
467
+ 457 29.32166302
468
+ 458 29.25764192
469
+ 459 29.19389978
470
+ 460 29.13043478
471
+ 461 29.06724512
472
+ 462 29.00432900
473
+ 463 29.15766739
474
+ 464 29.31034483
475
+ 465 29.46236559
476
+ 466 29.39914163
477
+ 467 29.33618844
478
+ 468 29.48717949
479
+ 469 29.42430704
480
+ 470 29.36170213
481
+ 471 29.51167728
482
+ 472 29.44915254
483
+ 473 29.38689218
484
+ 474 29.32489451
485
+ 475 29.47368421
486
+ 476 29.41176471
487
+ 477 29.55974843
488
+ 478 29.49790795
489
+ 479 29.43632568
490
+ 480 29.37500000
491
+ 481 29.31392931
492
+ 482 29.46058091
493
+ 483 29.39958592
494
+ 484 29.54545455
495
+ 485 29.69072165
496
+ 486 29.62962963
497
+ 487 29.77412731
498
+ 488 29.71311475
499
+ 489 29.85685072
500
+ 490 30.00000000
501
+ 491 30.14256619
502
+ 492 30.28455285
503
+ 493 30.22312373
504
+ 494 30.16194332
505
+ 495 30.30303030
506
+ 496 30.24193548
507
+ 497 30.18108652
508
+ 498 30.12048193
509
+ 499 30.26052104
510
+ 500 30.20000000
511
+ 501 30.33932136
512
+ 502 30.27888446
513
+ 503 30.21868787
514
+ 504 30.35714286
515
+ 505 30.29702970
516
+ 506 30.23715415
517
+ 507 30.37475345
518
+ 508 30.51181102
519
+ 509 30.64833006
520
+ 510 30.78431373
521
+ 511 30.72407045
522
+ 512 30.66406250
523
+ 513 30.60428850
524
+ 514 30.54474708
525
+ 515 30.67961165
526
+ 516 30.62015504
527
+ 517 30.75435203
528
+ 518 30.69498069
529
+ 519 30.63583815
530
+ 520 30.57692308
531
+ 521 30.51823417
532
+ 522 30.45977011
533
+ 523 30.40152964
534
+ 524 30.34351145
535
+ 525 30.28571429
536
+ 526 30.22813688
537
+ 527 30.36053131
538
+ 528 30.30303030
539
+ 529 30.24574669
540
+ 530 30.18867925
541
+ 531 30.13182674
542
+ 532 30.07518797
543
+ 533 30.20637899
544
+ 534 30.14981273
545
+ 535 30.09345794
546
+ 536 30.03731343
547
+ 537 30.16759777
548
+ 538 30.11152416
549
+ 539 30.05565863
550
+ 540 30.00000000
551
+ 541 29.94454713
552
+ 542 29.88929889
553
+ 543 29.83425414
554
+ 544 29.77941176
555
+ 545 29.72477064
556
+ 546 29.85347985
557
+ 547 29.98171846
558
+ 548 30.10948905
559
+ 549 30.05464481
560
+ 550 30.00000000
561
+ 551 29.94555354
562
+ 552 30.07246377
563
+ 553 30.19891501
564
+ 554 30.14440433
565
+ 555 30.09009009
566
+ 556 30.03597122
567
+ 557 29.98204668
568
+ 558 29.92831541
569
+ 559 29.87477639
570
+ 560 30.00000000
571
+ 561 30.12477718
572
+ 562 30.07117438
573
+ 563 30.01776199
574
+ 564 30.14184397
575
+ 565 30.26548673
576
+ 566 30.38869258
577
+ 567 30.33509700
578
+ 568 30.45774648
579
+ 569 30.57996485
580
+ 570 30.52631579
581
+ 571 30.47285464
582
+ 572 30.41958042
583
+ 573 30.36649215
584
+ 574 30.48780488
585
+ 575 30.60869565
586
+ 576 30.55555556
587
+ 577 30.50259965
588
+ 578 30.62283737
589
+ 579 30.56994819
590
+ 580 30.68965517
591
+ 581 30.63683305
592
+ 582 30.75601375
593
+ 583 30.70325901
594
+ 584 30.82191781
595
+ 585 30.76923077
596
+ 586 30.88737201
597
+ 587 30.83475298
598
+ 588 30.78231293
599
+ 589 30.73005093
600
+ 590 30.84745763
601
+ 591 30.79526227
602
+ 592 30.74324324
603
+ 593 30.69139966
604
+ 594 30.63973064
605
+ 595 30.58823529
606
+ 596 30.53691275
607
+ 597 30.48576214
608
+ 598 30.43478261
609
+ 599 30.38397329
610
+ 600 30.33333333
611
+ 601 30.44925125
612
+ 602 30.39867110
613
+ 603 30.51409619
614
+ 604 30.46357616
615
+ 605 30.41322314
616
+ 606 30.52805281
617
+ 607 30.47775947
618
+ 608 30.42763158
619
+ 609 30.37766831
620
+ 610 30.32786885
621
+ 611 30.27823241
622
+ 612 30.39215686
623
+ 613 30.34257749
624
+ 614 30.29315961
625
+ 615 30.24390244
626
+ 616 30.35714286
627
+ 617 30.47001621
628
+ 618 30.42071197
629
+ 619 30.53311793
630
+ 620 30.48387097
631
+ 621 30.43478261
632
+ 622 30.54662379
633
+ 623 30.49759230
634
+ 624 30.44871795
635
+ 625 30.40000000
636
+ 626 30.35143770
637
+ 627 30.46251994
638
+ 628 30.41401274
639
+ 629 30.36565978
640
+ 630 30.31746032
641
+ 631 30.42789223
642
+ 632 30.37974684
643
+ 633 30.33175355
644
+ 634 30.44164038
645
+ 635 30.39370079
646
+ 636 30.34591195
647
+ 637 30.29827316
648
+ 638 30.25078370
649
+ 639 30.20344288
650
+ 640 30.15625000
651
+ 641 30.10920437
652
+ 642 30.21806854
653
+ 643 30.17107309
654
+ 644 30.12422360
655
+ 645 30.23255814
656
+ 646 30.18575851
657
+ 647 30.13910355
658
+ 648 30.24691358
659
+ 649 30.20030817
660
+ 650 30.30769231
661
+ 651 30.41474654
662
+ 652 30.52147239
663
+ 653 30.62787136
664
+ 654 30.58103976
665
+ 655 30.53435115
666
+ 656 30.64024390
667
+ 657 30.74581431
668
+ 658 30.69908815
669
+ 659 30.80424886
670
+ 660 30.75757576
671
+ 661 30.71104387
672
+ 662 30.66465257
673
+ 663 30.61840121
674
+ 664 30.57228916
675
+ 665 30.52631579
676
+ 666 30.63063063
677
+ 667 30.58470765
678
+ 668 30.53892216
679
+ 669 30.49327354
680
+ 670 30.59701493
681
+ 671 30.55141580
682
+ 672 30.50595238
683
+ 673 30.46062407
684
+ 674 30.41543027
685
+ 675 30.51851852
686
+ 676 30.62130178
687
+ 677 30.57607090
688
+ 678 30.53097345
689
+ 679 30.48600884
690
+ 680 30.44117647
691
+ 681 30.39647577
692
+ 682 30.35190616
693
+ 683 30.30746706
694
+ 684 30.40935673
695
+ 685 30.36496350
696
+ 686 30.32069971
697
+ 687 30.27656477
698
+ 688 30.23255814
699
+ 689 30.33381713
700
+ 690 30.28985507
701
+ 691 30.39073806
702
+ 692 30.49132948
703
+ 693 30.59163059
704
+ 694 30.54755043
705
+ 695 30.50359712
706
+ 696 30.60344828
707
+ 697 30.55954089
708
+ 698 30.51575931
709
+ 699 30.47210300
710
+ 700 30.42857143
711
+ 701 30.38516405
712
+ 702 30.34188034
713
+ 703 30.29871977
714
+ 704 30.25568182
715
+ 705 30.21276596
716
+ 706 30.16997167
717
+ 707 30.26874116
718
+ 708 30.22598870
719
+ 709 30.18335684
720
+ 710 30.14084507
721
+ 711 30.09845288
722
+ 712 30.05617978
723
+ 713 30.01402525
724
+ 714 30.11204482
725
+ 715 30.20979021
726
+ 716 30.16759777
727
+ 717 30.12552301
728
+ 718 30.08356546
729
+ 719 30.04172462
730
+ 720 30.00000000
731
+ 721 30.09708738
732
+ 722 30.05540166
733
+ 723 30.01383126
734
+ 724 30.11049724
735
+ 725 30.06896552
736
+ 726 30.16528926
737
+ 727 30.12379642
738
+ 728 30.08241758
739
+ 729 30.17832647
740
+ 730 30.13698630
741
+ 731 30.09575923
742
+ 732 30.19125683
743
+ 733 30.15006821
744
+ 734 30.10899183
745
+ 735 30.06802721
746
+ 736 30.02717391
747
+ 737 30.12211669
748
+ 738 30.08130081
749
+ 739 30.17591340
750
+ 740 30.13513514
751
+ 741 30.22941970
752
+ 742 30.18867925
753
+ 743 30.14804845
754
+ 744 30.24193548
755
+ 745 30.20134228
756
+ 746 30.16085791
757
+ 747 30.25435074
758
+ 748 30.21390374
759
+ 749 30.30707610
760
+ 750 30.40000000
761
+
762
+ Final result: 30.4000 +/- 1.6807
763
+ Random chance: 19.8992 +/- 1.4588
scores/Hammer2.1-7b-F16.wng ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ winogrande_score : loaded 1267 tasks from prompt.
6
+ winogrande_score : selecting 750 random tasks
7
+ winogrande_score : tokenizing selected tasks
8
+ winogrande_score : calculating winogrande score over selected tasks.
9
+ 1 100.0000 -4.055439 -4.419021 1 1
10
+ 2 100.0000 -1.994390 -2.414457 1 1
11
+ 3 100.0000 -2.838952 -2.357486 2 2
12
+ 4 75.0000 -5.716558 -5.131632 2 1
13
+ 5 60.0000 -2.325860 -3.096202 1 2
14
+ 6 66.6667 -5.037614 -5.074246 1 1
15
+ 7 71.4286 -4.322001 -3.242329 2 2
16
+ 8 75.0000 -2.126535 -2.010911 2 2
17
+ 9 77.7778 -3.590984 -5.942347 1 1
18
+ 10 80.0000 -1.842690 -1.169015 2 2
19
+ 11 81.8182 -11.182351 -5.625857 2 2
20
+ 12 83.3333 -3.757430 -3.044646 2 2
21
+ 13 84.6154 -2.749303 -5.718965 1 1
22
+ 14 85.7143 -3.908376 -4.026354 1 1
23
+ 15 80.0000 -4.219364 -5.679240 1 2
24
+ 16 81.2500 -4.231910 -3.701456 2 2
25
+ 17 76.4706 -6.272729 -6.447972 1 2
26
+ 18 77.7778 -1.322470 -3.143499 1 1
27
+ 19 78.9474 -4.247412 -5.186660 1 1
28
+ 20 80.0000 -2.935635 -3.672295 1 1
29
+ 21 80.9524 -2.413390 -3.021779 1 1
30
+ 22 81.8182 -2.625816 -2.115423 2 2
31
+ 23 78.2609 -4.855376 -4.853319 2 1
32
+ 24 75.0000 -2.757748 -2.688151 2 1
33
+ 25 76.0000 -2.939506 -2.787461 2 2
34
+ 26 76.9231 -3.558465 -4.293455 1 1
35
+ 27 77.7778 -5.321280 -5.081777 2 2
36
+ 28 78.5714 -4.520099 -5.536541 1 1
37
+ 29 79.3103 -4.019023 -4.219948 1 1
38
+ 30 76.6667 -1.418497 -1.778071 1 2
39
+ 31 74.1935 -5.258681 -5.232336 2 1
40
+ 32 75.0000 -4.122805 -5.975681 1 1
41
+ 33 75.7576 -3.359194 -6.618768 1 1
42
+ 34 73.5294 -2.437492 -2.285865 2 1
43
+ 35 74.2857 -4.680112 -4.565025 2 2
44
+ 36 75.0000 -4.268436 -3.331906 2 2
45
+ 37 72.9730 -3.508010 -3.136950 2 1
46
+ 38 73.6842 -3.400168 -3.644126 1 1
47
+ 39 74.3590 -4.921366 -4.467425 2 2
48
+ 40 72.5000 -4.184834 -5.428998 1 2
49
+ 41 73.1707 -2.441648 -2.718920 1 1
50
+ 42 73.8095 -1.690651 -2.270757 1 1
51
+ 43 72.0930 -4.102617 -4.205776 1 2
52
+ 44 72.7273 -1.846514 -1.698038 2 2
53
+ 45 73.3333 -0.091718 -15.342812 1 1
54
+ 46 73.9130 -2.575360 -2.866401 1 1
55
+ 47 72.3404 -4.209410 -4.718967 1 2
56
+ 48 72.9167 -5.025632 -4.666284 2 2
57
+ 49 71.4286 -3.022136 -0.998877 2 1
58
+ 50 70.0000 -2.627678 -2.714285 1 2
59
+ 51 70.5882 -2.298244 -2.658348 1 1
60
+ 52 71.1538 -3.003663 -2.279978 2 2
61
+ 53 71.6981 -4.153790 -5.159662 1 1
62
+ 54 72.2222 -3.413091 -3.166169 2 2
63
+ 55 72.7273 -3.444228 -3.113577 2 2
64
+ 56 71.4286 -4.784309 -5.284554 1 2
65
+ 57 70.1754 -1.684957 -2.614417 1 2
66
+ 58 70.6897 -3.296965 -4.336903 1 1
67
+ 59 71.1864 -3.278659 -2.997957 2 2
68
+ 60 71.6667 -4.872578 -4.488165 2 2
69
+ 61 72.1311 -3.798878 -4.386929 1 1
70
+ 62 70.9677 -6.798470 -6.648386 2 1
71
+ 63 69.8413 -4.294256 -4.905307 1 2
72
+ 64 70.3125 -4.029755 -4.164216 1 1
73
+ 65 70.7692 -5.029077 -4.484825 2 2
74
+ 66 71.2121 -2.557986 -3.482089 1 1
75
+ 67 71.6418 -2.301429 -1.991850 2 2
76
+ 68 70.5882 -4.723915 -4.969136 1 2
77
+ 69 71.0145 -4.600227 -8.000354 1 1
78
+ 70 71.4286 -3.467141 -2.845032 2 2
79
+ 71 70.4225 -3.271152 -3.060358 2 1
80
+ 72 69.4444 -2.949559 -3.228271 1 2
81
+ 73 69.8630 -2.968552 -3.367588 1 1
82
+ 74 70.2703 -2.510476 -2.527653 1 1
83
+ 75 70.6667 -1.405147 -2.228080 1 1
84
+ 76 71.0526 -1.794828 -3.984558 1 1
85
+ 77 70.1299 -2.863695 -3.181273 1 2
86
+ 78 70.5128 -1.044888 -1.572483 1 1
87
+ 79 69.6203 -2.318248 -2.445590 1 2
88
+ 80 70.0000 -2.067252 -3.576698 1 1
89
+ 81 69.1358 -3.354719 -3.522744 1 2
90
+ 82 68.2927 -3.046368 -3.402303 1 2
91
+ 83 67.4699 -6.077556 -6.104643 1 2
92
+ 84 67.8571 -2.507330 -2.413816 2 2
93
+ 85 68.2353 -4.727517 -5.214082 1 1
94
+ 86 67.4419 -5.888114 -6.221765 1 2
95
+ 87 66.6667 -1.344104 -1.114527 2 1
96
+ 88 65.9091 -4.609307 -4.338995 2 1
97
+ 89 66.2921 -2.861644 -2.935848 1 1
98
+ 90 66.6667 -6.049223 -4.062026 2 2
99
+ 91 65.9341 -4.660932 -4.292127 2 1
100
+ 92 66.3043 -4.823711 -3.652225 2 2
101
+ 93 66.6667 -2.428390 -1.770556 2 2
102
+ 94 67.0213 -4.350206 -3.202328 2 2
103
+ 95 67.3684 -5.176978 -5.942940 1 1
104
+ 96 67.7083 -1.797323 -2.746444 1 1
105
+ 97 68.0412 -4.487276 -4.732371 1 1
106
+ 98 68.3673 -4.651319 -3.609334 2 2
107
+ 99 67.6768 -2.805462 -2.215777 2 1
108
+ 100 68.0000 -6.732390 -6.455460 2 2
109
+ 101 67.3267 -4.710756 -4.752909 1 2
110
+ 102 67.6471 -3.454156 -4.455014 1 1
111
+ 103 66.9903 -2.503772 -3.195619 1 2
112
+ 104 66.3462 -8.104036 -8.107222 1 2
113
+ 105 66.6667 -1.224225 -2.197790 1 1
114
+ 106 66.9811 -2.643758 -2.293319 2 2
115
+ 107 67.2897 -3.271029 -2.324349 2 2
116
+ 108 67.5926 -3.887403 -2.851280 2 2
117
+ 109 66.9725 -5.721020 -5.750739 1 2
118
+ 110 67.2727 -4.389151 -3.388770 2 2
119
+ 111 67.5676 -4.269416 -3.696424 2 2
120
+ 112 66.9643 -3.576009 -3.208990 2 1
121
+ 113 66.3717 -4.384474 -4.661866 1 2
122
+ 114 66.6667 -3.649970 -4.880840 1 1
123
+ 115 66.9565 -1.689978 -2.642224 1 1
124
+ 116 67.2414 -3.811832 -2.614350 2 2
125
+ 117 66.6667 -4.344290 -4.694312 1 2
126
+ 118 66.9492 -0.721044 -2.258560 1 1
127
+ 119 67.2269 -2.553636 -2.906963 1 1
128
+ 120 66.6667 -2.594529 -2.487128 2 1
129
+ 121 66.9421 -0.981829 -0.577011 2 2
130
+ 122 67.2131 -6.314806 -4.913580 2 2
131
+ 123 67.4797 -3.108168 -3.301544 1 1
132
+ 124 67.7419 -2.567195 -2.135424 2 2
133
+ 125 68.0000 -3.772996 -2.860655 2 2
134
+ 126 67.4603 -6.485851 -6.913919 1 2
135
+ 127 67.7165 -2.170513 -1.646936 2 2
136
+ 128 67.9688 -1.827901 -2.062166 1 1
137
+ 129 68.2171 -1.768827 -1.231508 2 2
138
+ 130 68.4615 -5.452911 -4.885611 2 2
139
+ 131 67.9389 -3.506332 -3.697697 1 2
140
+ 132 68.1818 -2.690641 -2.313704 2 2
141
+ 133 68.4211 -1.390372 -3.895249 1 1
142
+ 134 68.6567 -5.322285 -4.277040 2 2
143
+ 135 68.8889 -4.232067 -5.179004 1 1
144
+ 136 69.1176 -5.780380 -4.570644 2 2
145
+ 137 68.6131 -6.462107 -6.428723 2 1
146
+ 138 68.1159 -3.734022 -4.323860 1 2
147
+ 139 67.6259 -4.478791 -4.399325 2 1
148
+ 140 67.8571 -1.499439 -1.167298 2 2
149
+ 141 68.0851 -4.154501 -3.378340 2 2
150
+ 142 67.6056 -2.243544 -2.260092 1 2
151
+ 143 67.8322 -2.042544 -6.271279 1 1
152
+ 144 67.3611 -2.051801 -1.975945 2 1
153
+ 145 66.8966 -1.442542 -12.974517 1 2
154
+ 146 67.1233 -4.249250 -3.872481 2 2
155
+ 147 66.6667 -4.872483 -4.919782 1 2
156
+ 148 66.8919 -1.786300 -2.396421 1 1
157
+ 149 67.1141 -4.435111 -3.667068 2 2
158
+ 150 67.3333 -4.040969 -3.064584 2 2
159
+ 151 66.8874 -2.053489 -1.981597 2 1
160
+ 152 66.4474 -3.723363 -4.032995 1 2
161
+ 153 66.6667 -3.556969 -3.788189 1 1
162
+ 154 66.8831 -3.329666 -3.520848 1 1
163
+ 155 66.4516 -2.301282 -2.330158 1 2
164
+ 156 66.0256 -2.417298 -2.287121 2 1
165
+ 157 66.2420 -2.864692 -4.246324 1 1
166
+ 158 66.4557 -4.222333 -3.619629 2 2
167
+ 159 66.6667 -3.463239 -5.302717 1 1
168
+ 160 66.8750 -4.045500 -3.351726 2 2
169
+ 161 67.0807 -2.441900 -2.746578 1 1
170
+ 162 66.6667 -0.605816 -15.583919 1 2
171
+ 163 66.8712 -1.895707 -1.914864 1 1
172
+ 164 67.0732 -4.967230 -4.378232 2 2
173
+ 165 67.2727 -4.482863 -4.443166 2 2
174
+ 166 67.4699 -4.138143 -3.553382 2 2
175
+ 167 67.0659 -4.937672 -4.503013 2 1
176
+ 168 66.6667 -4.986084 -5.308964 1 2
177
+ 169 66.2722 -3.364907 -3.796789 1 2
178
+ 170 65.8824 -4.342420 -3.858494 2 1
179
+ 171 66.0819 -3.157801 -4.515477 1 1
180
+ 172 66.2791 -4.464201 -5.872163 1 1
181
+ 173 66.4740 -3.272618 -2.847452 2 2
182
+ 174 66.6667 -6.701015 -6.228278 2 2
183
+ 175 66.2857 -4.348508 -4.220059 2 1
184
+ 176 66.4773 -1.764183 -3.419310 1 1
185
+ 177 66.6667 -5.968771 -5.382366 2 2
186
+ 178 66.2921 -2.519285 -2.291575 2 1
187
+ 179 66.4804 -2.650846 -2.769904 1 1
188
+ 180 66.6667 -5.943086 -5.459014 2 2
189
+ 181 66.8508 -3.789927 -2.564274 2 2
190
+ 182 67.0330 -4.053130 -3.407503 2 2
191
+ 183 67.2131 -6.018665 -5.573976 2 2
192
+ 184 67.3913 -1.917318 -3.055713 1 1
193
+ 185 67.0270 -5.965355 -6.046276 1 2
194
+ 186 67.2043 -2.893565 -3.409472 1 1
195
+ 187 67.3797 -4.809973 -4.510880 2 2
196
+ 188 67.0213 -3.241713 -2.941165 2 1
197
+ 189 66.6667 -5.199727 -5.748722 1 2
198
+ 190 66.8421 -5.141917 -4.948495 2 2
199
+ 191 67.0157 -4.125053 -4.207387 1 1
200
+ 192 67.1875 -5.968366 -4.117095 2 2
201
+ 193 67.3575 -4.017663 -3.878347 2 2
202
+ 194 67.5258 -1.871131 -1.257631 2 2
203
+ 195 67.1795 -4.232763 -4.020350 2 1
204
+ 196 67.3469 -3.504522 -2.597405 2 2
205
+ 197 67.5127 -0.532241 -2.750402 1 1
206
+ 198 67.6768 -5.826796 -6.333598 1 1
207
+ 199 67.8392 -4.930836 -3.987738 2 2
208
+ 200 68.0000 -3.251847 -2.856095 2 2
209
+ 201 68.1592 -2.707251 -3.383317 1 1
210
+ 202 68.3168 -7.625721 -7.265589 2 2
211
+ 203 68.4729 -3.833396 -3.570021 2 2
212
+ 204 68.6275 -3.047026 -3.345161 1 1
213
+ 205 68.7805 -3.532288 -2.131426 2 2
214
+ 206 68.4466 -5.617566 -5.857633 1 2
215
+ 207 68.5990 -2.641374 -2.870759 1 1
216
+ 208 68.7500 -4.523323 -2.994968 2 2
217
+ 209 68.8995 -4.949806 -4.441855 2 2
218
+ 210 68.5714 -7.984277 -8.961130 1 2
219
+ 211 68.2464 -5.746709 -5.262517 2 1
220
+ 212 68.3962 -4.772958 -3.535461 2 2
221
+ 213 68.0751 -5.202567 -5.799647 1 2
222
+ 214 68.2243 -2.838898 -3.064564 1 1
223
+ 215 68.3721 -5.390200 -4.899236 2 2
224
+ 216 68.0556 -3.643626 -3.429540 2 1
225
+ 217 68.2028 -0.886357 -1.528328 1 1
226
+ 218 68.3486 -2.227410 -2.180412 2 2
227
+ 219 68.4932 -2.674753 -3.019070 1 1
228
+ 220 68.6364 -3.250026 -3.196657 2 2
229
+ 221 68.7783 -2.722757 -8.409518 1 1
230
+ 222 68.4685 -0.346051 -7.540808 1 2
231
+ 223 68.6099 -4.592046 -4.352857 2 2
232
+ 224 68.3036 -5.896787 -6.223004 1 2
233
+ 225 68.4444 -4.074914 -3.067667 2 2
234
+ 226 68.5841 -2.903729 -3.384342 1 1
235
+ 227 68.2819 -3.851228 -3.494368 2 1
236
+ 228 68.4211 -1.424058 -3.150252 1 1
237
+ 229 68.5590 -4.204830 -3.973340 2 2
238
+ 230 68.6957 -3.039863 -3.353922 1 1
239
+ 231 68.8312 -2.031369 -2.412291 1 1
240
+ 232 68.9655 -2.420022 -1.073308 2 2
241
+ 233 69.0987 -1.657681 -2.059717 1 1
242
+ 234 69.2308 -4.091949 -3.171844 2 2
243
+ 235 69.3617 -3.495027 -2.812989 2 2
244
+ 236 69.0678 -2.196687 -1.778580 2 1
245
+ 237 69.1983 -1.250920 -1.557483 1 1
246
+ 238 69.3277 -3.001484 -2.715834 2 2
247
+ 239 69.4561 -2.726393 -2.433312 2 2
248
+ 240 69.1667 -7.107066 -7.479774 1 2
249
+ 241 69.2946 -4.332081 -3.480189 2 2
250
+ 242 69.4215 -6.098055 -5.740554 2 2
251
+ 243 69.1358 -4.767206 -4.617966 2 1
252
+ 244 69.2623 -2.027227 -3.485415 1 1
253
+ 245 69.3878 -4.285238 -5.580378 1 1
254
+ 246 69.5122 -2.411857 -2.953819 1 1
255
+ 247 69.2308 -4.617576 -5.207825 1 2
256
+ 248 69.3548 -3.121220 -2.802647 2 2
257
+ 249 69.4779 -3.386327 -2.120906 2 2
258
+ 250 69.6000 -3.380254 -4.053829 1 1
259
+ 251 69.7211 -0.640752 -1.002419 1 1
260
+ 252 69.8413 -6.508970 -6.319885 2 2
261
+ 253 69.9605 -5.508479 -5.623730 1 1
262
+ 254 70.0787 -4.274956 -3.035597 2 2
263
+ 255 70.1961 -2.793911 -4.141489 1 1
264
+ 256 70.3125 -3.135646 -4.057714 1 1
265
+ 257 70.0389 -2.566107 -2.030937 2 1
266
+ 258 69.7674 -3.320908 -3.648064 1 2
267
+ 259 69.8842 -4.268399 -4.492817 1 1
268
+ 260 70.0000 -4.713462 -5.530634 1 1
269
+ 261 70.1149 -3.810337 -3.621779 2 2
270
+ 262 70.2290 -5.318979 -5.136152 2 2
271
+ 263 70.3422 -4.593876 -6.012805 1 1
272
+ 264 70.4545 -4.091850 -3.853411 2 2
273
+ 265 70.5660 -1.672003 -2.401845 1 1
274
+ 266 70.6767 -5.850142 -7.794765 1 1
275
+ 267 70.7865 -3.514038 -3.540711 1 1
276
+ 268 70.5224 -2.931465 -2.646491 2 1
277
+ 269 70.6320 -3.247788 -3.132723 2 2
278
+ 270 70.7407 -2.285630 -3.715402 1 1
279
+ 271 70.8487 -3.401312 -3.128446 2 2
280
+ 272 70.9559 -4.385225 -3.864730 2 2
281
+ 273 71.0623 -3.825910 -3.517444 2 2
282
+ 274 71.1679 -2.776444 -2.094572 2 2
283
+ 275 71.2727 -1.546691 -2.552037 1 1
284
+ 276 71.0145 -2.168691 -2.097320 2 1
285
+ 277 71.1191 -3.058795 -3.099674 1 1
286
+ 278 71.2230 -2.667455 -2.707298 1 1
287
+ 279 71.3262 -4.434260 -2.894139 2 2
288
+ 280 71.4286 -4.127873 -3.829644 2 2
289
+ 281 71.5302 -2.425580 -2.012678 2 2
290
+ 282 71.6312 -4.164600 -3.799936 2 2
291
+ 283 71.7314 -3.311673 -3.249197 2 2
292
+ 284 71.4789 -2.805728 -2.585389 2 1
293
+ 285 71.5789 -5.217248 -5.701823 1 1
294
+ 286 71.6783 -3.624409 -3.720116 1 1
295
+ 287 71.7770 -1.958489 -3.952594 1 1
296
+ 288 71.8750 -3.523180 -2.496242 2 2
297
+ 289 71.9723 -2.218095 -2.571024 1 1
298
+ 290 72.0690 -5.602778 -6.318905 1 1
299
+ 291 72.1649 -1.582557 -3.241511 1 1
300
+ 292 71.9178 -1.774837 -1.907137 1 2
301
+ 293 72.0137 -3.519303 -2.862439 2 2
302
+ 294 71.7687 -4.320015 -3.079051 2 1
303
+ 295 71.8644 -1.461996 -1.950170 1 1
304
+ 296 71.9595 -4.535289 -4.400023 2 2
305
+ 297 71.7172 -0.778177 -1.002825 1 2
306
+ 298 71.4765 -5.782339 -6.823362 1 2
307
+ 299 71.5719 -2.133753 -1.745987 2 2
308
+ 300 71.6667 -3.038756 -3.772575 1 1
309
+ 301 71.7608 -4.278540 -5.474163 1 1
310
+ 302 71.5232 -4.315574 -4.398522 1 2
311
+ 303 71.2871 -8.605207 -7.583033 2 1
312
+ 304 71.3816 -3.851751 -3.527867 2 2
313
+ 305 71.1475 -2.781876 -2.931115 1 2
314
+ 306 71.2418 -2.540228 -2.252013 2 2
315
+ 307 71.3355 -2.860733 -3.049699 1 1
316
+ 308 71.1039 -4.911412 -5.081537 1 2
317
+ 309 71.1974 -2.635463 -3.266017 1 1
318
+ 310 71.2903 -4.573822 -4.520030 2 2
319
+ 311 71.3826 -1.047166 -4.598636 1 1
320
+ 312 71.1538 -5.177258 -4.037105 2 1
321
+ 313 70.9265 -4.383779 -4.871078 1 2
322
+ 314 71.0191 -6.254673 -5.354595 2 2
323
+ 315 71.1111 -2.103314 -3.629093 1 1
324
+ 316 71.2025 -6.615143 -5.377004 2 2
325
+ 317 70.9779 -4.900149 -4.535094 2 1
326
+ 318 71.0692 -5.238170 -4.475380 2 2
327
+ 319 71.1599 -1.481243 -2.247088 1 1
328
+ 320 71.2500 -4.131887 -5.291605 1 1
329
+ 321 71.0280 -3.046349 -3.047068 1 2
330
+ 322 71.1180 -2.137508 -2.085015 2 2
331
+ 323 71.2074 -2.579125 -3.014108 1 1
332
+ 324 70.9877 -3.937276 -3.864552 2 1
333
+ 325 71.0769 -5.044379 -4.579688 2 2
334
+ 326 71.1656 -2.790554 -1.931764 2 2
335
+ 327 71.2538 -2.254442 -3.111208 1 1
336
+ 328 71.3415 -3.920905 -4.342911 1 1
337
+ 329 71.4286 -3.502520 -9.424062 1 1
338
+ 330 71.5152 -0.983730 -2.006125 1 1
339
+ 331 71.2991 -5.753674 -3.823415 2 1
340
+ 332 71.3855 -1.846766 -2.698520 1 1
341
+ 333 71.4715 -2.827264 -1.658004 2 2
342
+ 334 71.5569 -2.191762 -5.428309 1 1
343
+ 335 71.3433 -4.551720 -4.395823 2 1
344
+ 336 71.1310 -3.419863 -3.972088 1 2
345
+ 337 71.2166 -1.671326 -1.623333 2 2
346
+ 338 71.0059 -4.146835 -3.354743 2 1
347
+ 339 71.0914 -5.359687 -4.731552 2 2
348
+ 340 71.1765 -5.314707 -5.717268 1 1
349
+ 341 71.2610 -2.444691 -4.436710 1 1
350
+ 342 71.3450 -6.460964 -6.251052 2 2
351
+ 343 71.4286 -4.101875 -4.528821 1 1
352
+ 344 71.5116 -2.352559 -2.714653 1 1
353
+ 345 71.5942 -1.030718 -3.104416 1 1
354
+ 346 71.3873 -4.589089 -7.076116 1 2
355
+ 347 71.4697 -4.103656 -3.491555 2 2
356
+ 348 71.5517 -0.790889 -2.688501 1 1
357
+ 349 71.3467 -3.472429 -5.556114 1 2
358
+ 350 71.1429 -2.073458 -3.626533 1 2
359
+ 351 71.2251 -2.339685 -3.364195 1 1
360
+ 352 71.3068 -3.401416 -2.591679 2 2
361
+ 353 71.1048 -1.960045 -1.783798 2 1
362
+ 354 71.1864 -5.162724 -3.657169 2 2
363
+ 355 71.2676 -2.536662 -2.116978 2 2
364
+ 356 71.3483 -3.378798 -3.135960 2 2
365
+ 357 71.4286 -3.694368 -3.352345 2 2
366
+ 358 71.5084 -2.355619 -2.172193 2 2
367
+ 359 71.5877 -6.646893 -4.039645 2 2
368
+ 360 71.3889 -2.454988 -2.823803 1 2
369
+ 361 71.4681 -2.663888 -1.255338 2 2
370
+ 362 71.5470 -5.118694 -4.861733 2 2
371
+ 363 71.3499 -2.570494 -2.467786 2 1
372
+ 364 71.4286 -5.539040 -5.872688 1 1
373
+ 365 71.5068 -5.546030 -6.133966 1 1
374
+ 366 71.3115 -3.661434 -9.179557 1 2
375
+ 367 71.3896 -5.598172 -6.330094 1 1
376
+ 368 71.4674 -3.126631 -1.457754 2 2
377
+ 369 71.2737 -3.832407 -3.844578 1 2
378
+ 370 71.3514 -5.103123 -5.276727 1 1
379
+ 371 71.4286 -4.328596 -4.522735 1 1
380
+ 372 71.2366 -2.673418 -2.341295 2 1
381
+ 373 71.3137 -5.052905 -5.238173 1 1
382
+ 374 71.3904 -1.600937 -2.722897 1 1
383
+ 375 71.4667 -3.187199 -2.544199 2 2
384
+ 376 71.5426 -6.037676 -4.472771 2 2
385
+ 377 71.3528 -3.568343 -3.777804 1 2
386
+ 378 71.1640 -3.310242 -3.458887 1 2
387
+ 379 70.9763 -0.837623 -1.774305 1 2
388
+ 380 70.7895 -8.255742 -7.608199 2 1
389
+ 381 70.8661 -11.494976 -8.678749 2 2
390
+ 382 70.9424 -2.311970 -2.284647 2 2
391
+ 383 71.0183 -2.021893 -0.679435 2 2
392
+ 384 70.8333 -3.552262 -2.532051 2 1
393
+ 385 70.6494 -1.155756 -0.998082 2 1
394
+ 386 70.7254 -1.789087 -10.189983 1 1
395
+ 387 70.8010 -1.774057 -2.057155 1 1
396
+ 388 70.6186 -5.188149 -6.053555 1 2
397
+ 389 70.6941 -2.487466 -1.614324 2 2
398
+ 390 70.7692 -2.582790 -3.696178 1 1
399
+ 391 70.5882 -3.870124 -5.339208 1 2
400
+ 392 70.4082 -5.365450 -3.388221 2 1
401
+ 393 70.4835 -7.139484 -6.416492 2 2
402
+ 394 70.5584 -1.572401 -4.415551 1 1
403
+ 395 70.3797 -3.727868 -3.809163 1 2
404
+ 396 70.2020 -5.385866 -5.130489 2 1
405
+ 397 70.0252 -7.373106 -6.998287 2 1
406
+ 398 70.1005 -3.935592 -7.223350 1 1
407
+ 399 69.9248 -4.399532 -4.365422 2 1
408
+ 400 70.0000 -2.252954 -5.818603 1 1
409
+ 401 70.0748 -2.566678 -2.767124 1 1
410
+ 402 69.9005 -4.128511 -4.370974 1 2
411
+ 403 69.9752 -6.899202 -6.886111 2 2
412
+ 404 70.0495 -3.980479 -2.911297 2 2
413
+ 405 70.1235 -2.946307 -4.110115 1 1
414
+ 406 70.1970 -8.165530 -9.444121 1 1
415
+ 407 70.2703 -5.506678 -5.424202 2 2
416
+ 408 70.3431 -3.327609 -3.213051 2 2
417
+ 409 70.4156 -3.964350 -4.321031 1 1
418
+ 410 70.4878 -5.115040 -5.714125 1 1
419
+ 411 70.5596 -4.846656 -5.366579 1 1
420
+ 412 70.6311 -4.035455 -2.699580 2 2
421
+ 413 70.7022 -3.839990 -2.809070 2 2
422
+ 414 70.5314 -4.555604 -4.785581 1 2
423
+ 415 70.6024 -2.417213 -2.137799 2 2
424
+ 416 70.6731 -4.177858 -2.257283 2 2
425
+ 417 70.7434 -4.604874 -5.575300 1 1
426
+ 418 70.8134 -5.473752 -7.076688 1 1
427
+ 419 70.6444 -4.559805 -5.043222 1 2
428
+ 420 70.4762 -3.271219 -3.520562 1 2
429
+ 421 70.5463 -4.316256 -3.700299 2 2
430
+ 422 70.3791 -2.611236 -2.243900 2 1
431
+ 423 70.2128 -3.294176 -3.225039 2 1
432
+ 424 70.2830 -5.085099 -5.700398 1 1
433
+ 425 70.3529 -4.077933 -3.100170 2 2
434
+ 426 70.1878 -1.554697 -2.013606 1 2
435
+ 427 70.2576 -3.668409 -5.051241 1 1
436
+ 428 70.0935 -4.262235 -3.310358 2 1
437
+ 429 69.9301 -3.658461 -3.687037 1 2
438
+ 430 69.7674 -5.937943 -5.045387 2 1
439
+ 431 69.8376 -4.779920 -6.051046 1 1
440
+ 432 69.6759 -4.177118 -4.660190 1 2
441
+ 433 69.7460 -1.705795 -1.543035 2 2
442
+ 434 69.8157 -7.129201 -7.716835 1 1
443
+ 435 69.8851 -3.243723 -3.745423 1 1
444
+ 436 69.9541 -0.602984 -2.206857 1 1
445
+ 437 70.0229 -3.366666 -3.214494 2 2
446
+ 438 70.0913 -3.642519 -3.630792 2 2
447
+ 439 69.9317 -3.332315 -2.716090 2 1
448
+ 440 70.0000 -4.074596 -4.828103 1 1
449
+ 441 70.0680 -3.518997 -5.034190 1 1
450
+ 442 70.1357 -5.189595 -5.405883 1 1
451
+ 443 70.2032 -1.452100 -3.620019 1 1
452
+ 444 70.0450 -5.225577 -6.389928 1 2
453
+ 445 70.1124 -1.382799 -2.685287 1 1
454
+ 446 70.1794 -4.552726 -3.730353 2 2
455
+ 447 70.0224 -1.984892 -1.769226 2 1
456
+ 448 70.0893 -3.872493 -3.302331 2 2
457
+ 449 70.1559 -2.543351 -2.541534 2 2
458
+ 450 70.2222 -4.191082 -3.224985 2 2
459
+ 451 70.2882 -4.038156 -3.286339 2 2
460
+ 452 70.3540 -5.045638 -5.208332 1 1
461
+ 453 70.4194 -5.083711 -4.693562 2 2
462
+ 454 70.4846 -2.942030 -3.392335 1 1
463
+ 455 70.3297 -1.698650 -8.711420 1 2
464
+ 456 70.3947 -4.527821 -4.222404 2 2
465
+ 457 70.2407 -10.862698 -9.811862 2 1
466
+ 458 70.3057 -3.935308 -3.534068 2 2
467
+ 459 70.3704 -4.490294 -4.298148 2 2
468
+ 460 70.4348 -0.204225 -7.264114 1 1
469
+ 461 70.4989 -2.606631 -3.684195 1 1
470
+ 462 70.3463 -5.280879 -5.068119 2 1
471
+ 463 70.4104 -1.036166 -1.044173 1 1
472
+ 464 70.2586 -2.814457 -2.760559 2 1
473
+ 465 70.3226 -2.208386 -1.980187 2 2
474
+ 466 70.3863 -5.405868 -4.912091 2 2
475
+ 467 70.4497 -5.299263 -5.600227 1 1
476
+ 468 70.5128 -5.202843 -4.105420 2 2
477
+ 469 70.3625 -1.694036 -1.268459 2 1
478
+ 470 70.4255 -1.488073 -1.302604 2 2
479
+ 471 70.4883 -2.220189 -2.300985 1 1
480
+ 472 70.5508 -4.171155 -3.556913 2 2
481
+ 473 70.6131 -4.131545 -2.565771 2 2
482
+ 474 70.6751 -4.408364 -5.086834 1 1
483
+ 475 70.7368 -1.303313 -1.103601 2 2
484
+ 476 70.7983 -5.061576 -4.998001 2 2
485
+ 477 70.6499 -4.502454 -5.143660 1 2
486
+ 478 70.5021 -4.536146 -3.895967 2 1
487
+ 479 70.5637 -3.076431 -3.268152 1 1
488
+ 480 70.6250 -5.606715 -5.398995 2 2
489
+ 481 70.6861 -2.151765 -2.678635 1 1
490
+ 482 70.5394 -4.584502 -5.326610 1 2
491
+ 483 70.3934 -5.316621 -4.398542 2 1
492
+ 484 70.2479 -2.505315 -1.545743 2 1
493
+ 485 70.3093 -1.516752 -2.542752 1 1
494
+ 486 70.3704 -2.457806 -3.108483 1 1
495
+ 487 70.4312 -2.658747 -2.035361 2 2
496
+ 488 70.4918 -3.411190 -2.448569 2 2
497
+ 489 70.5521 -1.888334 -2.256632 1 1
498
+ 490 70.6122 -3.400435 -3.907507 1 1
499
+ 491 70.6721 -5.165452 -4.263887 2 2
500
+ 492 70.7317 -5.578628 -5.048689 2 2
501
+ 493 70.7911 -1.018975 -2.437360 1 1
502
+ 494 70.8502 -4.266336 -3.981777 2 2
503
+ 495 70.9091 -4.023591 -4.157147 1 1
504
+ 496 70.7661 -3.493346 -3.533143 1 2
505
+ 497 70.6237 -5.277612 -4.720178 2 1
506
+ 498 70.4819 -1.333077 -1.937936 1 2
507
+ 499 70.5411 -2.734182 -3.307225 1 1
508
+ 500 70.6000 -5.343741 -4.051307 2 2
509
+ 501 70.6587 -0.714235 -1.514685 1 1
510
+ 502 70.7171 -1.068557 -1.706221 1 1
511
+ 503 70.7753 -2.639599 -2.617210 2 2
512
+ 504 70.8333 -2.524579 -2.746256 1 1
513
+ 505 70.6931 -8.292686 -8.780754 1 2
514
+ 506 70.5534 -5.398997 -5.131922 2 1
515
+ 507 70.6114 -4.440670 -4.545952 1 1
516
+ 508 70.6693 -6.459938 -3.301046 2 2
517
+ 509 70.7269 -3.182092 -7.977067 1 1
518
+ 510 70.5882 -3.906206 -3.373446 2 1
519
+ 511 70.6458 -5.098053 -3.719061 2 2
520
+ 512 70.7031 -3.507412 -4.081929 1 1
521
+ 513 70.7602 -3.531200 -3.381005 2 2
522
+ 514 70.6226 -5.086666 -4.519742 2 1
523
+ 515 70.6796 -6.527020 -7.064812 1 1
524
+ 516 70.7364 -4.269112 -4.366969 1 1
525
+ 517 70.5996 -1.638813 -1.513707 2 1
526
+ 518 70.6564 -2.688463 -1.832598 2 2
527
+ 519 70.7129 -9.277114 -8.908725 2 2
528
+ 520 70.7692 -2.562927 -1.916426 2 2
529
+ 521 70.8253 -0.878932 -1.142905 1 1
530
+ 522 70.8812 -3.870482 -3.200752 2 2
531
+ 523 70.7457 -4.020824 -4.167881 1 2
532
+ 524 70.6107 -6.998219 -6.910716 2 1
533
+ 525 70.6667 -5.118424 -1.967628 2 2
534
+ 526 70.7224 -0.184965 -16.954638 1 1
535
+ 527 70.7780 -6.287274 -4.334663 2 2
536
+ 528 70.8333 -6.375299 -5.847348 2 2
537
+ 529 70.8885 -4.851692 -5.904748 1 1
538
+ 530 70.9434 -5.447305 -4.076790 2 2
539
+ 531 70.8098 -3.731169 -3.425460 2 1
540
+ 532 70.8647 -2.262762 -3.062081 1 1
541
+ 533 70.9193 -3.335866 -2.807637 2 2
542
+ 534 70.9738 -2.664642 -3.347289 1 1
543
+ 535 70.8411 -3.467824 -3.590633 1 2
544
+ 536 70.7090 -2.350196 -2.166621 2 1
545
+ 537 70.7635 -5.990780 -6.403310 1 1
546
+ 538 70.8178 -2.218878 -2.832970 1 1
547
+ 539 70.8720 -4.238069 -2.565144 2 2
548
+ 540 70.7407 -2.180368 -2.636844 1 2
549
+ 541 70.7948 -2.465219 -2.688230 1 1
550
+ 542 70.6642 -3.590309 -4.343888 1 2
551
+ 543 70.7182 -5.322906 -4.651900 2 2
552
+ 544 70.7721 -2.498329 -4.059923 1 1
553
+ 545 70.6422 -1.435482 -4.677163 1 2
554
+ 546 70.6960 -2.528832 -3.327073 1 1
555
+ 547 70.7495 -6.228964 -4.995418 2 2
556
+ 548 70.8029 -4.026350 -4.483803 1 1
557
+ 549 70.6740 -5.518496 -5.402013 2 1
558
+ 550 70.5455 -3.451243 -3.465600 1 2
559
+ 551 70.4174 -1.145833 -8.153238 1 2
560
+ 552 70.4710 -2.251844 -3.009525 1 1
561
+ 553 70.5244 -5.915712 -5.262798 2 2
562
+ 554 70.3971 -4.816751 -5.169965 1 2
563
+ 555 70.4505 -2.600281 -1.822891 2 2
564
+ 556 70.5036 -5.675880 -4.423440 2 2
565
+ 557 70.5566 -4.530901 -5.215079 1 1
566
+ 558 70.6093 -1.539825 -3.324613 1 1
567
+ 559 70.6619 -2.370357 -2.337057 2 2
568
+ 560 70.5357 -2.392219 -2.087353 2 1
569
+ 561 70.5882 -1.939399 -2.400058 1 1
570
+ 562 70.6406 -0.477545 -1.436081 1 1
571
+ 563 70.5151 -1.386916 -2.634726 1 2
572
+ 564 70.5674 -2.829196 -3.415468 1 1
573
+ 565 70.6195 -3.410762 -3.272108 2 2
574
+ 566 70.6714 -3.808056 -4.998009 1 1
575
+ 567 70.7231 -2.724698 -2.558107 2 2
576
+ 568 70.7746 -6.790422 -4.455820 2 2
577
+ 569 70.6503 -5.762756 -5.166182 2 1
578
+ 570 70.7018 -5.138204 -6.587003 1 1
579
+ 571 70.7531 -4.549551 -3.030166 2 2
580
+ 572 70.8042 -2.728963 -2.599685 2 2
581
+ 573 70.6806 -3.894226 -4.390620 1 2
582
+ 574 70.7317 -3.413262 -2.619639 2 2
583
+ 575 70.7826 -4.964934 -4.509287 2 2
584
+ 576 70.8333 -2.611061 -3.261628 1 1
585
+ 577 70.8839 -4.204469 -3.971157 2 2
586
+ 578 70.9343 -3.312596 -3.093146 2 2
587
+ 579 70.8117 -5.131408 -5.177495 1 2
588
+ 580 70.8621 -1.790658 -3.384573 1 1
589
+ 581 70.9122 -6.061077 -8.283747 1 1
590
+ 582 70.7904 -5.499031 -5.730000 1 2
591
+ 583 70.8405 -4.273887 -4.092963 2 2
592
+ 584 70.7192 -4.688212 -4.563324 2 1
593
+ 585 70.7692 -0.420785 -1.242411 1 1
594
+ 586 70.8191 -3.575155 -4.021549 1 1
595
+ 587 70.6985 -4.643723 -4.425922 2 1
596
+ 588 70.7483 -4.491407 -1.947061 2 2
597
+ 589 70.6282 -3.259464 -3.820544 1 2
598
+ 590 70.6780 -3.985079 -3.509401 2 2
599
+ 591 70.5584 -3.599056 -3.842359 1 2
600
+ 592 70.6081 -3.295534 -4.244094 1 1
601
+ 593 70.6577 -4.497426 -4.424315 2 2
602
+ 594 70.5387 -5.135957 -4.349959 2 1
603
+ 595 70.5882 -1.809062 -1.791337 2 2
604
+ 596 70.6376 -5.047042 -4.803388 2 2
605
+ 597 70.6868 -2.233700 -2.314943 1 1
606
+ 598 70.7358 -3.104489 -3.449020 1 1
607
+ 599 70.7846 -3.004057 -4.386514 1 1
608
+ 600 70.8333 -3.743092 -3.666520 2 2
609
+ 601 70.8819 -3.057730 -2.642990 2 2
610
+ 602 70.9302 -2.415833 -4.602804 1 1
611
+ 603 70.9784 -6.683168 -5.012005 2 2
612
+ 604 70.8609 -2.927099 -2.962867 1 2
613
+ 605 70.7438 -2.779218 -1.812948 2 1
614
+ 606 70.7921 -3.143466 -2.916504 2 2
615
+ 607 70.6755 -2.868215 -2.580229 2 1
616
+ 608 70.7237 -2.603439 -3.064582 1 1
617
+ 609 70.6076 -7.732558 -7.299867 2 1
618
+ 610 70.6557 -2.652200 -2.904614 1 1
619
+ 611 70.5401 -4.907964 -4.846665 2 1
620
+ 612 70.5882 -4.968147 -6.546373 1 1
621
+ 613 70.6362 -3.165543 -2.133586 2 2
622
+ 614 70.6840 -5.470451 -5.176260 2 2
623
+ 615 70.7317 -2.886368 -2.597464 2 2
624
+ 616 70.6169 -5.639049 -5.759394 1 2
625
+ 617 70.6645 -11.142954 -5.644093 2 2
626
+ 618 70.5502 -3.613536 -3.884947 1 2
627
+ 619 70.5977 -3.906893 -3.462810 2 2
628
+ 620 70.4839 -4.957769 -4.157003 2 1
629
+ 621 70.5314 -3.194190 -1.938962 2 2
630
+ 622 70.5788 -3.394912 -3.822767 1 1
631
+ 623 70.6260 -2.798499 -2.887762 1 1
632
+ 624 70.6731 -4.724769 -3.787686 2 2
633
+ 625 70.7200 -3.619578 -5.985927 1 1
634
+ 626 70.6070 -1.481186 -0.821479 2 1
635
+ 627 70.6539 -3.423674 -4.173325 1 1
636
+ 628 70.7006 -2.733102 -2.232347 2 2
637
+ 629 70.7472 -1.946440 -2.511717 1 1
638
+ 630 70.7937 -0.935350 -1.573635 1 1
639
+ 631 70.8399 -2.273640 -3.643271 1 1
640
+ 632 70.8861 -1.823042 -1.266262 2 2
641
+ 633 70.9321 -1.395659 -1.272924 2 2
642
+ 634 70.9779 -2.226963 -3.416445 1 1
643
+ 635 70.8661 -4.967723 -5.204242 1 2
644
+ 636 70.9119 -0.946573 -2.843366 1 1
645
+ 637 70.9576 -3.170663 -5.364122 1 1
646
+ 638 71.0031 -5.033858 -5.663279 1 1
647
+ 639 71.0485 -1.287129 -2.022165 1 1
648
+ 640 71.0938 -1.422994 -1.410901 2 2
649
+ 641 70.9828 -7.005897 -5.732227 2 1
650
+ 642 71.0280 -4.524508 -3.274645 2 2
651
+ 643 71.0731 -1.436684 -0.768591 2 2
652
+ 644 71.1180 -3.231464 -4.205094 1 1
653
+ 645 71.1628 -1.050954 -3.240395 1 1
654
+ 646 71.0526 -4.058825 -3.726668 2 1
655
+ 647 70.9428 -3.530064 -2.286288 2 1
656
+ 648 70.9877 -3.203631 -3.055989 2 2
657
+ 649 71.0324 -3.484091 -3.871486 1 1
658
+ 650 71.0769 -3.350658 -3.930535 1 1
659
+ 651 70.9677 -5.948650 -6.159408 1 2
660
+ 652 71.0123 -5.431910 -4.771805 2 2
661
+ 653 71.0567 -5.243145 -3.742166 2 2
662
+ 654 70.9480 -3.396511 -3.864609 1 2
663
+ 655 70.8397 -2.306956 -2.395268 1 2
664
+ 656 70.8841 -4.023368 -2.514392 2 2
665
+ 657 70.7763 -5.345562 -5.569863 1 2
666
+ 658 70.8207 -4.004905 -1.713020 2 2
667
+ 659 70.8649 -2.295775 -1.597748 2 2
668
+ 660 70.7576 -3.068860 -3.724971 1 2
669
+ 661 70.8018 -3.232944 -3.486387 1 1
670
+ 662 70.8459 -3.197667 -4.243658 1 1
671
+ 663 70.8899 -6.524930 -6.374431 2 2
672
+ 664 70.7831 -3.580426 -3.449940 2 1
673
+ 665 70.6767 -3.064586 -3.164265 1 2
674
+ 666 70.7207 -3.767120 -4.780398 1 1
675
+ 667 70.7646 -8.259327 -7.020382 2 2
676
+ 668 70.8084 -1.913052 -1.987221 1 1
677
+ 669 70.8520 -2.702781 -3.252136 1 1
678
+ 670 70.7463 -3.456743 -2.984778 2 1
679
+ 671 70.7899 -3.191680 -4.623847 1 1
680
+ 672 70.8333 -2.691851 -2.412564 2 2
681
+ 673 70.7281 -1.707229 -1.734760 1 2
682
+ 674 70.6231 -5.623527 -5.041965 2 1
683
+ 675 70.5185 -3.604696 -3.521129 2 1
684
+ 676 70.4142 -2.107622 -2.751870 1 2
685
+ 677 70.4579 -3.880929 -4.270965 1 1
686
+ 678 70.3540 -5.608590 -5.935071 1 2
687
+ 679 70.3976 -5.725621 -4.785592 2 2
688
+ 680 70.4412 -3.767906 -4.644830 1 1
689
+ 681 70.4846 -3.045997 -3.216846 1 1
690
+ 682 70.5279 -2.109326 -1.950077 2 2
691
+ 683 70.5710 -1.924433 -3.062739 1 1
692
+ 684 70.6140 -5.333826 -7.655520 1 1
693
+ 685 70.6569 -4.465683 -2.269764 2 2
694
+ 686 70.6997 -5.556933 -5.139643 2 2
695
+ 687 70.7424 -3.786813 -2.742905 2 2
696
+ 688 70.7849 -8.081927 -8.119596 1 1
697
+ 689 70.8273 -2.776150 -2.253634 2 2
698
+ 690 70.7246 -2.667347 -2.809800 1 2
699
+ 691 70.6223 -2.638168 -2.041507 2 1
700
+ 692 70.6647 -1.694146 -2.672471 1 1
701
+ 693 70.7071 -4.535972 -4.967057 1 1
702
+ 694 70.7493 -3.424591 -3.329029 2 2
703
+ 695 70.7914 -3.812458 -4.652622 1 1
704
+ 696 70.8333 -4.974956 -5.531126 1 1
705
+ 697 70.8752 -2.127799 -2.619899 1 1
706
+ 698 70.9169 -4.108116 -3.734190 2 2
707
+ 699 70.9585 -3.383797 -4.557210 1 1
708
+ 700 71.0000 -4.162767 -3.631733 2 2
709
+ 701 70.8987 -2.525863 -2.241963 2 1
710
+ 702 70.7977 -4.058728 -4.995845 1 2
711
+ 703 70.8393 -3.708426 -3.851468 1 1
712
+ 704 70.8807 -1.268450 -1.630908 1 1
713
+ 705 70.7801 -3.060692 -3.438881 1 2
714
+ 706 70.8215 -3.311731 -3.204509 2 2
715
+ 707 70.7214 -4.425401 -4.331439 2 1
716
+ 708 70.7627 -4.122179 -3.129723 2 2
717
+ 709 70.8039 -2.202878 -1.797971 2 2
718
+ 710 70.8451 -3.470824 -4.340011 1 1
719
+ 711 70.7454 -1.561155 -2.060009 1 2
720
+ 712 70.6461 -2.452839 -3.779442 1 2
721
+ 713 70.6872 -3.225349 -3.154224 2 2
722
+ 714 70.7283 -3.069831 -3.032118 2 2
723
+ 715 70.7692 -4.430510 -3.635266 2 2
724
+ 716 70.6704 -4.381250 -4.390914 1 2
725
+ 717 70.7113 -1.951425 -0.184089 2 2
726
+ 718 70.7521 -4.860132 -4.994365 1 1
727
+ 719 70.6537 -5.009146 -4.711335 2 1
728
+ 720 70.5556 -1.969976 -4.643537 1 2
729
+ 721 70.4577 -4.921077 -5.849406 1 2
730
+ 722 70.3601 -2.270892 -2.042681 2 1
731
+ 723 70.4011 -2.179467 -2.593886 1 1
732
+ 724 70.4420 -4.084664 -3.988583 2 2
733
+ 725 70.3448 -2.266943 -3.025933 1 2
734
+ 726 70.3857 -4.285173 -4.133379 2 2
735
+ 727 70.4264 -2.975390 -2.383319 2 2
736
+ 728 70.3297 -3.343948 -3.206280 2 1
737
+ 729 70.3704 -1.106402 -2.260861 1 1
738
+ 730 70.4110 -3.981153 -4.473436 1 1
739
+ 731 70.3146 -4.830592 -4.719742 2 1
740
+ 732 70.3552 -3.047035 -2.283702 2 2
741
+ 733 70.3956 -6.529848 -6.298981 2 2
742
+ 734 70.4360 -1.324886 -11.519940 1 1
743
+ 735 70.3401 -3.525287 -2.835543 2 1
744
+ 736 70.3804 -1.796965 -2.916759 1 1
745
+ 737 70.4206 -4.476719 -4.937063 1 1
746
+ 738 70.3252 -4.151071 -4.624264 1 2
747
+ 739 70.3654 -0.798690 -7.214448 1 1
748
+ 740 70.4054 -4.728065 -3.271910 2 2
749
+ 741 70.3104 -4.747229 -3.938300 2 1
750
+ 742 70.3504 -2.448153 -2.092525 2 2
751
+ 743 70.2557 -3.950738 -4.021932 1 2
752
+ 744 70.2957 -4.772694 -3.964431 2 2
753
+ 745 70.3356 -2.659345 -3.565043 1 1
754
+ 746 70.3753 -7.419818 -7.421209 1 1
755
+ 747 70.4150 -5.148148 -6.809007 1 1
756
+ 748 70.4545 -4.156235 -3.190386 2 2
757
+ 749 70.4940 -3.834512 -4.919285 1 1
758
+ 750 70.5333 -4.152263 -9.600417 1 1
759
+
760
+ Final Winogrande score(750 tasks): 70.5333 +/- 1.6658
scores/Hammer2.1-7b-iq3_m.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 100.00000000
13
+ 3 100.00000000
14
+ 4 75.00000000
15
+ 5 80.00000000
16
+ 6 83.33333333
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 66.66666667
20
+ 10 70.00000000
21
+ 11 63.63636364
22
+ 12 66.66666667
23
+ 13 69.23076923
24
+ 14 71.42857143
25
+ 15 73.33333333
26
+ 16 68.75000000
27
+ 17 70.58823529
28
+ 18 66.66666667
29
+ 19 63.15789474
30
+ 20 60.00000000
31
+ 21 57.14285714
32
+ 22 59.09090909
33
+ 23 60.86956522
34
+ 24 62.50000000
35
+ 25 60.00000000
36
+ 26 57.69230769
37
+ 27 59.25925926
38
+ 28 57.14285714
39
+ 29 58.62068966
40
+ 30 60.00000000
41
+ 31 58.06451613
42
+ 32 56.25000000
43
+ 33 54.54545455
44
+ 34 52.94117647
45
+ 35 54.28571429
46
+ 36 52.77777778
47
+ 37 54.05405405
48
+ 38 52.63157895
49
+ 39 53.84615385
50
+ 40 55.00000000
51
+ 41 53.65853659
52
+ 42 52.38095238
53
+ 43 51.16279070
54
+ 44 50.00000000
55
+ 45 51.11111111
56
+ 46 52.17391304
57
+ 47 53.19148936
58
+ 48 54.16666667
59
+ 49 53.06122449
60
+ 50 52.00000000
61
+ 51 50.98039216
62
+ 52 51.92307692
63
+ 53 52.83018868
64
+ 54 51.85185185
65
+ 55 50.90909091
66
+ 56 51.78571429
67
+ 57 52.63157895
68
+ 58 53.44827586
69
+ 59 54.23728814
70
+ 60 55.00000000
71
+ 61 54.09836066
72
+ 62 54.83870968
73
+ 63 55.55555556
74
+ 64 56.25000000
75
+ 65 55.38461538
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 56.52173913
80
+ 70 55.71428571
81
+ 71 54.92957746
82
+ 72 54.16666667
83
+ 73 54.79452055
84
+ 74 55.40540541
85
+ 75 56.00000000
86
+ 76 55.26315789
87
+ 77 55.84415584
88
+ 78 56.41025641
89
+ 79 55.69620253
90
+ 80 56.25000000
91
+ 81 55.55555556
92
+ 82 56.09756098
93
+ 83 55.42168675
94
+ 84 54.76190476
95
+ 85 55.29411765
96
+ 86 55.81395349
97
+ 87 56.32183908
98
+ 88 56.81818182
99
+ 89 57.30337079
100
+ 90 56.66666667
101
+ 91 57.14285714
102
+ 92 57.60869565
103
+ 93 56.98924731
104
+ 94 56.38297872
105
+ 95 56.84210526
106
+ 96 57.29166667
107
+ 97 56.70103093
108
+ 98 56.12244898
109
+ 99 56.56565657
110
+ 100 57.00000000
111
+ 101 56.43564356
112
+ 102 56.86274510
113
+ 103 56.31067961
114
+ 104 55.76923077
115
+ 105 56.19047619
116
+ 106 55.66037736
117
+ 107 56.07476636
118
+ 108 55.55555556
119
+ 109 55.04587156
120
+ 110 55.45454545
121
+ 111 55.85585586
122
+ 112 56.25000000
123
+ 113 55.75221239
124
+ 114 56.14035088
125
+ 115 56.52173913
126
+ 116 56.03448276
127
+ 117 55.55555556
128
+ 118 55.93220339
129
+ 119 55.46218487
130
+ 120 55.83333333
131
+ 121 55.37190083
132
+ 122 55.73770492
133
+ 123 56.09756098
134
+ 124 55.64516129
135
+ 125 56.00000000
136
+ 126 56.34920635
137
+ 127 55.90551181
138
+ 128 56.25000000
139
+ 129 55.81395349
140
+ 130 55.38461538
141
+ 131 55.72519084
142
+ 132 56.06060606
143
+ 133 56.39097744
144
+ 134 56.71641791
145
+ 135 57.03703704
146
+ 136 56.61764706
147
+ 137 56.93430657
148
+ 138 57.24637681
149
+ 139 57.55395683
150
+ 140 57.85714286
151
+ 141 58.15602837
152
+ 142 57.74647887
153
+ 143 58.04195804
154
+ 144 58.33333333
155
+ 145 58.62068966
156
+ 146 58.21917808
157
+ 147 58.50340136
158
+ 148 58.10810811
159
+ 149 58.38926174
160
+ 150 58.66666667
161
+ 151 58.94039735
162
+ 152 59.21052632
163
+ 153 59.47712418
164
+ 154 59.09090909
165
+ 155 58.70967742
166
+ 156 58.97435897
167
+ 157 59.23566879
168
+ 158 58.86075949
169
+ 159 59.11949686
170
+ 160 58.75000000
171
+ 161 58.38509317
172
+ 162 58.64197531
173
+ 163 58.28220859
174
+ 164 57.92682927
175
+ 165 57.57575758
176
+ 166 57.83132530
177
+ 167 57.48502994
178
+ 168 57.73809524
179
+ 169 57.98816568
180
+ 170 58.23529412
181
+ 171 57.89473684
182
+ 172 58.13953488
183
+ 173 58.38150289
184
+ 174 58.62068966
185
+ 175 58.28571429
186
+ 176 57.95454545
187
+ 177 58.19209040
188
+ 178 57.86516854
189
+ 179 57.54189944
190
+ 180 57.22222222
191
+ 181 57.45856354
192
+ 182 57.69230769
193
+ 183 57.37704918
194
+ 184 57.60869565
195
+ 185 57.83783784
196
+ 186 58.06451613
197
+ 187 58.28877005
198
+ 188 57.97872340
199
+ 189 57.67195767
200
+ 190 57.36842105
201
+ 191 57.59162304
202
+ 192 57.29166667
203
+ 193 56.99481865
204
+ 194 57.21649485
205
+ 195 57.43589744
206
+ 196 57.65306122
207
+ 197 57.86802030
208
+ 198 57.57575758
209
+ 199 57.28643216
210
+ 200 57.50000000
211
+ 201 57.71144279
212
+ 202 57.92079208
213
+ 203 58.12807882
214
+ 204 58.33333333
215
+ 205 58.04878049
216
+ 206 57.76699029
217
+ 207 57.48792271
218
+ 208 57.69230769
219
+ 209 57.89473684
220
+ 210 58.09523810
221
+ 211 58.29383886
222
+ 212 58.01886792
223
+ 213 57.74647887
224
+ 214 57.94392523
225
+ 215 58.13953488
226
+ 216 57.87037037
227
+ 217 57.60368664
228
+ 218 57.33944954
229
+ 219 57.53424658
230
+ 220 57.27272727
231
+ 221 57.01357466
232
+ 222 57.20720721
233
+ 223 56.95067265
234
+ 224 57.14285714
235
+ 225 56.88888889
236
+ 226 56.63716814
237
+ 227 56.82819383
238
+ 228 57.01754386
239
+ 229 57.20524017
240
+ 230 57.39130435
241
+ 231 57.14285714
242
+ 232 56.89655172
243
+ 233 57.08154506
244
+ 234 56.83760684
245
+ 235 56.59574468
246
+ 236 56.77966102
247
+ 237 56.54008439
248
+ 238 56.72268908
249
+ 239 56.90376569
250
+ 240 57.08333333
251
+ 241 57.26141079
252
+ 242 57.02479339
253
+ 243 56.79012346
254
+ 244 56.55737705
255
+ 245 56.73469388
256
+ 246 56.91056911
257
+ 247 57.08502024
258
+ 248 57.25806452
259
+ 249 57.42971888
260
+ 250 57.60000000
261
+ 251 57.76892430
262
+ 252 57.93650794
263
+ 253 58.10276680
264
+ 254 57.87401575
265
+ 255 58.03921569
266
+ 256 58.20312500
267
+ 257 58.36575875
268
+ 258 58.52713178
269
+ 259 58.30115830
270
+ 260 58.46153846
271
+ 261 58.62068966
272
+ 262 58.77862595
273
+ 263 58.93536122
274
+ 264 59.09090909
275
+ 265 59.24528302
276
+ 266 59.02255639
277
+ 267 59.17602996
278
+ 268 58.95522388
279
+ 269 59.10780669
280
+ 270 59.25925926
281
+ 271 59.04059041
282
+ 272 58.82352941
283
+ 273 58.60805861
284
+ 274 58.39416058
285
+ 275 58.18181818
286
+ 276 58.33333333
287
+ 277 58.48375451
288
+ 278 58.27338129
289
+ 279 58.42293907
290
+ 280 58.57142857
291
+ 281 58.71886121
292
+ 282 58.86524823
293
+ 283 58.65724382
294
+ 284 58.80281690
295
+ 285 58.94736842
296
+ 286 58.74125874
297
+ 287 58.53658537
298
+ 288 58.68055556
299
+ 289 58.47750865
300
+ 290 58.62068966
301
+ 291 58.76288660
302
+ 292 58.56164384
303
+ 293 58.70307167
304
+ 294 58.50340136
305
+ 295 58.64406780
306
+ 296 58.78378378
307
+ 297 58.92255892
308
+ 298 59.06040268
309
+ 299 59.19732441
310
+ 300 59.00000000
311
+ 301 58.80398671
312
+ 302 58.94039735
313
+ 303 59.07590759
314
+ 304 59.21052632
315
+ 305 59.34426230
316
+ 306 59.15032680
317
+ 307 59.28338762
318
+ 308 59.41558442
319
+ 309 59.54692557
320
+ 310 59.35483871
321
+ 311 59.48553055
322
+ 312 59.29487179
323
+ 313 59.42492013
324
+ 314 59.23566879
325
+ 315 59.36507937
326
+ 316 59.49367089
327
+ 317 59.62145110
328
+ 318 59.43396226
329
+ 319 59.24764890
330
+ 320 59.06250000
331
+ 321 58.87850467
332
+ 322 59.00621118
333
+ 323 59.13312693
334
+ 324 58.95061728
335
+ 325 58.76923077
336
+ 326 58.89570552
337
+ 327 58.71559633
338
+ 328 58.84146341
339
+ 329 58.96656535
340
+ 330 59.09090909
341
+ 331 59.21450151
342
+ 332 59.33734940
343
+ 333 59.45945946
344
+ 334 59.28143713
345
+ 335 59.10447761
346
+ 336 59.22619048
347
+ 337 59.34718101
348
+ 338 59.17159763
349
+ 339 59.29203540
350
+ 340 59.41176471
351
+ 341 59.53079179
352
+ 342 59.64912281
353
+ 343 59.76676385
354
+ 344 59.88372093
355
+ 345 59.71014493
356
+ 346 59.82658960
357
+ 347 59.65417867
358
+ 348 59.48275862
359
+ 349 59.59885387
360
+ 350 59.42857143
361
+ 351 59.54415954
362
+ 352 59.65909091
363
+ 353 59.77337110
364
+ 354 59.60451977
365
+ 355 59.43661972
366
+ 356 59.26966292
367
+ 357 59.10364146
368
+ 358 59.21787709
369
+ 359 59.05292479
370
+ 360 59.16666667
371
+ 361 59.27977839
372
+ 362 59.39226519
373
+ 363 59.50413223
374
+ 364 59.34065934
375
+ 365 59.17808219
376
+ 366 59.01639344
377
+ 367 59.12806540
378
+ 368 59.23913043
379
+ 369 59.34959350
380
+ 370 59.45945946
381
+ 371 59.29919137
382
+ 372 59.13978495
383
+ 373 58.98123324
384
+ 374 59.09090909
385
+ 375 58.93333333
386
+ 376 58.77659574
387
+ 377 58.62068966
388
+ 378 58.73015873
389
+ 379 58.83905013
390
+ 380 58.94736842
391
+ 381 58.79265092
392
+ 382 58.63874346
393
+ 383 58.48563969
394
+ 384 58.59375000
395
+ 385 58.70129870
396
+ 386 58.80829016
397
+ 387 58.65633075
398
+ 388 58.50515464
399
+ 389 58.35475578
400
+ 390 58.20512821
401
+ 391 58.05626598
402
+ 392 58.16326531
403
+ 393 58.01526718
404
+ 394 58.12182741
405
+ 395 58.22784810
406
+ 396 58.08080808
407
+ 397 58.18639798
408
+ 398 58.04020101
409
+ 399 58.14536341
410
+ 400 58.25000000
411
+ 401 58.35411471
412
+ 402 58.45771144
413
+ 403 58.56079404
414
+ 404 58.66336634
415
+ 405 58.76543210
416
+ 406 58.86699507
417
+ 407 58.96805897
418
+ 408 59.06862745
419
+ 409 58.92420538
420
+ 410 59.02439024
421
+ 411 59.12408759
422
+ 412 59.22330097
423
+ 413 59.07990315
424
+ 414 59.17874396
425
+ 415 59.03614458
426
+ 416 59.13461538
427
+ 417 58.99280576
428
+ 418 59.09090909
429
+ 419 58.94988067
430
+ 420 58.80952381
431
+ 421 58.66983373
432
+ 422 58.76777251
433
+ 423 58.86524823
434
+ 424 58.72641509
435
+ 425 58.82352941
436
+ 426 58.68544601
437
+ 427 58.78220141
438
+ 428 58.64485981
439
+ 429 58.74125874
440
+ 430 58.60465116
441
+ 431 58.70069606
442
+ 432 58.79629630
443
+ 433 58.66050808
444
+ 434 58.75576037
445
+ 435 58.85057471
446
+ 436 58.94495413
447
+ 437 59.03890160
448
+ 438 58.90410959
449
+ 439 58.99772210
450
+ 440 58.86363636
451
+ 441 58.95691610
452
+ 442 58.82352941
453
+ 443 58.69074492
454
+ 444 58.78378378
455
+ 445 58.87640449
456
+ 446 58.96860987
457
+ 447 59.06040268
458
+ 448 58.92857143
459
+ 449 59.02004454
460
+ 450 58.88888889
461
+ 451 58.98004435
462
+ 452 59.07079646
463
+ 453 58.94039735
464
+ 454 58.81057269
465
+ 455 58.90109890
466
+ 456 58.77192982
467
+ 457 58.86214442
468
+ 458 58.95196507
469
+ 459 59.04139434
470
+ 460 58.91304348
471
+ 461 58.78524946
472
+ 462 58.87445887
473
+ 463 58.74730022
474
+ 464 58.62068966
475
+ 465 58.49462366
476
+ 466 58.58369099
477
+ 467 58.67237687
478
+ 468 58.76068376
479
+ 469 58.63539446
480
+ 470 58.72340426
481
+ 471 58.81104034
482
+ 472 58.68644068
483
+ 473 58.56236786
484
+ 474 58.64978903
485
+ 475 58.52631579
486
+ 476 58.61344538
487
+ 477 58.70020964
488
+ 478 58.78661088
489
+ 479 58.87265136
490
+ 480 58.95833333
491
+ 481 58.83575884
492
+ 482 58.92116183
493
+ 483 59.00621118
494
+ 484 58.88429752
495
+ 485 58.96907216
496
+ 486 59.05349794
497
+ 487 58.93223819
498
+ 488 59.01639344
499
+ 489 58.89570552
500
+ 490 58.77551020
501
+ 491 58.85947047
502
+ 492 58.94308943
503
+ 493 58.82352941
504
+ 494 58.90688259
505
+ 495 58.78787879
506
+ 496 58.87096774
507
+ 497 58.95372233
508
+ 498 59.03614458
509
+ 499 59.11823647
510
+ 500 59.20000000
511
+ 501 59.08183633
512
+ 502 59.16334661
513
+ 503 59.24453280
514
+ 504 59.32539683
515
+ 505 59.40594059
516
+ 506 59.28853755
517
+ 507 59.17159763
518
+ 508 59.05511811
519
+ 509 58.93909627
520
+ 510 59.01960784
521
+ 511 58.90410959
522
+ 512 58.98437500
523
+ 513 58.86939571
524
+ 514 58.94941634
525
+ 515 59.02912621
526
+ 516 58.91472868
527
+ 517 58.99419729
528
+ 518 59.07335907
529
+ 519 58.95953757
530
+ 520 58.84615385
531
+ 521 58.92514395
532
+ 522 58.81226054
533
+ 523 58.69980880
534
+ 524 58.58778626
535
+ 525 58.47619048
536
+ 526 58.36501901
537
+ 527 58.25426945
538
+ 528 58.14393939
539
+ 529 58.03402647
540
+ 530 58.11320755
541
+ 531 58.00376648
542
+ 532 58.08270677
543
+ 533 58.16135084
544
+ 534 58.05243446
545
+ 535 57.94392523
546
+ 536 58.02238806
547
+ 537 58.10055866
548
+ 538 57.99256506
549
+ 539 58.07050093
550
+ 540 57.96296296
551
+ 541 58.04066543
552
+ 542 57.93357934
553
+ 543 58.01104972
554
+ 544 57.90441176
555
+ 545 57.98165138
556
+ 546 57.87545788
557
+ 547 57.76965265
558
+ 548 57.84671533
559
+ 549 57.92349727
560
+ 550 58.00000000
561
+ 551 57.89473684
562
+ 552 57.78985507
563
+ 553 57.86618445
564
+ 554 57.94223827
565
+ 555 58.01801802
566
+ 556 57.91366906
567
+ 557 57.98922801
568
+ 558 57.88530466
569
+ 559 57.96064401
570
+ 560 58.03571429
571
+ 561 58.11051693
572
+ 562 58.18505338
573
+ 563 58.25932504
574
+ 564 58.15602837
575
+ 565 58.05309735
576
+ 566 58.12720848
577
+ 567 58.02469136
578
+ 568 58.09859155
579
+ 569 57.99648506
580
+ 570 57.89473684
581
+ 571 57.96847636
582
+ 572 57.86713287
583
+ 573 57.94066318
584
+ 574 58.01393728
585
+ 575 58.08695652
586
+ 576 58.15972222
587
+ 577 58.23223570
588
+ 578 58.30449827
589
+ 579 58.37651123
590
+ 580 58.27586207
591
+ 581 58.17555938
592
+ 582 58.07560137
593
+ 583 58.14751286
594
+ 584 58.21917808
595
+ 585 58.29059829
596
+ 586 58.36177474
597
+ 587 58.26235094
598
+ 588 58.16326531
599
+ 589 58.23429542
600
+ 590 58.30508475
601
+ 591 58.37563452
602
+ 592 58.27702703
603
+ 593 58.17875211
604
+ 594 58.24915825
605
+ 595 58.15126050
606
+ 596 58.05369128
607
+ 597 57.95644891
608
+ 598 57.85953177
609
+ 599 57.92988314
610
+ 600 58.00000000
611
+ 601 58.06988353
612
+ 602 57.97342193
613
+ 603 57.87728027
614
+ 604 57.78145695
615
+ 605 57.68595041
616
+ 606 57.75577558
617
+ 607 57.82537068
618
+ 608 57.89473684
619
+ 609 57.79967159
620
+ 610 57.86885246
621
+ 611 57.77414075
622
+ 612 57.67973856
623
+ 613 57.58564437
624
+ 614 57.49185668
625
+ 615 57.56097561
626
+ 616 57.46753247
627
+ 617 57.53646677
628
+ 618 57.44336570
629
+ 619 57.35056543
630
+ 620 57.41935484
631
+ 621 57.48792271
632
+ 622 57.39549839
633
+ 623 57.46388443
634
+ 624 57.37179487
635
+ 625 57.28000000
636
+ 626 57.34824281
637
+ 627 57.41626794
638
+ 628 57.32484076
639
+ 629 57.39268680
640
+ 630 57.46031746
641
+ 631 57.36925515
642
+ 632 57.27848101
643
+ 633 57.18799368
644
+ 634 57.25552050
645
+ 635 57.32283465
646
+ 636 57.38993711
647
+ 637 57.29984301
648
+ 638 57.36677116
649
+ 639 57.43348983
650
+ 640 57.34375000
651
+ 641 57.41029641
652
+ 642 57.32087227
653
+ 643 57.23172628
654
+ 644 57.29813665
655
+ 645 57.20930233
656
+ 646 57.27554180
657
+ 647 57.34157651
658
+ 648 57.25308642
659
+ 649 57.16486903
660
+ 650 57.07692308
661
+ 651 57.14285714
662
+ 652 57.05521472
663
+ 653 57.12098009
664
+ 654 57.18654434
665
+ 655 57.09923664
666
+ 656 57.01219512
667
+ 657 56.92541857
668
+ 658 56.83890578
669
+ 659 56.90440061
670
+ 660 56.96969697
671
+ 661 56.88350983
672
+ 662 56.94864048
673
+ 663 57.01357466
674
+ 664 57.07831325
675
+ 665 57.14285714
676
+ 666 57.05705706
677
+ 667 57.12143928
678
+ 668 57.03592814
679
+ 669 56.95067265
680
+ 670 57.01492537
681
+ 671 57.07898659
682
+ 672 57.14285714
683
+ 673 57.05794948
684
+ 674 57.12166172
685
+ 675 57.18518519
686
+ 676 57.10059172
687
+ 677 57.16395864
688
+ 678 57.22713864
689
+ 679 57.29013255
690
+ 680 57.35294118
691
+ 681 57.26872247
692
+ 682 57.18475073
693
+ 683 57.10102489
694
+ 684 57.16374269
695
+ 685 57.22627737
696
+ 686 57.14285714
697
+ 687 57.05967977
698
+ 688 57.12209302
699
+ 689 57.18432511
700
+ 690 57.24637681
701
+ 691 57.30824891
702
+ 692 57.36994220
703
+ 693 57.43145743
704
+ 694 57.49279539
705
+ 695 57.41007194
706
+ 696 57.32758621
707
+ 697 57.38880918
708
+ 698 57.30659026
709
+ 699 57.22460658
710
+ 700 57.28571429
711
+ 701 57.34664765
712
+ 702 57.40740741
713
+ 703 57.32574680
714
+ 704 57.38636364
715
+ 705 57.44680851
716
+ 706 57.50708215
717
+ 707 57.56718529
718
+ 708 57.62711864
719
+ 709 57.68688293
720
+ 710 57.74647887
721
+ 711 57.80590717
722
+ 712 57.86516854
723
+ 713 57.92426367
724
+ 714 57.84313725
725
+ 715 57.76223776
726
+ 716 57.82122905
727
+ 717 57.74058577
728
+ 718 57.66016713
729
+ 719 57.57997218
730
+ 720 57.63888889
731
+ 721 57.55894591
732
+ 722 57.61772853
733
+ 723 57.67634855
734
+ 724 57.73480663
735
+ 725 57.79310345
736
+ 726 57.71349862
737
+ 727 57.63411279
738
+ 728 57.69230769
739
+ 729 57.75034294
740
+ 730 57.80821918
741
+ 731 57.86593707
742
+ 732 57.92349727
743
+ 733 57.98090041
744
+ 734 58.03814714
745
+ 735 58.09523810
746
+ 736 58.15217391
747
+ 737 58.07327001
748
+ 738 57.99457995
749
+ 739 57.91610284
750
+ 740 57.83783784
751
+ 741 57.89473684
752
+ 742 57.95148248
753
+ 743 58.00807537
754
+ 744 57.93010753
755
+ 745 57.98657718
756
+ 746 58.04289544
757
+ 747 57.96519411
758
+ 748 57.88770053
759
+ 749 57.81041389
760
+ 750 57.86666667
761
+
762
+ Final result: 57.8667 +/- 1.8042
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-iq3_m.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 75.00000000
15
+ 5 80.00000000
16
+ 6 83.33333333
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 77.77777778
20
+ 10 80.00000000
21
+ 11 81.81818182
22
+ 12 75.00000000
23
+ 13 76.92307692
24
+ 14 78.57142857
25
+ 15 80.00000000
26
+ 16 81.25000000
27
+ 17 76.47058824
28
+ 18 72.22222222
29
+ 19 68.42105263
30
+ 20 70.00000000
31
+ 21 71.42857143
32
+ 22 72.72727273
33
+ 23 73.91304348
34
+ 24 70.83333333
35
+ 25 68.00000000
36
+ 26 65.38461538
37
+ 27 62.96296296
38
+ 28 64.28571429
39
+ 29 65.51724138
40
+ 30 63.33333333
41
+ 31 64.51612903
42
+ 32 62.50000000
43
+ 33 63.63636364
44
+ 34 64.70588235
45
+ 35 65.71428571
46
+ 36 63.88888889
47
+ 37 62.16216216
48
+ 38 63.15789474
49
+ 39 61.53846154
50
+ 40 62.50000000
51
+ 41 63.41463415
52
+ 42 61.90476190
53
+ 43 60.46511628
54
+ 44 61.36363636
55
+ 45 60.00000000
56
+ 46 60.86956522
57
+ 47 59.57446809
58
+ 48 58.33333333
59
+ 49 59.18367347
60
+ 50 60.00000000
61
+ 51 60.78431373
62
+ 52 59.61538462
63
+ 53 58.49056604
64
+ 54 57.40740741
65
+ 55 58.18181818
66
+ 56 57.14285714
67
+ 57 57.89473684
68
+ 58 56.89655172
69
+ 59 57.62711864
70
+ 60 58.33333333
71
+ 61 57.37704918
72
+ 62 56.45161290
73
+ 63 57.14285714
74
+ 64 57.81250000
75
+ 65 56.92307692
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 57.97101449
80
+ 70 58.57142857
81
+ 71 57.74647887
82
+ 72 58.33333333
83
+ 73 58.90410959
84
+ 74 59.45945946
85
+ 75 58.66666667
86
+ 76 59.21052632
87
+ 77 59.74025974
88
+ 78 60.25641026
89
+ 79 60.75949367
90
+ 80 61.25000000
91
+ 81 60.49382716
92
+ 82 60.97560976
93
+ 83 61.44578313
94
+ 84 61.90476190
95
+ 85 61.17647059
96
+ 86 61.62790698
97
+ 87 62.06896552
98
+ 88 62.50000000
99
+ 89 61.79775281
100
+ 90 62.22222222
101
+ 91 62.63736264
102
+ 92 61.95652174
103
+ 93 62.36559140
104
+ 94 62.76595745
105
+ 95 63.15789474
106
+ 96 63.54166667
107
+ 97 63.91752577
108
+ 98 64.28571429
109
+ 99 64.64646465
110
+ 100 64.00000000
111
+ 101 64.35643564
112
+ 102 63.72549020
113
+ 103 63.10679612
114
+ 104 63.46153846
115
+ 105 63.80952381
116
+ 106 64.15094340
117
+ 107 64.48598131
118
+ 108 64.81481481
119
+ 109 64.22018349
120
+ 110 64.54545455
121
+ 111 63.96396396
122
+ 112 64.28571429
123
+ 113 64.60176991
124
+ 114 64.91228070
125
+ 115 65.21739130
126
+ 116 64.65517241
127
+ 117 64.10256410
128
+ 118 64.40677966
129
+ 119 64.70588235
130
+ 120 64.16666667
131
+ 121 63.63636364
132
+ 122 63.93442623
133
+ 123 64.22764228
134
+ 124 63.70967742
135
+ 125 64.00000000
136
+ 126 64.28571429
137
+ 127 64.56692913
138
+ 128 64.06250000
139
+ 129 64.34108527
140
+ 130 63.84615385
141
+ 131 64.12213740
142
+ 132 63.63636364
143
+ 133 63.90977444
144
+ 134 64.17910448
145
+ 135 64.44444444
146
+ 136 64.70588235
147
+ 137 64.23357664
148
+ 138 64.49275362
149
+ 139 64.74820144
150
+ 140 65.00000000
151
+ 141 65.24822695
152
+ 142 65.49295775
153
+ 143 65.73426573
154
+ 144 65.97222222
155
+ 145 66.20689655
156
+ 146 66.43835616
157
+ 147 66.66666667
158
+ 148 66.89189189
159
+ 149 67.11409396
160
+ 150 67.33333333
161
+ 151 66.88741722
162
+ 152 66.44736842
163
+ 153 66.66666667
164
+ 154 66.88311688
165
+ 155 67.09677419
166
+ 156 66.66666667
167
+ 157 66.24203822
168
+ 158 66.45569620
169
+ 159 66.03773585
170
+ 160 66.25000000
171
+ 161 66.45962733
172
+ 162 66.66666667
173
+ 163 66.87116564
174
+ 164 67.07317073
175
+ 165 67.27272727
176
+ 166 67.46987952
177
+ 167 67.66467066
178
+ 168 67.85714286
179
+ 169 67.45562130
180
+ 170 67.64705882
181
+ 171 67.83625731
182
+ 172 68.02325581
183
+ 173 67.63005780
184
+ 174 67.24137931
185
+ 175 67.42857143
186
+ 176 67.61363636
187
+ 177 67.79661017
188
+ 178 67.97752809
189
+ 179 67.59776536
190
+ 180 67.77777778
191
+ 181 67.40331492
192
+ 182 67.58241758
193
+ 183 67.75956284
194
+ 184 67.39130435
195
+ 185 67.56756757
196
+ 186 67.20430108
197
+ 187 66.84491979
198
+ 188 66.48936170
199
+ 189 66.13756614
200
+ 190 66.31578947
201
+ 191 66.49214660
202
+ 192 66.66666667
203
+ 193 66.32124352
204
+ 194 66.49484536
205
+ 195 66.15384615
206
+ 196 65.81632653
207
+ 197 65.48223350
208
+ 198 65.15151515
209
+ 199 65.32663317
210
+ 200 65.50000000
211
+ 201 65.67164179
212
+ 202 65.84158416
213
+ 203 65.51724138
214
+ 204 65.68627451
215
+ 205 65.36585366
216
+ 206 65.53398058
217
+ 207 65.70048309
218
+ 208 65.38461538
219
+ 209 65.55023923
220
+ 210 65.71428571
221
+ 211 65.87677725
222
+ 212 66.03773585
223
+ 213 66.19718310
224
+ 214 66.35514019
225
+ 215 66.04651163
226
+ 216 66.20370370
227
+ 217 65.89861751
228
+ 218 65.59633028
229
+ 219 65.75342466
230
+ 220 65.90909091
231
+ 221 65.61085973
232
+ 222 65.76576577
233
+ 223 65.91928251
234
+ 224 66.07142857
235
+ 225 66.22222222
236
+ 226 66.37168142
237
+ 227 66.07929515
238
+ 228 66.22807018
239
+ 229 66.37554585
240
+ 230 66.08695652
241
+ 231 65.80086580
242
+ 232 65.51724138
243
+ 233 65.23605150
244
+ 234 65.38461538
245
+ 235 65.10638298
246
+ 236 64.83050847
247
+ 237 64.97890295
248
+ 238 65.12605042
249
+ 239 65.27196653
250
+ 240 65.41666667
251
+ 241 65.56016598
252
+ 242 65.70247934
253
+ 243 65.84362140
254
+ 244 65.57377049
255
+ 245 65.30612245
256
+ 246 65.44715447
257
+ 247 65.18218623
258
+ 248 65.32258065
259
+ 249 65.46184739
260
+ 250 65.60000000
261
+ 251 65.73705179
262
+ 252 65.87301587
263
+ 253 65.61264822
264
+ 254 65.74803150
265
+ 255 65.88235294
266
+ 256 66.01562500
267
+ 257 65.75875486
268
+ 258 65.89147287
269
+ 259 66.02316602
270
+ 260 66.15384615
271
+ 261 66.28352490
272
+ 262 66.03053435
273
+ 263 66.15969582
274
+ 264 66.28787879
275
+ 265 66.41509434
276
+ 266 66.16541353
277
+ 267 66.29213483
278
+ 268 66.41791045
279
+ 269 66.54275093
280
+ 270 66.66666667
281
+ 271 66.42066421
282
+ 272 66.54411765
283
+ 273 66.30036630
284
+ 274 66.42335766
285
+ 275 66.18181818
286
+ 276 66.30434783
287
+ 277 66.42599278
288
+ 278 66.54676259
289
+ 279 66.66666667
290
+ 280 66.78571429
291
+ 281 66.90391459
292
+ 282 67.02127660
293
+ 283 66.78445230
294
+ 284 66.90140845
295
+ 285 66.66666667
296
+ 286 66.43356643
297
+ 287 66.55052265
298
+ 288 66.66666667
299
+ 289 66.78200692
300
+ 290 66.89655172
301
+ 291 67.01030928
302
+ 292 66.78082192
303
+ 293 66.89419795
304
+ 294 67.00680272
305
+ 295 67.11864407
306
+ 296 67.22972973
307
+ 297 67.34006734
308
+ 298 67.44966443
309
+ 299 67.22408027
310
+ 300 67.00000000
311
+ 301 66.77740864
312
+ 302 66.88741722
313
+ 303 66.66666667
314
+ 304 66.44736842
315
+ 305 66.55737705
316
+ 306 66.66666667
317
+ 307 66.44951140
318
+ 308 66.55844156
319
+ 309 66.66666667
320
+ 310 66.77419355
321
+ 311 66.88102894
322
+ 312 66.98717949
323
+ 313 67.09265176
324
+ 314 67.19745223
325
+ 315 67.30158730
326
+ 316 67.40506329
327
+ 317 67.50788644
328
+ 318 67.29559748
329
+ 319 67.08463950
330
+ 320 67.18750000
331
+ 321 66.97819315
332
+ 322 66.77018634
333
+ 323 66.87306502
334
+ 324 66.97530864
335
+ 325 67.07692308
336
+ 326 66.87116564
337
+ 327 66.66666667
338
+ 328 66.76829268
339
+ 329 66.86930091
340
+ 330 66.96969697
341
+ 331 67.06948640
342
+ 332 66.86746988
343
+ 333 66.96696697
344
+ 334 66.76646707
345
+ 335 66.86567164
346
+ 336 66.96428571
347
+ 337 66.76557864
348
+ 338 66.56804734
349
+ 339 66.66666667
350
+ 340 66.76470588
351
+ 341 66.86217009
352
+ 342 66.95906433
353
+ 343 66.76384840
354
+ 344 66.56976744
355
+ 345 66.66666667
356
+ 346 66.47398844
357
+ 347 66.57060519
358
+ 348 66.37931034
359
+ 349 66.18911175
360
+ 350 66.28571429
361
+ 351 66.38176638
362
+ 352 66.47727273
363
+ 353 66.28895184
364
+ 354 66.38418079
365
+ 355 66.47887324
366
+ 356 66.57303371
367
+ 357 66.66666667
368
+ 358 66.75977654
369
+ 359 66.85236769
370
+ 360 66.66666667
371
+ 361 66.75900277
372
+ 362 66.85082873
373
+ 363 66.94214876
374
+ 364 67.03296703
375
+ 365 67.12328767
376
+ 366 67.21311475
377
+ 367 67.30245232
378
+ 368 67.39130435
379
+ 369 67.47967480
380
+ 370 67.29729730
381
+ 371 67.38544474
382
+ 372 67.47311828
383
+ 373 67.29222520
384
+ 374 67.11229947
385
+ 375 67.20000000
386
+ 376 67.28723404
387
+ 377 67.10875332
388
+ 378 67.19576720
389
+ 379 67.28232190
390
+ 380 67.36842105
391
+ 381 67.45406824
392
+ 382 67.27748691
393
+ 383 67.36292428
394
+ 384 67.44791667
395
+ 385 67.53246753
396
+ 386 67.61658031
397
+ 387 67.44186047
398
+ 388 67.26804124
399
+ 389 67.35218509
400
+ 390 67.43589744
401
+ 391 67.26342711
402
+ 392 67.34693878
403
+ 393 67.43002545
404
+ 394 67.51269036
405
+ 395 67.59493671
406
+ 396 67.67676768
407
+ 397 67.75818640
408
+ 398 67.83919598
409
+ 399 67.91979950
410
+ 400 67.75000000
411
+ 401 67.83042394
412
+ 402 67.66169154
413
+ 403 67.49379653
414
+ 404 67.57425743
415
+ 405 67.40740741
416
+ 406 67.48768473
417
+ 407 67.56756757
418
+ 408 67.64705882
419
+ 409 67.72616137
420
+ 410 67.56097561
421
+ 411 67.39659367
422
+ 412 67.23300971
423
+ 413 67.31234867
424
+ 414 67.39130435
425
+ 415 67.22891566
426
+ 416 67.30769231
427
+ 417 67.38609113
428
+ 418 67.46411483
429
+ 419 67.54176611
430
+ 420 67.61904762
431
+ 421 67.45843230
432
+ 422 67.53554502
433
+ 423 67.61229314
434
+ 424 67.68867925
435
+ 425 67.76470588
436
+ 426 67.84037559
437
+ 427 67.68149883
438
+ 428 67.75700935
439
+ 429 67.59906760
440
+ 430 67.67441860
441
+ 431 67.74941995
442
+ 432 67.82407407
443
+ 433 67.89838337
444
+ 434 67.97235023
445
+ 435 67.81609195
446
+ 436 67.88990826
447
+ 437 67.73455378
448
+ 438 67.80821918
449
+ 439 67.88154897
450
+ 440 67.95454545
451
+ 441 68.02721088
452
+ 442 68.09954751
453
+ 443 68.17155756
454
+ 444 68.24324324
455
+ 445 68.31460674
456
+ 446 68.38565022
457
+ 447 68.45637584
458
+ 448 68.30357143
459
+ 449 68.15144766
460
+ 450 68.22222222
461
+ 451 68.07095344
462
+ 452 68.14159292
463
+ 453 68.21192053
464
+ 454 68.28193833
465
+ 455 68.35164835
466
+ 456 68.42105263
467
+ 457 68.49015317
468
+ 458 68.55895197
469
+ 459 68.62745098
470
+ 460 68.69565217
471
+ 461 68.76355748
472
+ 462 68.83116883
473
+ 463 68.89848812
474
+ 464 68.96551724
475
+ 465 68.81720430
476
+ 466 68.88412017
477
+ 467 68.95074946
478
+ 468 68.80341880
479
+ 469 68.65671642
480
+ 470 68.51063830
481
+ 471 68.57749469
482
+ 472 68.43220339
483
+ 473 68.49894292
484
+ 474 68.56540084
485
+ 475 68.63157895
486
+ 476 68.48739496
487
+ 477 68.34381551
488
+ 478 68.41004184
489
+ 479 68.47599165
490
+ 480 68.54166667
491
+ 481 68.39916840
492
+ 482 68.25726141
493
+ 483 68.32298137
494
+ 484 68.38842975
495
+ 485 68.24742268
496
+ 486 68.31275720
497
+ 487 68.37782341
498
+ 488 68.23770492
499
+ 489 68.30265849
500
+ 490 68.36734694
501
+ 491 68.43177189
502
+ 492 68.49593496
503
+ 493 68.55983773
504
+ 494 68.62348178
505
+ 495 68.68686869
506
+ 496 68.75000000
507
+ 497 68.81287726
508
+ 498 68.87550201
509
+ 499 68.73747495
510
+ 500 68.80000000
511
+ 501 68.86227545
512
+ 502 68.92430279
513
+ 503 68.98608350
514
+ 504 69.04761905
515
+ 505 69.10891089
516
+ 506 69.16996047
517
+ 507 69.03353057
518
+ 508 69.09448819
519
+ 509 69.15520629
520
+ 510 69.21568627
521
+ 511 69.27592955
522
+ 512 69.14062500
523
+ 513 69.00584795
524
+ 514 69.06614786
525
+ 515 69.12621359
526
+ 516 68.99224806
527
+ 517 68.85880077
528
+ 518 68.91891892
529
+ 519 68.78612717
530
+ 520 68.84615385
531
+ 521 68.71401152
532
+ 522 68.58237548
533
+ 523 68.64244742
534
+ 524 68.70229008
535
+ 525 68.76190476
536
+ 526 68.63117871
537
+ 527 68.50094877
538
+ 528 68.56060606
539
+ 529 68.43100189
540
+ 530 68.49056604
541
+ 531 68.54990584
542
+ 532 68.60902256
543
+ 533 68.48030019
544
+ 534 68.53932584
545
+ 535 68.59813084
546
+ 536 68.65671642
547
+ 537 68.71508380
548
+ 538 68.77323420
549
+ 539 68.64564007
550
+ 540 68.70370370
551
+ 541 68.76155268
552
+ 542 68.81918819
553
+ 543 68.87661142
554
+ 544 68.93382353
555
+ 545 68.99082569
556
+ 546 69.04761905
557
+ 547 69.10420475
558
+ 548 69.16058394
559
+ 549 69.21675774
560
+ 550 69.09090909
561
+ 551 69.14700544
562
+ 552 69.02173913
563
+ 553 69.07775769
564
+ 554 69.13357401
565
+ 555 69.18918919
566
+ 556 69.24460432
567
+ 557 69.12028725
568
+ 558 68.99641577
569
+ 559 68.87298748
570
+ 560 68.92857143
571
+ 561 68.98395722
572
+ 562 68.86120996
573
+ 563 68.91651865
574
+ 564 68.79432624
575
+ 565 68.67256637
576
+ 566 68.55123675
577
+ 567 68.43033510
578
+ 568 68.48591549
579
+ 569 68.36555360
580
+ 570 68.42105263
581
+ 571 68.47635727
582
+ 572 68.53146853
583
+ 573 68.58638743
584
+ 574 68.64111498
585
+ 575 68.69565217
586
+ 576 68.75000000
587
+ 577 68.63084922
588
+ 578 68.68512111
589
+ 579 68.73920553
590
+ 580 68.62068966
591
+ 581 68.67469880
592
+ 582 68.72852234
593
+ 583 68.78216123
594
+ 584 68.83561644
595
+ 585 68.88888889
596
+ 586 68.77133106
597
+ 587 68.82453152
598
+ 588 68.70748299
599
+ 589 68.59083192
600
+ 590 68.47457627
601
+ 591 68.52791878
602
+ 592 68.58108108
603
+ 593 68.46543002
604
+ 594 68.35016835
605
+ 595 68.40336134
606
+ 596 68.45637584
607
+ 597 68.50921273
608
+ 598 68.56187291
609
+ 599 68.44741235
610
+ 600 68.50000000
611
+ 601 68.55241265
612
+ 602 68.60465116
613
+ 603 68.49087894
614
+ 604 68.54304636
615
+ 605 68.59504132
616
+ 606 68.48184818
617
+ 607 68.53377265
618
+ 608 68.42105263
619
+ 609 68.47290640
620
+ 610 68.52459016
621
+ 611 68.57610475
622
+ 612 68.46405229
623
+ 613 68.51549755
624
+ 614 68.40390879
625
+ 615 68.45528455
626
+ 616 68.34415584
627
+ 617 68.39546191
628
+ 618 68.44660194
629
+ 619 68.33602585
630
+ 620 68.38709677
631
+ 621 68.27697262
632
+ 622 68.32797428
633
+ 623 68.37881220
634
+ 624 68.42948718
635
+ 625 68.48000000
636
+ 626 68.37060703
637
+ 627 68.42105263
638
+ 628 68.47133758
639
+ 629 68.36248013
640
+ 630 68.41269841
641
+ 631 68.46275753
642
+ 632 68.35443038
643
+ 633 68.40442338
644
+ 634 68.45425868
645
+ 635 68.50393701
646
+ 636 68.55345912
647
+ 637 68.60282575
648
+ 638 68.65203762
649
+ 639 68.70109546
650
+ 640 68.59375000
651
+ 641 68.64274571
652
+ 642 68.69158879
653
+ 643 68.74027994
654
+ 644 68.63354037
655
+ 645 68.68217054
656
+ 646 68.73065015
657
+ 647 68.77897991
658
+ 648 68.82716049
659
+ 649 68.72110940
660
+ 650 68.76923077
661
+ 651 68.81720430
662
+ 652 68.86503067
663
+ 653 68.91271057
664
+ 654 68.96024465
665
+ 655 69.00763359
666
+ 656 69.05487805
667
+ 657 69.10197869
668
+ 658 69.14893617
669
+ 659 69.04400607
670
+ 660 69.09090909
671
+ 661 69.13767020
672
+ 662 69.18429003
673
+ 663 69.23076923
674
+ 664 69.27710843
675
+ 665 69.32330827
676
+ 666 69.36936937
677
+ 667 69.41529235
678
+ 668 69.46107784
679
+ 669 69.50672646
680
+ 670 69.55223881
681
+ 671 69.59761550
682
+ 672 69.64285714
683
+ 673 69.68796434
684
+ 674 69.58456973
685
+ 675 69.62962963
686
+ 676 69.52662722
687
+ 677 69.57163959
688
+ 678 69.61651917
689
+ 679 69.66126657
690
+ 680 69.70588235
691
+ 681 69.60352423
692
+ 682 69.64809384
693
+ 683 69.69253294
694
+ 684 69.73684211
695
+ 685 69.63503650
696
+ 686 69.53352770
697
+ 687 69.57787482
698
+ 688 69.62209302
699
+ 689 69.66618287
700
+ 690 69.71014493
701
+ 691 69.75397974
702
+ 692 69.79768786
703
+ 693 69.84126984
704
+ 694 69.88472622
705
+ 695 69.92805755
706
+ 696 69.97126437
707
+ 697 70.01434720
708
+ 698 70.05730659
709
+ 699 70.10014306
710
+ 700 70.14285714
711
+ 701 70.18544936
712
+ 702 70.22792023
713
+ 703 70.27027027
714
+ 704 70.31250000
715
+ 705 70.35460993
716
+ 706 70.39660057
717
+ 707 70.43847242
718
+ 708 70.48022599
719
+ 709 70.52186178
720
+ 710 70.56338028
721
+ 711 70.60478200
722
+ 712 70.64606742
723
+ 713 70.68723703
724
+ 714 70.72829132
725
+ 715 70.76923077
726
+ 716 70.81005587
727
+ 717 70.85076709
728
+ 718 70.89136490
729
+ 719 70.79276773
730
+ 720 70.83333333
731
+ 721 70.73509015
732
+ 722 70.77562327
733
+ 723 70.81604426
734
+ 724 70.85635359
735
+ 725 70.89655172
736
+ 726 70.93663912
737
+ 727 70.97661623
738
+ 728 71.01648352
739
+ 729 71.05624143
740
+ 730 70.95890411
741
+ 731 70.99863201
742
+ 732 70.90163934
743
+ 733 70.94133697
744
+ 734 70.98092643
745
+ 735 71.02040816
746
+ 736 71.05978261
747
+ 737 70.96336499
748
+ 738 70.86720867
749
+ 739 70.77131258
750
+ 740 70.81081081
751
+ 741 70.85020243
752
+ 742 70.88948787
753
+ 743 70.92866756
754
+ 744 70.96774194
755
+ 745 70.87248322
756
+ 746 70.91152815
757
+ 747 70.81659973
758
+ 748 70.85561497
759
+ 749 70.89452603
760
+ 750 70.93333333
scores/Hammer2.1-7b-iq3_m.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 0.00000000
13
+ 3 0.00000000
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 28.57142857
18
+ 8 37.50000000
19
+ 9 33.33333333
20
+ 10 30.00000000
21
+ 11 27.27272727
22
+ 12 25.00000000
23
+ 13 30.76923077
24
+ 14 28.57142857
25
+ 15 26.66666667
26
+ 16 31.25000000
27
+ 17 35.29411765
28
+ 18 33.33333333
29
+ 19 36.84210526
30
+ 20 35.00000000
31
+ 21 33.33333333
32
+ 22 31.81818182
33
+ 23 30.43478261
34
+ 24 33.33333333
35
+ 25 36.00000000
36
+ 26 34.61538462
37
+ 27 37.03703704
38
+ 28 35.71428571
39
+ 29 37.93103448
40
+ 30 36.66666667
41
+ 31 38.70967742
42
+ 32 40.62500000
43
+ 33 39.39393939
44
+ 34 41.17647059
45
+ 35 42.85714286
46
+ 36 41.66666667
47
+ 37 40.54054054
48
+ 38 39.47368421
49
+ 39 38.46153846
50
+ 40 37.50000000
51
+ 41 36.58536585
52
+ 42 35.71428571
53
+ 43 37.20930233
54
+ 44 36.36363636
55
+ 45 35.55555556
56
+ 46 34.78260870
57
+ 47 34.04255319
58
+ 48 35.41666667
59
+ 49 36.73469388
60
+ 50 36.00000000
61
+ 51 35.29411765
62
+ 52 34.61538462
63
+ 53 33.96226415
64
+ 54 33.33333333
65
+ 55 34.54545455
66
+ 56 35.71428571
67
+ 57 35.08771930
68
+ 58 36.20689655
69
+ 59 37.28813559
70
+ 60 36.66666667
71
+ 61 36.06557377
72
+ 62 35.48387097
73
+ 63 34.92063492
74
+ 64 34.37500000
75
+ 65 35.38461538
76
+ 66 36.36363636
77
+ 67 35.82089552
78
+ 68 35.29411765
79
+ 69 34.78260870
80
+ 70 35.71428571
81
+ 71 36.61971831
82
+ 72 36.11111111
83
+ 73 35.61643836
84
+ 74 36.48648649
85
+ 75 37.33333333
86
+ 76 38.15789474
87
+ 77 38.96103896
88
+ 78 39.74358974
89
+ 79 39.24050633
90
+ 80 40.00000000
91
+ 81 40.74074074
92
+ 82 40.24390244
93
+ 83 40.96385542
94
+ 84 41.66666667
95
+ 85 42.35294118
96
+ 86 41.86046512
97
+ 87 42.52873563
98
+ 88 43.18181818
99
+ 89 42.69662921
100
+ 90 42.22222222
101
+ 91 41.75824176
102
+ 92 41.30434783
103
+ 93 40.86021505
104
+ 94 41.48936170
105
+ 95 41.05263158
106
+ 96 41.66666667
107
+ 97 41.23711340
108
+ 98 40.81632653
109
+ 99 40.40404040
110
+ 100 40.00000000
111
+ 101 40.59405941
112
+ 102 40.19607843
113
+ 103 39.80582524
114
+ 104 40.38461538
115
+ 105 40.00000000
116
+ 106 40.56603774
117
+ 107 40.18691589
118
+ 108 40.74074074
119
+ 109 40.36697248
120
+ 110 40.00000000
121
+ 111 39.63963964
122
+ 112 40.17857143
123
+ 113 39.82300885
124
+ 114 40.35087719
125
+ 115 40.86956522
126
+ 116 40.51724138
127
+ 117 40.17094017
128
+ 118 39.83050847
129
+ 119 39.49579832
130
+ 120 39.16666667
131
+ 121 39.66942149
132
+ 122 39.34426230
133
+ 123 39.02439024
134
+ 124 38.70967742
135
+ 125 39.20000000
136
+ 126 38.88888889
137
+ 127 38.58267717
138
+ 128 38.28125000
139
+ 129 37.98449612
140
+ 130 37.69230769
141
+ 131 38.16793893
142
+ 132 37.87878788
143
+ 133 37.59398496
144
+ 134 37.31343284
145
+ 135 37.03703704
146
+ 136 37.50000000
147
+ 137 37.95620438
148
+ 138 37.68115942
149
+ 139 38.12949640
150
+ 140 38.57142857
151
+ 141 38.29787234
152
+ 142 38.73239437
153
+ 143 39.16083916
154
+ 144 38.88888889
155
+ 145 38.62068966
156
+ 146 39.04109589
157
+ 147 38.77551020
158
+ 148 39.18918919
159
+ 149 39.59731544
160
+ 150 39.33333333
161
+ 151 39.07284768
162
+ 152 39.47368421
163
+ 153 39.21568627
164
+ 154 38.96103896
165
+ 155 38.70967742
166
+ 156 38.46153846
167
+ 157 38.85350318
168
+ 158 39.24050633
169
+ 159 38.99371069
170
+ 160 39.37500000
171
+ 161 39.13043478
172
+ 162 38.88888889
173
+ 163 38.65030675
174
+ 164 38.41463415
175
+ 165 38.78787879
176
+ 166 38.55421687
177
+ 167 38.92215569
178
+ 168 38.69047619
179
+ 169 39.05325444
180
+ 170 38.82352941
181
+ 171 38.59649123
182
+ 172 38.37209302
183
+ 173 38.15028902
184
+ 174 38.50574713
185
+ 175 38.28571429
186
+ 176 38.06818182
187
+ 177 37.85310734
188
+ 178 37.64044944
189
+ 179 37.98882682
190
+ 180 37.77777778
191
+ 181 37.56906077
192
+ 182 37.36263736
193
+ 183 37.70491803
194
+ 184 37.50000000
195
+ 185 37.83783784
196
+ 186 38.17204301
197
+ 187 37.96791444
198
+ 188 37.76595745
199
+ 189 38.09523810
200
+ 190 37.89473684
201
+ 191 38.21989529
202
+ 192 38.02083333
203
+ 193 37.82383420
204
+ 194 38.14432990
205
+ 195 38.46153846
206
+ 196 38.26530612
207
+ 197 38.57868020
208
+ 198 38.38383838
209
+ 199 38.19095477
210
+ 200 38.00000000
211
+ 201 37.81094527
212
+ 202 37.62376238
213
+ 203 37.43842365
214
+ 204 37.25490196
215
+ 205 37.56097561
216
+ 206 37.37864078
217
+ 207 37.68115942
218
+ 208 37.50000000
219
+ 209 37.32057416
220
+ 210 37.14285714
221
+ 211 37.44075829
222
+ 212 37.26415094
223
+ 213 37.08920188
224
+ 214 36.91588785
225
+ 215 37.20930233
226
+ 216 37.50000000
227
+ 217 37.32718894
228
+ 218 37.61467890
229
+ 219 37.89954338
230
+ 220 38.18181818
231
+ 221 38.00904977
232
+ 222 38.28828829
233
+ 223 38.56502242
234
+ 224 38.83928571
235
+ 225 39.11111111
236
+ 226 39.38053097
237
+ 227 39.20704846
238
+ 228 39.47368421
239
+ 229 39.30131004
240
+ 230 39.13043478
241
+ 231 38.96103896
242
+ 232 38.79310345
243
+ 233 38.62660944
244
+ 234 38.88888889
245
+ 235 38.72340426
246
+ 236 38.55932203
247
+ 237 38.39662447
248
+ 238 38.23529412
249
+ 239 38.07531381
250
+ 240 37.91666667
251
+ 241 38.17427386
252
+ 242 38.42975207
253
+ 243 38.68312757
254
+ 244 38.52459016
255
+ 245 38.77551020
256
+ 246 38.61788618
257
+ 247 38.86639676
258
+ 248 39.11290323
259
+ 249 38.95582329
260
+ 250 38.80000000
261
+ 251 38.64541833
262
+ 252 38.49206349
263
+ 253 38.33992095
264
+ 254 38.18897638
265
+ 255 38.03921569
266
+ 256 37.89062500
267
+ 257 38.13229572
268
+ 258 37.98449612
269
+ 259 37.83783784
270
+ 260 37.69230769
271
+ 261 37.54789272
272
+ 262 37.40458015
273
+ 263 37.26235741
274
+ 264 37.50000000
275
+ 265 37.35849057
276
+ 266 37.59398496
277
+ 267 37.45318352
278
+ 268 37.31343284
279
+ 269 37.54646840
280
+ 270 37.77777778
281
+ 271 37.63837638
282
+ 272 37.50000000
283
+ 273 37.36263736
284
+ 274 37.22627737
285
+ 275 37.09090909
286
+ 276 37.31884058
287
+ 277 37.54512635
288
+ 278 37.76978417
289
+ 279 37.63440860
290
+ 280 37.85714286
291
+ 281 37.72241993
292
+ 282 37.58865248
293
+ 283 37.45583039
294
+ 284 37.32394366
295
+ 285 37.19298246
296
+ 286 37.06293706
297
+ 287 37.28222997
298
+ 288 37.15277778
299
+ 289 37.02422145
300
+ 290 37.24137931
301
+ 291 37.45704467
302
+ 292 37.67123288
303
+ 293 37.88395904
304
+ 294 37.75510204
305
+ 295 37.62711864
306
+ 296 37.50000000
307
+ 297 37.37373737
308
+ 298 37.58389262
309
+ 299 37.79264214
310
+ 300 38.00000000
311
+ 301 37.87375415
312
+ 302 38.07947020
313
+ 303 37.95379538
314
+ 304 37.82894737
315
+ 305 38.03278689
316
+ 306 37.90849673
317
+ 307 38.11074919
318
+ 308 37.98701299
319
+ 309 37.86407767
320
+ 310 37.74193548
321
+ 311 37.94212219
322
+ 312 38.14102564
323
+ 313 38.33865815
324
+ 314 38.53503185
325
+ 315 38.41269841
326
+ 316 38.29113924
327
+ 317 38.48580442
328
+ 318 38.36477987
329
+ 319 38.55799373
330
+ 320 38.43750000
331
+ 321 38.31775701
332
+ 322 38.19875776
333
+ 323 38.08049536
334
+ 324 37.96296296
335
+ 325 38.15384615
336
+ 326 38.03680982
337
+ 327 38.22629969
338
+ 328 38.41463415
339
+ 329 38.60182371
340
+ 330 38.48484848
341
+ 331 38.36858006
342
+ 332 38.55421687
343
+ 333 38.43843844
344
+ 334 38.32335329
345
+ 335 38.50746269
346
+ 336 38.69047619
347
+ 337 38.57566766
348
+ 338 38.46153846
349
+ 339 38.34808260
350
+ 340 38.52941176
351
+ 341 38.70967742
352
+ 342 38.88888889
353
+ 343 38.77551020
354
+ 344 38.66279070
355
+ 345 38.55072464
356
+ 346 38.72832370
357
+ 347 38.90489914
358
+ 348 38.79310345
359
+ 349 38.96848138
360
+ 350 39.14285714
361
+ 351 39.03133903
362
+ 352 38.92045455
363
+ 353 39.09348442
364
+ 354 39.26553672
365
+ 355 39.15492958
366
+ 356 39.32584270
367
+ 357 39.49579832
368
+ 358 39.38547486
369
+ 359 39.27576602
370
+ 360 39.44444444
371
+ 361 39.33518006
372
+ 362 39.50276243
373
+ 363 39.66942149
374
+ 364 39.56043956
375
+ 365 39.45205479
376
+ 366 39.61748634
377
+ 367 39.50953678
378
+ 368 39.40217391
379
+ 369 39.29539295
380
+ 370 39.45945946
381
+ 371 39.62264151
382
+ 372 39.51612903
383
+ 373 39.41018767
384
+ 374 39.57219251
385
+ 375 39.73333333
386
+ 376 39.62765957
387
+ 377 39.52254642
388
+ 378 39.68253968
389
+ 379 39.57783641
390
+ 380 39.47368421
391
+ 381 39.63254593
392
+ 382 39.79057592
393
+ 383 39.94778068
394
+ 384 39.84375000
395
+ 385 40.00000000
396
+ 386 39.89637306
397
+ 387 39.79328165
398
+ 388 39.69072165
399
+ 389 39.84575835
400
+ 390 40.00000000
401
+ 391 40.15345269
402
+ 392 40.30612245
403
+ 393 40.20356234
404
+ 394 40.10152284
405
+ 395 40.25316456
406
+ 396 40.40404040
407
+ 397 40.55415617
408
+ 398 40.45226131
409
+ 399 40.60150376
410
+ 400 40.50000000
411
+ 401 40.64837905
412
+ 402 40.54726368
413
+ 403 40.44665012
414
+ 404 40.34653465
415
+ 405 40.24691358
416
+ 406 40.14778325
417
+ 407 40.04914005
418
+ 408 39.95098039
419
+ 409 40.09779951
420
+ 410 40.00000000
421
+ 411 39.90267640
422
+ 412 39.80582524
423
+ 413 39.70944310
424
+ 414 39.61352657
425
+ 415 39.75903614
426
+ 416 39.66346154
427
+ 417 39.56834532
428
+ 418 39.47368421
429
+ 419 39.61813842
430
+ 420 39.76190476
431
+ 421 39.66745843
432
+ 422 39.57345972
433
+ 423 39.47990544
434
+ 424 39.62264151
435
+ 425 39.76470588
436
+ 426 39.90610329
437
+ 427 39.81264637
438
+ 428 39.71962617
439
+ 429 39.86013986
440
+ 430 39.76744186
441
+ 431 39.90719258
442
+ 432 40.04629630
443
+ 433 39.95381062
444
+ 434 39.86175115
445
+ 435 39.77011494
446
+ 436 39.67889908
447
+ 437 39.58810069
448
+ 438 39.72602740
449
+ 439 39.86332574
450
+ 440 39.77272727
451
+ 441 39.68253968
452
+ 442 39.59276018
453
+ 443 39.50338600
454
+ 444 39.41441441
455
+ 445 39.32584270
456
+ 446 39.23766816
457
+ 447 39.37360179
458
+ 448 39.28571429
459
+ 449 39.19821826
460
+ 450 39.33333333
461
+ 451 39.46784922
462
+ 452 39.60176991
463
+ 453 39.51434879
464
+ 454 39.42731278
465
+ 455 39.34065934
466
+ 456 39.47368421
467
+ 457 39.60612691
468
+ 458 39.51965066
469
+ 459 39.43355120
470
+ 460 39.56521739
471
+ 461 39.47939262
472
+ 462 39.39393939
473
+ 463 39.30885529
474
+ 464 39.43965517
475
+ 465 39.35483871
476
+ 466 39.27038627
477
+ 467 39.40042827
478
+ 468 39.31623932
479
+ 469 39.44562900
480
+ 470 39.36170213
481
+ 471 39.27813163
482
+ 472 39.19491525
483
+ 473 39.11205074
484
+ 474 39.02953586
485
+ 475 38.94736842
486
+ 476 39.07563025
487
+ 477 38.99371069
488
+ 478 39.12133891
489
+ 479 39.24843424
490
+ 480 39.37500000
491
+ 481 39.29313929
492
+ 482 39.21161826
493
+ 483 39.13043478
494
+ 484 39.04958678
495
+ 485 38.96907216
496
+ 486 39.09465021
497
+ 487 39.21971253
498
+ 488 39.13934426
499
+ 489 39.05930470
500
+ 490 39.18367347
501
+ 491 39.10386965
502
+ 492 39.02439024
503
+ 493 38.94523327
504
+ 494 39.06882591
505
+ 495 39.19191919
506
+ 496 39.31451613
507
+ 497 39.23541247
508
+ 498 39.15662651
509
+ 499 39.07815631
510
+ 500 39.00000000
511
+ 501 38.92215569
512
+ 502 38.84462151
513
+ 503 38.76739563
514
+ 504 38.88888889
515
+ 505 38.81188119
516
+ 506 38.93280632
517
+ 507 38.85601578
518
+ 508 38.97637795
519
+ 509 38.89980354
520
+ 510 39.01960784
521
+ 511 38.94324853
522
+ 512 38.86718750
523
+ 513 38.79142300
524
+ 514 38.71595331
525
+ 515 38.64077670
526
+ 516 38.75968992
527
+ 517 38.87814313
528
+ 518 38.99613900
529
+ 519 38.92100193
530
+ 520 39.03846154
531
+ 521 38.96353167
532
+ 522 39.08045977
533
+ 523 39.00573614
534
+ 524 39.12213740
535
+ 525 39.04761905
536
+ 526 38.97338403
537
+ 527 39.08918406
538
+ 528 39.01515152
539
+ 529 38.94139887
540
+ 530 39.05660377
541
+ 531 38.98305085
542
+ 532 38.90977444
543
+ 533 38.83677298
544
+ 534 38.76404494
545
+ 535 38.87850467
546
+ 536 38.80597015
547
+ 537 38.91992551
548
+ 538 39.03345725
549
+ 539 38.96103896
550
+ 540 38.88888889
551
+ 541 38.81700555
552
+ 542 38.92988930
553
+ 543 38.85819521
554
+ 544 38.78676471
555
+ 545 38.71559633
556
+ 546 38.64468864
557
+ 547 38.75685558
558
+ 548 38.86861314
559
+ 549 38.79781421
560
+ 550 38.72727273
561
+ 551 38.83847550
562
+ 552 38.76811594
563
+ 553 38.69801085
564
+ 554 38.80866426
565
+ 555 38.73873874
566
+ 556 38.84892086
567
+ 557 38.77917415
568
+ 558 38.88888889
569
+ 559 38.81932021
570
+ 560 38.75000000
571
+ 561 38.68092692
572
+ 562 38.79003559
573
+ 563 38.89875666
574
+ 564 38.82978723
575
+ 565 38.93805310
576
+ 566 39.04593640
577
+ 567 38.97707231
578
+ 568 38.90845070
579
+ 569 38.84007030
580
+ 570 38.94736842
581
+ 571 38.87915937
582
+ 572 38.98601399
583
+ 573 38.91797557
584
+ 574 39.02439024
585
+ 575 38.95652174
586
+ 576 39.06250000
587
+ 577 39.16811092
588
+ 578 39.10034602
589
+ 579 39.03281520
590
+ 580 38.96551724
591
+ 581 38.89845095
592
+ 582 38.83161512
593
+ 583 38.93653516
594
+ 584 39.04109589
595
+ 585 38.97435897
596
+ 586 39.07849829
597
+ 587 39.01192504
598
+ 588 38.94557823
599
+ 589 38.87945671
600
+ 590 38.81355932
601
+ 591 38.74788494
602
+ 592 38.68243243
603
+ 593 38.61720067
604
+ 594 38.72053872
605
+ 595 38.65546218
606
+ 596 38.75838926
607
+ 597 38.69346734
608
+ 598 38.62876254
609
+ 599 38.73121870
610
+ 600 38.66666667
611
+ 601 38.76871880
612
+ 602 38.70431894
613
+ 603 38.80597015
614
+ 604 38.74172185
615
+ 605 38.67768595
616
+ 606 38.61386139
617
+ 607 38.71499176
618
+ 608 38.65131579
619
+ 609 38.58784893
620
+ 610 38.52459016
621
+ 611 38.62520458
622
+ 612 38.56209150
623
+ 613 38.49918434
624
+ 614 38.43648208
625
+ 615 38.37398374
626
+ 616 38.47402597
627
+ 617 38.41166937
628
+ 618 38.34951456
629
+ 619 38.28756058
630
+ 620 38.22580645
631
+ 621 38.16425121
632
+ 622 38.10289389
633
+ 623 38.04173355
634
+ 624 37.98076923
635
+ 625 37.92000000
636
+ 626 38.01916933
637
+ 627 37.95853270
638
+ 628 37.89808917
639
+ 629 37.99682035
640
+ 630 38.09523810
641
+ 631 38.19334390
642
+ 632 38.29113924
643
+ 633 38.23064771
644
+ 634 38.17034700
645
+ 635 38.11023622
646
+ 636 38.05031447
647
+ 637 38.14756672
648
+ 638 38.08777429
649
+ 639 38.18466354
650
+ 640 38.28125000
651
+ 641 38.22152886
652
+ 642 38.31775701
653
+ 643 38.25816485
654
+ 644 38.35403727
655
+ 645 38.29457364
656
+ 646 38.23529412
657
+ 647 38.17619784
658
+ 648 38.11728395
659
+ 649 38.05855162
660
+ 650 38.00000000
661
+ 651 37.94162826
662
+ 652 37.88343558
663
+ 653 37.82542113
664
+ 654 37.92048930
665
+ 655 37.86259542
666
+ 656 37.80487805
667
+ 657 37.74733638
668
+ 658 37.84194529
669
+ 659 37.78452200
670
+ 660 37.72727273
671
+ 661 37.67019667
672
+ 662 37.61329305
673
+ 663 37.55656109
674
+ 664 37.65060241
675
+ 665 37.74436090
676
+ 666 37.68768769
677
+ 667 37.78110945
678
+ 668 37.87425150
679
+ 669 37.96711510
680
+ 670 38.05970149
681
+ 671 38.00298063
682
+ 672 37.94642857
683
+ 673 37.89004458
684
+ 674 37.83382789
685
+ 675 37.77777778
686
+ 676 37.72189349
687
+ 677 37.66617430
688
+ 678 37.61061947
689
+ 679 37.55522828
690
+ 680 37.64705882
691
+ 681 37.73861968
692
+ 682 37.68328446
693
+ 683 37.62811127
694
+ 684 37.57309942
695
+ 685 37.51824818
696
+ 686 37.46355685
697
+ 687 37.40902475
698
+ 688 37.35465116
699
+ 689 37.30043541
700
+ 690 37.24637681
701
+ 691 37.33719247
702
+ 692 37.28323699
703
+ 693 37.22943723
704
+ 694 37.17579251
705
+ 695 37.12230216
706
+ 696 37.06896552
707
+ 697 37.15925395
708
+ 698 37.10601719
709
+ 699 37.19599428
710
+ 700 37.14285714
711
+ 701 37.23252496
712
+ 702 37.32193732
713
+ 703 37.41109531
714
+ 704 37.35795455
715
+ 705 37.30496454
716
+ 706 37.25212465
717
+ 707 37.19943423
718
+ 708 37.14689266
719
+ 709 37.23554302
720
+ 710 37.32394366
721
+ 711 37.41209564
722
+ 712 37.35955056
723
+ 713 37.44740533
724
+ 714 37.53501401
725
+ 715 37.48251748
726
+ 716 37.56983240
727
+ 717 37.51743375
728
+ 718 37.46518106
729
+ 719 37.55215577
730
+ 720 37.63888889
731
+ 721 37.58668516
732
+ 722 37.67313019
733
+ 723 37.75933610
734
+ 724 37.70718232
735
+ 725 37.79310345
736
+ 726 37.74104683
737
+ 727 37.68913343
738
+ 728 37.63736264
739
+ 729 37.58573388
740
+ 730 37.53424658
741
+ 731 37.48290014
742
+ 732 37.56830601
743
+ 733 37.65347885
744
+ 734 37.73841962
745
+ 735 37.68707483
746
+ 736 37.63586957
747
+ 737 37.72048847
748
+ 738 37.66937669
749
+ 739 37.75372124
750
+ 740 37.70270270
751
+ 741 37.65182186
752
+ 742 37.73584906
753
+ 743 37.81965007
754
+ 744 37.90322581
755
+ 745 37.85234899
756
+ 746 37.80160858
757
+ 747 37.75100402
758
+ 748 37.70053476
759
+ 749 37.65020027
760
+ 750 37.73333333
761
+
762
+ Final result: 37.7333 +/- 1.7711
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-iq3_m.ppx ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ====== Perplexity statistics ======
2
+ Mean PPL(Q) : 10.511302 ± 0.075491
3
+ Mean PPL(base) : 9.366577 ± 0.066397
4
+ Cor(ln(PPL(Q)), ln(PPL(base))): 97.82%
5
+ Mean ln(PPL(Q)/PPL(base)) : 0.115303 ± 0.001493
6
+ Mean PPL(Q)/PPL(base) : 1.122214 ± 0.001676
7
+ Mean PPL(Q)-PPL(base) : 1.144725 ± 0.017361
8
+
9
+ ====== KL divergence statistics ======
10
+ Mean KLD: 0.138301 ± 0.000410
11
+ Maximum KLD: 5.158908
12
+ 99.9% KLD: 1.448158
13
+ 99.0% KLD: 0.724885
14
+ 99.0% KLD: 0.724885
15
+ Median KLD: 0.100701
16
+ 10.0% KLD: 0.002606
17
+ 5.0% KLD: 0.000562
18
+ 1.0% KLD: 0.000044
19
+ Minimum KLD: -0.000005
20
+
21
+ ====== Token probability statistics ======
22
+ Mean Δp: -2.210 ± 0.024 %
23
+ Maximum Δp: 91.480%
24
+ 99.9% Δp: 40.259%
25
+ 99.0% Δp: 22.257%
26
+ 95.0% Δp: 10.085%
27
+ 90.0% Δp: 5.065%
28
+ 75.0% Δp: 0.353%
29
+ Median Δp: -0.202%
30
+ 25.0% Δp: -4.080%
31
+ 10.0% Δp: -12.639%
32
+ 5.0% Δp: -19.206%
33
+ 1.0% Δp: -34.978%
34
+ 0.1% Δp: -58.627%
35
+ Minimum Δp: -95.079%
36
+ RMS Δp : 9.594 ± 0.040 %
37
+ Same top p: 80.519 ± 0.103 %
scores/Hammer2.1-7b-iq3_m.tqa ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 817 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 817 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 14.28571429
18
+ 8 12.50000000
19
+ 9 11.11111111
20
+ 10 10.00000000
21
+ 11 9.09090909
22
+ 12 16.66666667
23
+ 13 23.07692308
24
+ 14 21.42857143
25
+ 15 20.00000000
26
+ 16 18.75000000
27
+ 17 17.64705882
28
+ 18 16.66666667
29
+ 19 21.05263158
30
+ 20 20.00000000
31
+ 21 19.04761905
32
+ 22 18.18181818
33
+ 23 17.39130435
34
+ 24 20.83333333
35
+ 25 20.00000000
36
+ 26 19.23076923
37
+ 27 18.51851852
38
+ 28 17.85714286
39
+ 29 20.68965517
40
+ 30 20.00000000
41
+ 31 19.35483871
42
+ 32 21.87500000
43
+ 33 24.24242424
44
+ 34 23.52941176
45
+ 35 22.85714286
46
+ 36 22.22222222
47
+ 37 24.32432432
48
+ 38 23.68421053
49
+ 39 23.07692308
50
+ 40 22.50000000
51
+ 41 21.95121951
52
+ 42 21.42857143
53
+ 43 23.25581395
54
+ 44 25.00000000
55
+ 45 26.66666667
56
+ 46 28.26086957
57
+ 47 27.65957447
58
+ 48 29.16666667
59
+ 49 28.57142857
60
+ 50 30.00000000
61
+ 51 31.37254902
62
+ 52 30.76923077
63
+ 53 30.18867925
64
+ 54 31.48148148
65
+ 55 32.72727273
66
+ 56 32.14285714
67
+ 57 31.57894737
68
+ 58 31.03448276
69
+ 59 30.50847458
70
+ 60 30.00000000
71
+ 61 29.50819672
72
+ 62 30.64516129
73
+ 63 30.15873016
74
+ 64 31.25000000
75
+ 65 30.76923077
76
+ 66 30.30303030
77
+ 67 29.85074627
78
+ 68 29.41176471
79
+ 69 30.43478261
80
+ 70 31.42857143
81
+ 71 30.98591549
82
+ 72 30.55555556
83
+ 73 30.13698630
84
+ 74 29.72972973
85
+ 75 29.33333333
86
+ 76 28.94736842
87
+ 77 28.57142857
88
+ 78 28.20512821
89
+ 79 27.84810127
90
+ 80 27.50000000
91
+ 81 27.16049383
92
+ 82 28.04878049
93
+ 83 27.71084337
94
+ 84 27.38095238
95
+ 85 27.05882353
96
+ 86 26.74418605
97
+ 87 26.43678161
98
+ 88 26.13636364
99
+ 89 26.96629213
100
+ 90 26.66666667
101
+ 91 27.47252747
102
+ 92 27.17391304
103
+ 93 26.88172043
104
+ 94 26.59574468
105
+ 95 27.36842105
106
+ 96 28.12500000
107
+ 97 28.86597938
108
+ 98 28.57142857
109
+ 99 28.28282828
110
+ 100 29.00000000
111
+ 101 29.70297030
112
+ 102 29.41176471
113
+ 103 29.12621359
114
+ 104 28.84615385
115
+ 105 28.57142857
116
+ 106 28.30188679
117
+ 107 28.97196262
118
+ 108 29.62962963
119
+ 109 29.35779817
120
+ 110 30.00000000
121
+ 111 29.72972973
122
+ 112 29.46428571
123
+ 113 29.20353982
124
+ 114 29.82456140
125
+ 115 29.56521739
126
+ 116 29.31034483
127
+ 117 29.05982906
128
+ 118 28.81355932
129
+ 119 28.57142857
130
+ 120 29.16666667
131
+ 121 29.75206612
132
+ 122 29.50819672
133
+ 123 30.08130081
134
+ 124 30.64516129
135
+ 125 30.40000000
136
+ 126 30.15873016
137
+ 127 29.92125984
138
+ 128 29.68750000
139
+ 129 29.45736434
140
+ 130 29.23076923
141
+ 131 29.00763359
142
+ 132 28.78787879
143
+ 133 28.57142857
144
+ 134 29.10447761
145
+ 135 29.62962963
146
+ 136 29.41176471
147
+ 137 29.92700730
148
+ 138 30.43478261
149
+ 139 30.93525180
150
+ 140 30.71428571
151
+ 141 30.49645390
152
+ 142 30.28169014
153
+ 143 30.06993007
154
+ 144 30.55555556
155
+ 145 30.34482759
156
+ 146 30.13698630
157
+ 147 29.93197279
158
+ 148 29.72972973
159
+ 149 29.53020134
160
+ 150 29.33333333
161
+ 151 29.80132450
162
+ 152 29.60526316
163
+ 153 29.41176471
164
+ 154 29.22077922
165
+ 155 29.67741935
166
+ 156 30.12820513
167
+ 157 29.93630573
168
+ 158 30.37974684
169
+ 159 30.18867925
170
+ 160 30.00000000
171
+ 161 29.81366460
172
+ 162 29.62962963
173
+ 163 29.44785276
174
+ 164 29.26829268
175
+ 165 29.69696970
176
+ 166 30.12048193
177
+ 167 29.94011976
178
+ 168 29.76190476
179
+ 169 29.58579882
180
+ 170 29.41176471
181
+ 171 29.23976608
182
+ 172 29.06976744
183
+ 173 28.90173410
184
+ 174 29.31034483
185
+ 175 29.14285714
186
+ 176 28.97727273
187
+ 177 28.81355932
188
+ 178 28.65168539
189
+ 179 28.49162011
190
+ 180 28.33333333
191
+ 181 28.17679558
192
+ 182 28.02197802
193
+ 183 27.86885246
194
+ 184 27.71739130
195
+ 185 27.56756757
196
+ 186 27.95698925
197
+ 187 27.80748663
198
+ 188 28.19148936
199
+ 189 28.04232804
200
+ 190 27.89473684
201
+ 191 27.74869110
202
+ 192 28.12500000
203
+ 193 27.97927461
204
+ 194 28.35051546
205
+ 195 28.20512821
206
+ 196 28.06122449
207
+ 197 28.42639594
208
+ 198 28.78787879
209
+ 199 28.64321608
210
+ 200 28.50000000
211
+ 201 28.35820896
212
+ 202 28.71287129
213
+ 203 28.57142857
214
+ 204 28.43137255
215
+ 205 28.29268293
216
+ 206 28.15533981
217
+ 207 28.50241546
218
+ 208 28.36538462
219
+ 209 28.22966507
220
+ 210 28.09523810
221
+ 211 27.96208531
222
+ 212 27.83018868
223
+ 213 27.69953052
224
+ 214 27.57009346
225
+ 215 27.44186047
226
+ 216 27.77777778
227
+ 217 28.11059908
228
+ 218 28.44036697
229
+ 219 28.31050228
230
+ 220 28.18181818
231
+ 221 28.05429864
232
+ 222 27.92792793
233
+ 223 27.80269058
234
+ 224 28.12500000
235
+ 225 28.00000000
236
+ 226 27.87610619
237
+ 227 28.19383260
238
+ 228 28.07017544
239
+ 229 27.94759825
240
+ 230 27.82608696
241
+ 231 27.70562771
242
+ 232 28.01724138
243
+ 233 28.32618026
244
+ 234 28.20512821
245
+ 235 28.08510638
246
+ 236 28.38983051
247
+ 237 28.69198312
248
+ 238 28.99159664
249
+ 239 28.87029289
250
+ 240 28.75000000
251
+ 241 28.63070539
252
+ 242 28.51239669
253
+ 243 28.39506173
254
+ 244 28.27868852
255
+ 245 28.16326531
256
+ 246 28.04878049
257
+ 247 27.93522267
258
+ 248 27.82258065
259
+ 249 27.71084337
260
+ 250 27.60000000
261
+ 251 27.88844622
262
+ 252 27.77777778
263
+ 253 27.66798419
264
+ 254 27.55905512
265
+ 255 27.84313725
266
+ 256 28.12500000
267
+ 257 28.01556420
268
+ 258 27.90697674
269
+ 259 27.79922780
270
+ 260 27.69230769
271
+ 261 27.96934866
272
+ 262 27.86259542
273
+ 263 27.75665399
274
+ 264 28.03030303
275
+ 265 28.30188679
276
+ 266 28.19548872
277
+ 267 28.46441948
278
+ 268 28.35820896
279
+ 269 28.25278810
280
+ 270 28.14814815
281
+ 271 28.04428044
282
+ 272 27.94117647
283
+ 273 27.83882784
284
+ 274 28.10218978
285
+ 275 28.00000000
286
+ 276 27.89855072
287
+ 277 27.79783394
288
+ 278 27.69784173
289
+ 279 27.59856631
290
+ 280 27.85714286
291
+ 281 28.11387900
292
+ 282 28.01418440
293
+ 283 27.91519435
294
+ 284 28.16901408
295
+ 285 28.07017544
296
+ 286 28.32167832
297
+ 287 28.57142857
298
+ 288 28.47222222
299
+ 289 28.37370242
300
+ 290 28.27586207
301
+ 291 28.52233677
302
+ 292 28.42465753
303
+ 293 28.66894198
304
+ 294 28.57142857
305
+ 295 28.81355932
306
+ 296 29.05405405
307
+ 297 29.29292929
308
+ 298 29.19463087
309
+ 299 29.09698997
310
+ 300 29.00000000
311
+ 301 28.90365449
312
+ 302 28.80794702
313
+ 303 28.71287129
314
+ 304 28.61842105
315
+ 305 28.52459016
316
+ 306 28.43137255
317
+ 307 28.33876221
318
+ 308 28.57142857
319
+ 309 28.47896440
320
+ 310 28.38709677
321
+ 311 28.29581994
322
+ 312 28.52564103
323
+ 313 28.43450479
324
+ 314 28.66242038
325
+ 315 28.57142857
326
+ 316 28.48101266
327
+ 317 28.39116719
328
+ 318 28.30188679
329
+ 319 28.21316614
330
+ 320 28.12500000
331
+ 321 28.03738318
332
+ 322 28.26086957
333
+ 323 28.17337461
334
+ 324 28.08641975
335
+ 325 28.00000000
336
+ 326 27.91411043
337
+ 327 28.13455657
338
+ 328 28.35365854
339
+ 329 28.26747720
340
+ 330 28.18181818
341
+ 331 28.09667674
342
+ 332 28.31325301
343
+ 333 28.22822823
344
+ 334 28.14371257
345
+ 335 28.35820896
346
+ 336 28.57142857
347
+ 337 28.48664688
348
+ 338 28.40236686
349
+ 339 28.61356932
350
+ 340 28.82352941
351
+ 341 29.03225806
352
+ 342 29.23976608
353
+ 343 29.15451895
354
+ 344 29.36046512
355
+ 345 29.56521739
356
+ 346 29.76878613
357
+ 347 29.97118156
358
+ 348 29.88505747
359
+ 349 29.79942693
360
+ 350 29.71428571
361
+ 351 29.62962963
362
+ 352 29.54545455
363
+ 353 29.46175637
364
+ 354 29.37853107
365
+ 355 29.57746479
366
+ 356 29.49438202
367
+ 357 29.69187675
368
+ 358 29.88826816
369
+ 359 29.80501393
370
+ 360 29.72222222
371
+ 361 29.91689751
372
+ 362 29.83425414
373
+ 363 30.02754821
374
+ 364 29.94505495
375
+ 365 30.13698630
376
+ 366 30.05464481
377
+ 367 30.24523161
378
+ 368 30.16304348
379
+ 369 30.08130081
380
+ 370 30.00000000
381
+ 371 29.91913747
382
+ 372 30.10752688
383
+ 373 30.02680965
384
+ 374 30.21390374
385
+ 375 30.13333333
386
+ 376 30.05319149
387
+ 377 30.23872679
388
+ 378 30.15873016
389
+ 379 30.07915567
390
+ 380 30.00000000
391
+ 381 29.92125984
392
+ 382 30.10471204
393
+ 383 30.02610966
394
+ 384 29.94791667
395
+ 385 29.87012987
396
+ 386 29.79274611
397
+ 387 29.71576227
398
+ 388 29.89690722
399
+ 389 29.82005141
400
+ 390 29.74358974
401
+ 391 29.66751918
402
+ 392 29.59183673
403
+ 393 29.51653944
404
+ 394 29.69543147
405
+ 395 29.62025316
406
+ 396 29.54545455
407
+ 397 29.72292191
408
+ 398 29.64824121
409
+ 399 29.57393484
410
+ 400 29.50000000
411
+ 401 29.42643392
412
+ 402 29.60199005
413
+ 403 29.77667494
414
+ 404 29.70297030
415
+ 405 29.62962963
416
+ 406 29.55665025
417
+ 407 29.48402948
418
+ 408 29.41176471
419
+ 409 29.33985330
420
+ 410 29.26829268
421
+ 411 29.19708029
422
+ 412 29.36893204
423
+ 413 29.29782082
424
+ 414 29.46859903
425
+ 415 29.63855422
426
+ 416 29.56730769
427
+ 417 29.49640288
428
+ 418 29.42583732
429
+ 419 29.35560859
430
+ 420 29.52380952
431
+ 421 29.69121140
432
+ 422 29.62085308
433
+ 423 29.55082742
434
+ 424 29.48113208
435
+ 425 29.41176471
436
+ 426 29.34272300
437
+ 427 29.27400468
438
+ 428 29.20560748
439
+ 429 29.37062937
440
+ 430 29.30232558
441
+ 431 29.23433875
442
+ 432 29.39814815
443
+ 433 29.56120092
444
+ 434 29.72350230
445
+ 435 29.65517241
446
+ 436 29.58715596
447
+ 437 29.51945080
448
+ 438 29.45205479
449
+ 439 29.38496583
450
+ 440 29.31818182
451
+ 441 29.25170068
452
+ 442 29.18552036
453
+ 443 29.11963883
454
+ 444 29.27927928
455
+ 445 29.21348315
456
+ 446 29.14798206
457
+ 447 29.08277405
458
+ 448 29.01785714
459
+ 449 29.17594655
460
+ 450 29.11111111
461
+ 451 29.04656319
462
+ 452 28.98230088
463
+ 453 28.91832230
464
+ 454 29.07488987
465
+ 455 29.01098901
466
+ 456 28.94736842
467
+ 457 28.88402626
468
+ 458 28.82096070
469
+ 459 28.75816993
470
+ 460 28.69565217
471
+ 461 28.63340564
472
+ 462 28.57142857
473
+ 463 28.50971922
474
+ 464 28.66379310
475
+ 465 28.81720430
476
+ 466 28.75536481
477
+ 467 28.69379015
478
+ 468 28.63247863
479
+ 469 28.57142857
480
+ 470 28.51063830
481
+ 471 28.66242038
482
+ 472 28.60169492
483
+ 473 28.54122622
484
+ 474 28.48101266
485
+ 475 28.63157895
486
+ 476 28.57142857
487
+ 477 28.72117400
488
+ 478 28.66108787
489
+ 479 28.60125261
490
+ 480 28.54166667
491
+ 481 28.48232848
492
+ 482 28.63070539
493
+ 483 28.57142857
494
+ 484 28.71900826
495
+ 485 28.86597938
496
+ 486 28.80658436
497
+ 487 28.74743326
498
+ 488 28.68852459
499
+ 489 28.83435583
500
+ 490 28.97959184
501
+ 491 29.12423625
502
+ 492 29.26829268
503
+ 493 29.20892495
504
+ 494 29.14979757
505
+ 495 29.29292929
506
+ 496 29.23387097
507
+ 497 29.17505030
508
+ 498 29.11646586
509
+ 499 29.25851703
510
+ 500 29.20000000
511
+ 501 29.34131737
512
+ 502 29.28286853
513
+ 503 29.22465209
514
+ 504 29.36507937
515
+ 505 29.30693069
516
+ 506 29.24901186
517
+ 507 29.38856016
518
+ 508 29.33070866
519
+ 509 29.46954813
520
+ 510 29.60784314
521
+ 511 29.54990215
522
+ 512 29.49218750
523
+ 513 29.43469786
524
+ 514 29.37743191
525
+ 515 29.51456311
526
+ 516 29.45736434
527
+ 517 29.59381044
528
+ 518 29.53667954
529
+ 519 29.47976879
530
+ 520 29.42307692
531
+ 521 29.36660269
532
+ 522 29.31034483
533
+ 523 29.25430210
534
+ 524 29.19847328
535
+ 525 29.14285714
536
+ 526 29.08745247
537
+ 527 29.03225806
538
+ 528 28.97727273
539
+ 529 28.92249527
540
+ 530 28.86792453
541
+ 531 28.81355932
542
+ 532 28.94736842
543
+ 533 29.08067542
544
+ 534 29.02621723
545
+ 535 28.97196262
546
+ 536 28.91791045
547
+ 537 29.05027933
548
+ 538 28.99628253
549
+ 539 28.94248609
550
+ 540 29.07407407
551
+ 541 29.02033272
552
+ 542 28.96678967
553
+ 543 28.91344383
554
+ 544 28.86029412
555
+ 545 28.80733945
556
+ 546 28.93772894
557
+ 547 29.06764168
558
+ 548 29.19708029
559
+ 549 29.14389800
560
+ 550 29.09090909
561
+ 551 29.03811252
562
+ 552 29.16666667
563
+ 553 29.29475588
564
+ 554 29.24187726
565
+ 555 29.18918919
566
+ 556 29.13669065
567
+ 557 29.08438061
568
+ 558 29.03225806
569
+ 559 28.98032200
570
+ 560 29.10714286
571
+ 561 29.23351159
572
+ 562 29.18149466
573
+ 563 29.12966252
574
+ 564 29.25531915
575
+ 565 29.38053097
576
+ 566 29.50530035
577
+ 567 29.45326279
578
+ 568 29.40140845
579
+ 569 29.52548330
580
+ 570 29.47368421
581
+ 571 29.42206655
582
+ 572 29.37062937
583
+ 573 29.31937173
584
+ 574 29.44250871
585
+ 575 29.56521739
586
+ 576 29.68750000
587
+ 577 29.80935875
588
+ 578 29.93079585
589
+ 579 30.05181347
590
+ 580 30.17241379
591
+ 581 30.12048193
592
+ 582 30.06872852
593
+ 583 30.01715266
594
+ 584 30.13698630
595
+ 585 30.08547009
596
+ 586 30.20477816
597
+ 587 30.15332198
598
+ 588 30.10204082
599
+ 589 30.05093379
600
+ 590 30.16949153
601
+ 591 30.28764805
602
+ 592 30.23648649
603
+ 593 30.18549747
604
+ 594 30.13468013
605
+ 595 30.08403361
606
+ 596 30.03355705
607
+ 597 30.15075377
608
+ 598 30.10033445
609
+ 599 30.05008347
610
+ 600 30.00000000
611
+ 601 30.11647255
612
+ 602 30.06644518
613
+ 603 30.18242123
614
+ 604 30.13245033
615
+ 605 30.08264463
616
+ 606 30.19801980
617
+ 607 30.14827018
618
+ 608 30.09868421
619
+ 609 30.21346470
620
+ 610 30.16393443
621
+ 611 30.11456628
622
+ 612 30.22875817
623
+ 613 30.17944535
624
+ 614 30.13029316
625
+ 615 30.08130081
626
+ 616 30.19480519
627
+ 617 30.30794165
628
+ 618 30.25889968
629
+ 619 30.37156704
630
+ 620 30.32258065
631
+ 621 30.27375201
632
+ 622 30.38585209
633
+ 623 30.33707865
634
+ 624 30.28846154
635
+ 625 30.24000000
636
+ 626 30.19169329
637
+ 627 30.14354067
638
+ 628 30.09554140
639
+ 629 30.04769475
640
+ 630 30.00000000
641
+ 631 30.11093502
642
+ 632 30.06329114
643
+ 633 30.01579779
644
+ 634 30.12618297
645
+ 635 30.07874016
646
+ 636 30.03144654
647
+ 637 29.98430141
648
+ 638 29.93730408
649
+ 639 30.04694836
650
+ 640 30.00000000
651
+ 641 29.95319813
652
+ 642 30.06230530
653
+ 643 30.01555210
654
+ 644 29.96894410
655
+ 645 30.07751938
656
+ 646 30.03095975
657
+ 647 29.98454405
658
+ 648 30.09259259
659
+ 649 30.04622496
660
+ 650 30.15384615
661
+ 651 30.26113671
662
+ 652 30.21472393
663
+ 653 30.32159265
664
+ 654 30.27522936
665
+ 655 30.22900763
666
+ 656 30.33536585
667
+ 657 30.44140030
668
+ 658 30.39513678
669
+ 659 30.34901366
670
+ 660 30.30303030
671
+ 661 30.25718608
672
+ 662 30.21148036
673
+ 663 30.31674208
674
+ 664 30.27108434
675
+ 665 30.37593985
676
+ 666 30.48048048
677
+ 667 30.43478261
678
+ 668 30.38922156
679
+ 669 30.34379671
680
+ 670 30.44776119
681
+ 671 30.40238450
682
+ 672 30.35714286
683
+ 673 30.31203566
684
+ 674 30.26706231
685
+ 675 30.37037037
686
+ 676 30.47337278
687
+ 677 30.42836041
688
+ 678 30.38348083
689
+ 679 30.33873343
690
+ 680 30.29411765
691
+ 681 30.24963289
692
+ 682 30.20527859
693
+ 683 30.16105417
694
+ 684 30.26315789
695
+ 685 30.21897810
696
+ 686 30.17492711
697
+ 687 30.13100437
698
+ 688 30.08720930
699
+ 689 30.18867925
700
+ 690 30.14492754
701
+ 691 30.24602026
702
+ 692 30.34682081
703
+ 693 30.44733045
704
+ 694 30.40345821
705
+ 695 30.35971223
706
+ 696 30.45977011
707
+ 697 30.41606887
708
+ 698 30.37249284
709
+ 699 30.32904149
710
+ 700 30.42857143
711
+ 701 30.38516405
712
+ 702 30.34188034
713
+ 703 30.29871977
714
+ 704 30.25568182
715
+ 705 30.21276596
716
+ 706 30.16997167
717
+ 707 30.26874116
718
+ 708 30.22598870
719
+ 709 30.18335684
720
+ 710 30.14084507
721
+ 711 30.09845288
722
+ 712 30.05617978
723
+ 713 30.01402525
724
+ 714 30.11204482
725
+ 715 30.20979021
726
+ 716 30.16759777
727
+ 717 30.12552301
728
+ 718 30.08356546
729
+ 719 30.04172462
730
+ 720 30.00000000
731
+ 721 29.95839112
732
+ 722 29.91689751
733
+ 723 29.87551867
734
+ 724 29.83425414
735
+ 725 29.79310345
736
+ 726 29.88980716
737
+ 727 29.84869326
738
+ 728 29.80769231
739
+ 729 29.90397805
740
+ 730 29.86301370
741
+ 731 29.82216142
742
+ 732 29.91803279
743
+ 733 29.87721692
744
+ 734 29.83651226
745
+ 735 29.79591837
746
+ 736 29.75543478
747
+ 737 29.85074627
748
+ 738 29.81029810
749
+ 739 29.90527740
750
+ 740 29.86486486
751
+ 741 29.95951417
752
+ 742 29.91913747
753
+ 743 29.87886945
754
+ 744 29.97311828
755
+ 745 29.93288591
756
+ 746 29.89276139
757
+ 747 29.98661312
758
+ 748 29.94652406
759
+ 749 30.04005340
760
+ 750 30.13333333
761
+
762
+ Final result: 30.1333 +/- 1.6766
763
+ Random chance: 19.8992 +/- 1.4588
scores/Hammer2.1-7b-iq3_m.wng ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ winogrande_score : loaded 1267 tasks from prompt.
6
+ winogrande_score : selecting 750 random tasks
7
+ winogrande_score : tokenizing selected tasks
8
+ winogrande_score : calculating winogrande score over selected tasks.
9
+ 1 100.0000 -4.472389 -4.917857 1 1
10
+ 2 100.0000 -2.339383 -2.487111 1 1
11
+ 3 100.0000 -2.765451 -2.277070 2 2
12
+ 4 75.0000 -4.979261 -4.620960 2 1
13
+ 5 60.0000 -2.197215 -3.222974 1 2
14
+ 6 50.0000 -5.002606 -4.858578 2 1
15
+ 7 57.1429 -3.963147 -2.873791 2 2
16
+ 8 50.0000 -2.127338 -2.140928 1 2
17
+ 9 55.5556 -3.685873 -6.036837 1 1
18
+ 10 60.0000 -1.937850 -1.399040 2 2
19
+ 11 63.6364 -11.069501 -5.560449 2 2
20
+ 12 66.6667 -4.286385 -3.749312 2 2
21
+ 13 69.2308 -2.963913 -5.950130 1 1
22
+ 14 64.2857 -4.002491 -3.910132 2 1
23
+ 15 60.0000 -4.220399 -5.729921 1 2
24
+ 16 56.2500 -4.046220 -4.142797 1 2
25
+ 17 52.9412 -5.928837 -6.298713 1 2
26
+ 18 55.5556 -1.143484 -2.915234 1 1
27
+ 19 57.8947 -4.328619 -5.116286 1 1
28
+ 20 60.0000 -3.194638 -4.197226 1 1
29
+ 21 61.9048 -2.383163 -3.098103 1 1
30
+ 22 63.6364 -2.841487 -2.398214 2 2
31
+ 23 65.2174 -5.093666 -5.159095 1 1
32
+ 24 62.5000 -2.701877 -2.468732 2 1
33
+ 25 64.0000 -2.780617 -2.650379 2 2
34
+ 26 65.3846 -3.891662 -4.370107 1 1
35
+ 27 66.6667 -5.343966 -5.081650 2 2
36
+ 28 67.8571 -4.753020 -5.394220 1 1
37
+ 29 65.5172 -4.190217 -4.166955 2 1
38
+ 30 63.3333 -1.505695 -1.819854 1 2
39
+ 31 64.5161 -5.397068 -5.414462 1 1
40
+ 32 65.6250 -4.473865 -6.434514 1 1
41
+ 33 66.6667 -3.106470 -5.729861 1 1
42
+ 34 64.7059 -2.520796 -2.353496 2 1
43
+ 35 65.7143 -4.464398 -4.432621 2 2
44
+ 36 66.6667 -5.081418 -3.949309 2 2
45
+ 37 64.8649 -3.508575 -3.343592 2 1
46
+ 38 65.7895 -3.178413 -3.371743 1 1
47
+ 39 66.6667 -4.458986 -4.386408 2 2
48
+ 40 65.0000 -4.460755 -5.578243 1 2
49
+ 41 65.8537 -2.890459 -3.142486 1 1
50
+ 42 66.6667 -1.892202 -2.467221 1 1
51
+ 43 65.1163 -4.178350 -4.274148 1 2
52
+ 44 65.9091 -1.951508 -1.930169 2 2
53
+ 45 66.6667 -0.076491 -13.375640 1 1
54
+ 46 67.3913 -2.587724 -2.830252 1 1
55
+ 47 65.9574 -4.412636 -4.950643 1 2
56
+ 48 66.6667 -5.074750 -4.572621 2 2
57
+ 49 65.3061 -2.775240 -1.231918 2 1
58
+ 50 64.0000 -2.568736 -2.633525 1 2
59
+ 51 64.7059 -2.642659 -2.970459 1 1
60
+ 52 65.3846 -2.926884 -2.205834 2 2
61
+ 53 66.0377 -3.902338 -4.866657 1 1
62
+ 54 66.6667 -3.645532 -3.442714 2 2
63
+ 55 65.4545 -3.205481 -3.271687 1 2
64
+ 56 64.2857 -5.096209 -5.111359 1 2
65
+ 57 63.1579 -1.856074 -2.776201 1 2
66
+ 58 63.7931 -3.433886 -4.441359 1 1
67
+ 59 64.4068 -3.084811 -3.019015 2 2
68
+ 60 65.0000 -4.432966 -4.084054 2 2
69
+ 61 65.5738 -3.943954 -4.388313 1 1
70
+ 62 64.5161 -7.535332 -7.494543 2 1
71
+ 63 63.4921 -4.197515 -4.901935 1 2
72
+ 64 64.0625 -4.096286 -4.418720 1 1
73
+ 65 64.6154 -5.076445 -4.558771 2 2
74
+ 66 65.1515 -2.536690 -3.626803 1 1
75
+ 67 65.6716 -2.590255 -2.170925 2 2
76
+ 68 66.1765 -4.800585 -4.779658 2 2
77
+ 69 66.6667 -5.040652 -8.598502 1 1
78
+ 70 67.1429 -3.429792 -2.810935 2 2
79
+ 71 66.1972 -3.377460 -2.978266 2 1
80
+ 72 65.2778 -2.878172 -3.374660 1 2
81
+ 73 65.7534 -3.234884 -3.371147 1 1
82
+ 74 64.8649 -2.510581 -2.239235 2 1
83
+ 75 65.3333 -1.437519 -2.293211 1 1
84
+ 76 65.7895 -1.934620 -4.150395 1 1
85
+ 77 64.9351 -2.963404 -3.153464 1 2
86
+ 78 65.3846 -1.366685 -1.608721 1 1
87
+ 79 64.5570 -2.210159 -2.723588 1 2
88
+ 80 65.0000 -2.281409 -3.572658 1 1
89
+ 81 64.1975 -3.696990 -3.760474 1 2
90
+ 82 64.6341 -2.759964 -2.456631 2 2
91
+ 83 63.8554 -6.695057 -6.868206 1 2
92
+ 84 64.2857 -2.349200 -2.253577 2 2
93
+ 85 64.7059 -4.416529 -4.909413 1 1
94
+ 86 63.9535 -5.893444 -6.193882 1 2
95
+ 87 63.2184 -1.560397 -1.392727 2 1
96
+ 88 62.5000 -4.852774 -4.668711 2 1
97
+ 89 62.9213 -2.735186 -3.114453 1 1
98
+ 90 63.3333 -5.825883 -3.947701 2 2
99
+ 91 62.6374 -4.626474 -4.100131 2 1
100
+ 92 63.0435 -5.705051 -4.075283 2 2
101
+ 93 63.4409 -2.595653 -2.365684 2 2
102
+ 94 63.8298 -3.874062 -3.068294 2 2
103
+ 95 64.2105 -5.312352 -5.975900 1 1
104
+ 96 64.5833 -2.227302 -3.443652 1 1
105
+ 97 63.9175 -5.628384 -5.606655 2 1
106
+ 98 64.2857 -5.515368 -3.911788 2 2
107
+ 99 63.6364 -3.061493 -2.518075 2 1
108
+ 100 64.0000 -6.181698 -6.147562 2 2
109
+ 101 63.3663 -4.785807 -4.819286 1 2
110
+ 102 63.7255 -4.323023 -5.006677 1 1
111
+ 103 63.1068 -2.992916 -3.125862 1 2
112
+ 104 63.4615 -8.618631 -8.420494 2 2
113
+ 105 63.8095 -1.310808 -2.435982 1 1
114
+ 106 64.1509 -3.098255 -2.819344 2 2
115
+ 107 64.4860 -3.638069 -2.520294 2 2
116
+ 108 64.8148 -3.989585 -3.180812 2 2
117
+ 109 65.1376 -5.755853 -5.637622 2 2
118
+ 110 65.4545 -4.432894 -3.625587 2 2
119
+ 111 65.7658 -4.175348 -3.657495 2 2
120
+ 112 65.1786 -3.446071 -3.351516 2 1
121
+ 113 64.6018 -4.859828 -5.082939 1 2
122
+ 114 64.9123 -3.846106 -4.629947 1 1
123
+ 115 65.2174 -2.578175 -3.234018 1 1
124
+ 116 65.5172 -3.621891 -2.739201 2 2
125
+ 117 64.9573 -4.635521 -4.764011 1 2
126
+ 118 65.2542 -0.938774 -2.775328 1 1
127
+ 119 65.5462 -2.429204 -2.899280 1 1
128
+ 120 65.0000 -2.482510 -2.301125 2 1
129
+ 121 65.2893 -1.390185 -1.005552 2 2
130
+ 122 65.5738 -6.424681 -4.619657 2 2
131
+ 123 65.8537 -3.001201 -3.179910 1 1
132
+ 124 66.1290 -3.148179 -2.616784 2 2
133
+ 125 66.4000 -4.138707 -3.207247 2 2
134
+ 126 65.8730 -7.088121 -7.162014 1 2
135
+ 127 66.1417 -2.359302 -1.483617 2 2
136
+ 128 66.4062 -2.117234 -2.474663 1 1
137
+ 129 66.6667 -2.290405 -1.278181 2 2
138
+ 130 66.9231 -5.602744 -5.489733 2 2
139
+ 131 66.4122 -3.414761 -3.892118 1 2
140
+ 132 66.6667 -3.104738 -2.937480 2 2
141
+ 133 66.9173 -1.745869 -3.882969 1 1
142
+ 134 67.1642 -4.974003 -4.273263 2 2
143
+ 135 67.4074 -4.535729 -5.427774 1 1
144
+ 136 67.6471 -5.397472 -4.145634 2 2
145
+ 137 67.8832 -6.184205 -6.375447 1 1
146
+ 138 67.3913 -4.477339 -4.545193 1 2
147
+ 139 66.9065 -5.022146 -4.584998 2 1
148
+ 140 67.1429 -1.511998 -1.200013 2 2
149
+ 141 67.3759 -4.474557 -3.960096 2 2
150
+ 142 67.6056 -2.980618 -2.862855 2 2
151
+ 143 67.8322 -1.863953 -6.483240 1 1
152
+ 144 67.3611 -2.074458 -1.968633 2 1
153
+ 145 66.8966 -1.856759 -14.360231 1 2
154
+ 146 67.1233 -4.220418 -3.734674 2 2
155
+ 147 67.3469 -5.172023 -5.121535 2 2
156
+ 148 67.5676 -2.035377 -2.578784 1 1
157
+ 149 67.7852 -3.423337 -3.355788 2 2
158
+ 150 68.0000 -3.965126 -3.232659 2 2
159
+ 151 67.5497 -2.427659 -2.245993 2 1
160
+ 152 67.1053 -3.892447 -4.056398 1 2
161
+ 153 67.3203 -4.387927 -4.636707 1 1
162
+ 154 67.5325 -3.760968 -3.941631 1 1
163
+ 155 67.7419 -2.144454 -2.099700 2 2
164
+ 156 67.3077 -2.564924 -2.407279 2 1
165
+ 157 67.5159 -3.645623 -4.799310 1 1
166
+ 158 67.7215 -4.320691 -3.993765 2 2
167
+ 159 67.9245 -3.388681 -5.209436 1 1
168
+ 160 68.1250 -4.176490 -3.004962 2 2
169
+ 161 68.3230 -3.345618 -3.699718 1 1
170
+ 162 67.9012 -0.999210 -14.527398 1 2
171
+ 163 68.0982 -1.997222 -2.020378 1 1
172
+ 164 68.2927 -5.135251 -4.463025 2 2
173
+ 165 67.8788 -4.561136 -4.691701 1 2
174
+ 166 68.0723 -3.994707 -3.661861 2 2
175
+ 167 67.6647 -5.565191 -5.328985 2 1
176
+ 168 67.2619 -4.589251 -5.087365 1 2
177
+ 169 66.8639 -3.659797 -4.197932 1 2
178
+ 170 66.4706 -4.815435 -4.216930 2 1
179
+ 171 66.6667 -3.389843 -4.680559 1 1
180
+ 172 66.8605 -4.928195 -5.679315 1 1
181
+ 173 67.0520 -3.532621 -3.043176 2 2
182
+ 174 67.2414 -7.174525 -6.593682 2 2
183
+ 175 66.8571 -4.788936 -4.518008 2 1
184
+ 176 67.0455 -1.524254 -3.458489 1 1
185
+ 177 67.2316 -5.528898 -4.988851 2 2
186
+ 178 66.8539 -2.618261 -2.206672 2 1
187
+ 179 66.4804 -2.924542 -2.885005 2 1
188
+ 180 66.6667 -5.630928 -5.145204 2 2
189
+ 181 66.8508 -3.893866 -2.914933 2 2
190
+ 182 67.0330 -4.760849 -4.372933 2 2
191
+ 183 67.2131 -6.427773 -6.154939 2 2
192
+ 184 67.3913 -2.342688 -3.597434 1 1
193
+ 185 67.0270 -6.181611 -6.374959 1 2
194
+ 186 67.2043 -3.237867 -3.825544 1 1
195
+ 187 67.3797 -4.992620 -4.812960 2 2
196
+ 188 67.0213 -3.281483 -3.178082 2 1
197
+ 189 66.6667 -5.170125 -5.389608 1 2
198
+ 190 66.8421 -5.231977 -4.599018 2 2
199
+ 191 67.0157 -4.091248 -4.187129 1 1
200
+ 192 67.1875 -5.957368 -4.208989 2 2
201
+ 193 67.3575 -4.127620 -4.054278 2 2
202
+ 194 67.5258 -2.096229 -1.367957 2 2
203
+ 195 67.1795 -4.110349 -4.057577 2 1
204
+ 196 67.3469 -4.061661 -3.314531 2 2
205
+ 197 67.5127 -0.912970 -3.258040 1 1
206
+ 198 67.6768 -5.338279 -5.871361 1 1
207
+ 199 67.8392 -4.379710 -3.863499 2 2
208
+ 200 68.0000 -3.251399 -3.152675 2 2
209
+ 201 68.1592 -2.655212 -3.446778 1 1
210
+ 202 68.3168 -7.849866 -7.130743 2 2
211
+ 203 68.4729 -3.464689 -3.394250 2 2
212
+ 204 68.6275 -3.269399 -3.415347 1 1
213
+ 205 68.7805 -4.125372 -3.188504 2 2
214
+ 206 68.4466 -5.857516 -6.193223 1 2
215
+ 207 68.5990 -2.164734 -2.537818 1 1
216
+ 208 68.7500 -4.564677 -3.110229 2 2
217
+ 209 68.8995 -4.648911 -4.249370 2 2
218
+ 210 68.5714 -7.781171 -8.527037 1 2
219
+ 211 68.2464 -6.496330 -5.935728 2 1
220
+ 212 68.3962 -4.669000 -3.415363 2 2
221
+ 213 68.0751 -5.293776 -5.797456 1 2
222
+ 214 68.2243 -2.861436 -3.083312 1 1
223
+ 215 68.3721 -5.710527 -4.967401 2 2
224
+ 216 68.0556 -3.743851 -3.634238 2 1
225
+ 217 68.2028 -1.044667 -1.619873 1 1
226
+ 218 68.3486 -2.560581 -2.477737 2 2
227
+ 219 68.4932 -2.734566 -3.201690 1 1
228
+ 220 68.1818 -3.284906 -3.440517 1 2
229
+ 221 68.3258 -2.646845 -8.728119 1 1
230
+ 222 68.0180 -0.249650 -5.923334 1 2
231
+ 223 68.1614 -4.624526 -4.428310 2 2
232
+ 224 67.8571 -5.991945 -6.463610 1 2
233
+ 225 68.0000 -3.906942 -3.123983 2 2
234
+ 226 68.1416 -3.110698 -3.807359 1 1
235
+ 227 67.8414 -4.409544 -3.593687 2 1
236
+ 228 67.9825 -1.584985 -3.342590 1 1
237
+ 229 68.1223 -4.795200 -4.592969 2 2
238
+ 230 68.2609 -3.252846 -3.414483 1 1
239
+ 231 68.3983 -2.074644 -2.411304 1 1
240
+ 232 68.5345 -2.426835 -1.722553 2 2
241
+ 233 68.6695 -2.044241 -2.646399 1 1
242
+ 234 68.8034 -4.320172 -3.748544 2 2
243
+ 235 68.9362 -3.937339 -3.436003 2 2
244
+ 236 68.6441 -2.594272 -2.346675 2 1
245
+ 237 68.3544 -0.993269 -0.924308 2 1
246
+ 238 68.4874 -3.472373 -3.099874 2 2
247
+ 239 68.6192 -2.695503 -2.400253 2 2
248
+ 240 68.3333 -7.094794 -7.406631 1 2
249
+ 241 68.4647 -3.853296 -3.029726 2 2
250
+ 242 68.5950 -6.401261 -5.885701 2 2
251
+ 243 68.7243 -4.285055 -4.506559 1 1
252
+ 244 68.8525 -2.055093 -3.489257 1 1
253
+ 245 68.9796 -4.902162 -5.706095 1 1
254
+ 246 69.1057 -2.580728 -3.460384 1 1
255
+ 247 68.8259 -4.841953 -5.512299 1 2
256
+ 248 68.9516 -3.117043 -2.925689 2 2
257
+ 249 69.0763 -3.170836 -1.897689 2 2
258
+ 250 69.2000 -3.223193 -3.885015 1 1
259
+ 251 68.9243 -0.917105 -0.828612 2 1
260
+ 252 69.0476 -6.718422 -6.517574 2 2
261
+ 253 69.1700 -5.258172 -5.641060 1 1
262
+ 254 69.2913 -4.002468 -3.301445 2 2
263
+ 255 69.4118 -3.333384 -4.897617 1 1
264
+ 256 69.5312 -3.762909 -4.245337 1 1
265
+ 257 69.2607 -2.667508 -2.172329 2 1
266
+ 258 68.9922 -3.459644 -3.837832 1 2
267
+ 259 68.7259 -4.296241 -4.142276 2 1
268
+ 260 68.8462 -4.670780 -5.146673 1 1
269
+ 261 68.9655 -3.649650 -3.552327 2 2
270
+ 262 69.0840 -5.404435 -5.284187 2 2
271
+ 263 69.2015 -4.013575 -5.416380 1 1
272
+ 264 69.3182 -4.618385 -4.209543 2 2
273
+ 265 69.4340 -1.547252 -2.123422 1 1
274
+ 266 69.5489 -6.228279 -8.746829 1 1
275
+ 267 69.6629 -3.839152 -3.867345 1 1
276
+ 268 69.4030 -2.896650 -2.653133 2 1
277
+ 269 69.5167 -3.553244 -3.132909 2 2
278
+ 270 69.6296 -2.605425 -4.325232 1 1
279
+ 271 69.7417 -3.062553 -2.940801 2 2
280
+ 272 69.8529 -4.354213 -3.832549 2 2
281
+ 273 69.9634 -3.689397 -3.397824 2 2
282
+ 274 70.0730 -3.021475 -2.353185 2 2
283
+ 275 70.1818 -1.294634 -2.022500 1 1
284
+ 276 69.9275 -2.557427 -2.444012 2 1
285
+ 277 69.6751 -3.336828 -3.286603 2 1
286
+ 278 69.7842 -2.636964 -2.646827 1 1
287
+ 279 69.8925 -4.269619 -3.134549 2 2
288
+ 280 70.0000 -3.953550 -3.626057 2 2
289
+ 281 70.1068 -2.366450 -1.853114 2 2
290
+ 282 70.2128 -4.224851 -3.766970 2 2
291
+ 283 70.3180 -3.356101 -3.298008 2 2
292
+ 284 70.4225 -2.741575 -2.835668 1 1
293
+ 285 70.5263 -6.081872 -6.447775 1 1
294
+ 286 70.6294 -3.509550 -3.565767 1 1
295
+ 287 70.7317 -2.430488 -3.878828 1 1
296
+ 288 70.8333 -3.595856 -2.884174 2 2
297
+ 289 70.9343 -2.532375 -3.012082 1 1
298
+ 290 71.0345 -5.478123 -6.332479 1 1
299
+ 291 71.1340 -1.934276 -3.334663 1 1
300
+ 292 71.2329 -2.491902 -2.205121 2 2
301
+ 293 71.3311 -3.620832 -2.898717 2 2
302
+ 294 71.0884 -5.014874 -3.755143 2 1
303
+ 295 71.1864 -1.985952 -2.417970 1 1
304
+ 296 71.2838 -4.880631 -4.719955 2 2
305
+ 297 71.0438 -1.024893 -1.247814 1 2
306
+ 298 70.8054 -6.475880 -7.794273 1 2
307
+ 299 70.9030 -2.526241 -1.908696 2 2
308
+ 300 71.0000 -3.248355 -3.877177 1 1
309
+ 301 71.0963 -4.596631 -5.245195 1 1
310
+ 302 71.1921 -4.551749 -4.500408 2 2
311
+ 303 70.9571 -9.148648 -7.746373 2 1
312
+ 304 70.7237 -3.490749 -3.499222 1 2
313
+ 305 70.4918 -2.801161 -3.108960 1 2
314
+ 306 70.5882 -2.653807 -2.243992 2 2
315
+ 307 70.6840 -2.731107 -2.898550 1 1
316
+ 308 70.4545 -4.518090 -4.810317 1 2
317
+ 309 70.5502 -3.236882 -3.946575 1 1
318
+ 310 70.6452 -5.119289 -4.905897 2 2
319
+ 311 70.7395 -1.726434 -5.283445 1 1
320
+ 312 70.5128 -5.724572 -4.413882 2 1
321
+ 313 70.2875 -4.591654 -5.501421 1 2
322
+ 314 70.3822 -6.194002 -5.532614 2 2
323
+ 315 70.4762 -2.416865 -3.656192 1 1
324
+ 316 70.5696 -6.825204 -5.893513 2 2
325
+ 317 70.3470 -4.697039 -4.464692 2 1
326
+ 318 70.4403 -5.102660 -4.627244 2 2
327
+ 319 70.5329 -2.375281 -3.336535 1 1
328
+ 320 70.6250 -3.793871 -5.449490 1 1
329
+ 321 70.4050 -3.059804 -3.083985 1 2
330
+ 322 70.1863 -2.613394 -2.624901 1 2
331
+ 323 70.2786 -2.767660 -3.232579 1 1
332
+ 324 70.3704 -4.263402 -4.381685 1 1
333
+ 325 70.4615 -5.662834 -4.755201 2 2
334
+ 326 70.5521 -2.886259 -2.094194 2 2
335
+ 327 70.6422 -2.757782 -3.533025 1 1
336
+ 328 70.7317 -4.181187 -4.506790 1 1
337
+ 329 70.8207 -4.082932 -11.074750 1 1
338
+ 330 70.9091 -0.970257 -1.840274 1 1
339
+ 331 70.6949 -5.945055 -4.129166 2 1
340
+ 332 70.7831 -1.916457 -2.869324 1 1
341
+ 333 70.8709 -2.748672 -2.508081 2 2
342
+ 334 70.9581 -2.590648 -5.710317 1 1
343
+ 335 71.0448 -4.482733 -4.629809 1 1
344
+ 336 70.8333 -3.396446 -4.021770 1 2
345
+ 337 70.6231 -1.928386 -2.026124 1 2
346
+ 338 70.4142 -4.405224 -3.558907 2 1
347
+ 339 70.5015 -5.890516 -5.003302 2 2
348
+ 340 70.5882 -5.267840 -5.774005 1 1
349
+ 341 70.6745 -2.862794 -4.550766 1 1
350
+ 342 70.7602 -6.219988 -6.059979 2 2
351
+ 343 70.8455 -3.803602 -4.255208 1 1
352
+ 344 70.6395 -2.838704 -2.758245 2 1
353
+ 345 70.7246 -1.303505 -3.170898 1 1
354
+ 346 70.5202 -4.600383 -6.589139 1 2
355
+ 347 70.6052 -4.038607 -3.631367 2 2
356
+ 348 70.6897 -1.248951 -2.664429 1 1
357
+ 349 70.4871 -3.572269 -5.543384 1 2
358
+ 350 70.2857 -1.142080 -4.867554 1 2
359
+ 351 70.3704 -2.229241 -3.193932 1 1
360
+ 352 70.4545 -3.877836 -2.927809 2 2
361
+ 353 70.2550 -2.301572 -1.977510 2 1
362
+ 354 70.3390 -5.313982 -3.743150 2 2
363
+ 355 70.4225 -3.083989 -2.515548 2 2
364
+ 356 70.5056 -3.796083 -3.495542 2 2
365
+ 357 70.5882 -4.215315 -3.871996 2 2
366
+ 358 70.6704 -2.573278 -2.507185 2 2
367
+ 359 70.7521 -6.242046 -3.878464 2 2
368
+ 360 70.5556 -2.744235 -3.114060 1 2
369
+ 361 70.6371 -3.566956 -1.687784 2 2
370
+ 362 70.4420 -5.348126 -5.394951 1 2
371
+ 363 70.2479 -2.470641 -2.419527 2 1
372
+ 364 70.3297 -6.165048 -6.209812 1 1
373
+ 365 70.4110 -5.747835 -6.352436 1 1
374
+ 366 70.2186 -4.722414 -10.606685 1 2
375
+ 367 70.2997 -5.460427 -5.732572 1 1
376
+ 368 70.3804 -3.287362 -1.502948 2 2
377
+ 369 70.1897 -3.173051 -3.511933 1 2
378
+ 370 70.2703 -4.577330 -4.882659 1 1
379
+ 371 70.3504 -4.467277 -5.074863 1 1
380
+ 372 70.1613 -2.651206 -2.341510 2 1
381
+ 373 70.2413 -5.488307 -5.771455 1 1
382
+ 374 70.3209 -1.981364 -2.913496 1 1
383
+ 375 70.4000 -3.238862 -2.749039 2 2
384
+ 376 70.4787 -5.446277 -3.581528 2 2
385
+ 377 70.2918 -3.820822 -4.158733 1 2
386
+ 378 70.1058 -3.007107 -3.329598 1 2
387
+ 379 69.9208 -1.702727 -2.747538 1 2
388
+ 380 69.7368 -7.929549 -7.350393 2 1
389
+ 381 69.8163 -9.983229 -8.884808 2 2
390
+ 382 69.6335 -2.357380 -2.397237 1 2
391
+ 383 69.7128 -1.900112 -0.850559 2 2
392
+ 384 69.5312 -4.008749 -2.991130 2 1
393
+ 385 69.3506 -1.528930 -1.123502 2 1
394
+ 386 69.4301 -2.942295 -9.603214 1 1
395
+ 387 69.5090 -1.488895 -2.044856 1 1
396
+ 388 69.3299 -5.365813 -6.096669 1 2
397
+ 389 69.4087 -2.940440 -2.244254 2 2
398
+ 390 69.4872 -2.662855 -3.872095 1 1
399
+ 391 69.3095 -3.854357 -5.123028 1 2
400
+ 392 69.1327 -5.686608 -4.445871 2 1
401
+ 393 69.2112 -7.679181 -6.749934 2 2
402
+ 394 69.2893 -2.045781 -4.975585 1 1
403
+ 395 69.3671 -4.369557 -4.339218 2 2
404
+ 396 69.4444 -4.696334 -5.168827 1 1
405
+ 397 69.2695 -7.451478 -6.853975 2 1
406
+ 398 69.3467 -5.284415 -8.163820 1 1
407
+ 399 69.4236 -5.248705 -5.421179 1 1
408
+ 400 69.5000 -2.069232 -5.004222 1 1
409
+ 401 69.5761 -2.668078 -3.017091 1 1
410
+ 402 69.4030 -3.909558 -4.056817 1 2
411
+ 403 69.2308 -6.621568 -6.924662 1 2
412
+ 404 69.3069 -3.792957 -2.415336 2 2
413
+ 405 69.3827 -3.269813 -4.322098 1 1
414
+ 406 69.4581 -8.165248 -9.376789 1 1
415
+ 407 69.2875 -5.306410 -5.332810 1 2
416
+ 408 69.3627 -3.303867 -3.009316 2 2
417
+ 409 69.4377 -3.975429 -4.279574 1 1
418
+ 410 69.5122 -5.108604 -5.728073 1 1
419
+ 411 69.5864 -5.686810 -6.140466 1 1
420
+ 412 69.6602 -4.221035 -3.114473 2 2
421
+ 413 69.7337 -4.200940 -2.913032 2 2
422
+ 414 69.5652 -4.894410 -5.183822 1 2
423
+ 415 69.3976 -2.743860 -2.758774 1 2
424
+ 416 69.4712 -5.450898 -3.341830 2 2
425
+ 417 69.5444 -4.896640 -5.671812 1 1
426
+ 418 69.6172 -5.223364 -7.824919 1 1
427
+ 419 69.4511 -4.458027 -5.022795 1 2
428
+ 420 69.2857 -3.340196 -3.432263 1 2
429
+ 421 69.3587 -4.168639 -3.420867 2 2
430
+ 422 69.1943 -2.630291 -2.325720 2 1
431
+ 423 69.0307 -3.503968 -3.227916 2 1
432
+ 424 69.1038 -5.413379 -5.922805 1 1
433
+ 425 69.1765 -4.220494 -3.345005 2 2
434
+ 426 69.0141 -1.903482 -2.545539 1 2
435
+ 427 69.0867 -3.582489 -4.804427 1 1
436
+ 428 68.9252 -3.965372 -3.095345 2 1
437
+ 429 68.9977 -4.125314 -4.017237 2 2
438
+ 430 68.8372 -6.201636 -5.699015 2 1
439
+ 431 68.9095 -4.778803 -5.852756 1 1
440
+ 432 68.7500 -4.238032 -4.743589 1 2
441
+ 433 68.8222 -1.988243 -1.698167 2 2
442
+ 434 68.8940 -7.299908 -7.904270 1 1
443
+ 435 68.9655 -3.277105 -3.761965 1 1
444
+ 436 69.0367 -0.722632 -2.284833 1 1
445
+ 437 69.1076 -3.620206 -3.445210 2 2
446
+ 438 69.1781 -3.941175 -3.834486 2 2
447
+ 439 69.0205 -3.929763 -2.840732 2 1
448
+ 440 69.0909 -4.568119 -5.293486 1 1
449
+ 441 69.1610 -3.947428 -5.191869 1 1
450
+ 442 69.0045 -4.906888 -4.899816 2 1
451
+ 443 69.0745 -1.509643 -3.440838 1 1
452
+ 444 68.9189 -5.909536 -7.512796 1 2
453
+ 445 68.9888 -1.796314 -3.094155 1 1
454
+ 446 69.0583 -6.093654 -5.010515 2 2
455
+ 447 69.1275 -1.806902 -1.844929 1 1
456
+ 448 69.1964 -4.164865 -3.607180 2 2
457
+ 449 69.2650 -3.007819 -2.904883 2 2
458
+ 450 69.3333 -4.163977 -3.707712 2 2
459
+ 451 69.4013 -4.474680 -3.333852 2 2
460
+ 452 69.4690 -5.102869 -5.329846 1 1
461
+ 453 69.5364 -5.491975 -4.853867 2 2
462
+ 454 69.6035 -3.065868 -3.529122 1 1
463
+ 455 69.4505 -1.762505 -9.117937 1 2
464
+ 456 69.5175 -4.552063 -4.481877 2 2
465
+ 457 69.3654 -10.508742 -9.923862 2 1
466
+ 458 69.4323 -4.233966 -3.892156 2 2
467
+ 459 69.4989 -4.640878 -4.374459 2 2
468
+ 460 69.5652 -3.720885 -8.372651 1 1
469
+ 461 69.6312 -2.828516 -3.862725 1 1
470
+ 462 69.6970 -4.733638 -5.072767 1 1
471
+ 463 69.7624 -0.956691 -1.155812 1 1
472
+ 464 69.6121 -2.826553 -2.720389 2 1
473
+ 465 69.4624 -3.213891 -3.333045 1 2
474
+ 466 69.5279 -5.392558 -4.964192 2 2
475
+ 467 69.5931 -5.142339 -5.414700 1 1
476
+ 468 69.6581 -5.476481 -4.021945 2 2
477
+ 469 69.5096 -2.046761 -1.762520 2 1
478
+ 470 69.5745 -1.566695 -1.329649 2 2
479
+ 471 69.6391 -2.324666 -2.374810 1 1
480
+ 472 69.7034 -4.201487 -3.792502 2 2
481
+ 473 69.7674 -4.441380 -3.124624 2 2
482
+ 474 69.8312 -5.213214 -5.630029 1 1
483
+ 475 69.8947 -1.688819 -1.318616 2 2
484
+ 476 69.7479 -4.289008 -4.567408 1 2
485
+ 477 69.6017 -4.772365 -5.328266 1 2
486
+ 478 69.4561 -4.438749 -3.954819 2 1
487
+ 479 69.5198 -3.230524 -3.584111 1 1
488
+ 480 69.3750 -5.225128 -5.484651 1 2
489
+ 481 69.4387 -2.173097 -2.575134 1 1
490
+ 482 69.2946 -4.366141 -5.347773 1 2
491
+ 483 69.1511 -5.710170 -4.973395 2 1
492
+ 484 69.0083 -2.556586 -1.743681 2 1
493
+ 485 69.0722 -1.849820 -2.682455 1 1
494
+ 486 69.1358 -2.749864 -3.547123 1 1
495
+ 487 69.1992 -2.777952 -1.914882 2 2
496
+ 488 69.2623 -3.309660 -2.308243 2 2
497
+ 489 69.3252 -2.388180 -2.502842 1 1
498
+ 490 69.3878 -3.154521 -3.722868 1 1
499
+ 491 69.4501 -5.712545 -4.869840 2 2
500
+ 492 69.5122 -5.617025 -5.282836 2 2
501
+ 493 69.5740 -1.681948 -3.363957 1 1
502
+ 494 69.6356 -4.738879 -4.600669 2 2
503
+ 495 69.6970 -4.226323 -4.384799 1 1
504
+ 496 69.5565 -3.578036 -3.838699 1 2
505
+ 497 69.4165 -6.032024 -5.426495 2 1
506
+ 498 69.2771 -1.435006 -1.870272 1 2
507
+ 499 69.3387 -2.575365 -3.161663 1 1
508
+ 500 69.4000 -5.479052 -4.513946 2 2
509
+ 501 69.4611 -1.539167 -2.662436 1 1
510
+ 502 69.5219 -1.442789 -2.154148 1 1
511
+ 503 69.5825 -2.968930 -2.656039 2 2
512
+ 504 69.6429 -2.773853 -2.875391 1 1
513
+ 505 69.5050 -7.039781 -7.892404 1 2
514
+ 506 69.3676 -6.744689 -6.219423 2 1
515
+ 507 69.4280 -4.438094 -4.775164 1 1
516
+ 508 69.4882 -6.357949 -2.667703 2 2
517
+ 509 69.5481 -3.392516 -7.960377 1 1
518
+ 510 69.6078 -3.376893 -3.655446 1 1
519
+ 511 69.6673 -5.252909 -3.934038 2 2
520
+ 512 69.7266 -3.699232 -4.290227 1 1
521
+ 513 69.7856 -3.959224 -3.720817 2 2
522
+ 514 69.8444 -5.351240 -5.584559 1 1
523
+ 515 69.9029 -6.198337 -6.305688 1 1
524
+ 516 69.9612 -4.414754 -4.502120 1 1
525
+ 517 69.8259 -1.799587 -1.613332 2 1
526
+ 518 69.8842 -2.944448 -1.770266 2 2
527
+ 519 69.9422 -9.584788 -9.379567 2 2
528
+ 520 70.0000 -2.656918 -2.072248 2 2
529
+ 521 70.0576 -1.156664 -1.344490 1 1
530
+ 522 70.1149 -3.831071 -3.179227 2 2
531
+ 523 69.9809 -4.042100 -4.170102 1 2
532
+ 524 69.8473 -7.407044 -7.105171 2 1
533
+ 525 69.9048 -5.089938 -1.840623 2 2
534
+ 526 69.9620 -0.542656 -17.279076 1 1
535
+ 527 70.0190 -7.026855 -5.135593 2 2
536
+ 528 69.8864 -6.271455 -6.303890 1 2
537
+ 529 69.9433 -4.848250 -6.173090 1 1
538
+ 530 70.0000 -5.430465 -4.376921 2 2
539
+ 531 69.8682 -4.122114 -3.733859 2 1
540
+ 532 69.9248 -2.311146 -3.155034 1 1
541
+ 533 69.9812 -3.404272 -2.823708 2 2
542
+ 534 70.0375 -2.924768 -3.562063 1 1
543
+ 535 69.9065 -3.331759 -3.406094 1 2
544
+ 536 69.7761 -2.541077 -2.300230 2 1
545
+ 537 69.8324 -5.885573 -6.431298 1 1
546
+ 538 69.8885 -2.576939 -3.004304 1 1
547
+ 539 69.9443 -4.389912 -3.068766 2 2
548
+ 540 69.8148 -2.412992 -3.037548 1 2
549
+ 541 69.8706 -2.259763 -2.964013 1 1
550
+ 542 69.7417 -4.057763 -5.206097 1 2
551
+ 543 69.7974 -5.297643 -4.815759 2 2
552
+ 544 69.8529 -2.610174 -4.482726 1 1
553
+ 545 69.7248 -2.014690 -5.658932 1 2
554
+ 546 69.7802 -2.741494 -3.868165 1 1
555
+ 547 69.8355 -5.954576 -4.660072 2 2
556
+ 548 69.7080 -4.575608 -4.470382 2 1
557
+ 549 69.5811 -5.426350 -5.417861 2 1
558
+ 550 69.6364 -3.433976 -3.404477 2 2
559
+ 551 69.5100 -2.668045 -7.523566 1 2
560
+ 552 69.5652 -2.648132 -3.136158 1 1
561
+ 553 69.6203 -6.123153 -5.544426 2 2
562
+ 554 69.4946 -4.765023 -5.070056 1 2
563
+ 555 69.5495 -3.034923 -2.174982 2 2
564
+ 556 69.6043 -5.884892 -5.241424 2 2
565
+ 557 69.6589 -4.797234 -5.567226 1 1
566
+ 558 69.7133 -2.072560 -3.357495 1 1
567
+ 559 69.7674 -2.654027 -2.514177 2 2
568
+ 560 69.6429 -2.761665 -1.849671 2 1
569
+ 561 69.6970 -2.197950 -2.494146 1 1
570
+ 562 69.7509 -1.119561 -1.798128 1 1
571
+ 563 69.6270 -1.526100 -2.926300 1 2
572
+ 564 69.6809 -3.275733 -3.737848 1 1
573
+ 565 69.7345 -3.785984 -3.616386 2 2
574
+ 566 69.7880 -4.615633 -5.048630 1 1
575
+ 567 69.8413 -3.022582 -2.820016 2 2
576
+ 568 69.8944 -6.914006 -4.407721 2 2
577
+ 569 69.7715 -5.654963 -5.032475 2 1
578
+ 570 69.8246 -4.669339 -7.053414 1 1
579
+ 571 69.8774 -4.685478 -3.426779 2 2
580
+ 572 69.9301 -2.827241 -2.656799 2 2
581
+ 573 69.8080 -4.258932 -4.661815 1 2
582
+ 574 69.8606 -3.723971 -2.543332 2 2
583
+ 575 69.9130 -4.842198 -4.490517 2 2
584
+ 576 69.9653 -2.919610 -3.545695 1 1
585
+ 577 70.0173 -4.530261 -4.481139 2 2
586
+ 578 70.0692 -3.269032 -3.043366 2 2
587
+ 579 69.9482 -5.512830 -5.638696 1 2
588
+ 580 70.0000 -1.780371 -3.269184 1 1
589
+ 581 70.0516 -5.620058 -7.905339 1 1
590
+ 582 69.9313 -5.251323 -5.542115 1 2
591
+ 583 69.9828 -4.556337 -4.316332 2 2
592
+ 584 69.8630 -4.425739 -4.394640 2 1
593
+ 585 69.9145 -0.388025 -1.111220 1 1
594
+ 586 69.9659 -3.911305 -4.284649 1 1
595
+ 587 69.8467 -5.417719 -5.061099 2 1
596
+ 588 69.8980 -4.206219 -2.345398 2 2
597
+ 589 69.7793 -3.260532 -3.641219 1 2
598
+ 590 69.8305 -4.104585 -3.698344 2 2
599
+ 591 69.8816 -4.154311 -3.669987 2 2
600
+ 592 69.9324 -3.625489 -4.328778 1 1
601
+ 593 69.9831 -4.684032 -4.556803 2 2
602
+ 594 69.8653 -5.160190 -4.359279 2 1
603
+ 595 69.9160 -2.051816 -1.973316 2 2
604
+ 596 69.9664 -4.974429 -4.564087 2 2
605
+ 597 70.0168 -2.355077 -2.360125 1 1
606
+ 598 70.0669 -2.845233 -3.265114 1 1
607
+ 599 70.1169 -2.990542 -4.079846 1 1
608
+ 600 70.1667 -3.745246 -3.476212 2 2
609
+ 601 70.2163 -3.319216 -2.736051 2 2
610
+ 602 70.2658 -3.143051 -5.542419 1 1
611
+ 603 70.3151 -6.808855 -5.423609 2 2
612
+ 604 70.1987 -3.051769 -3.130581 1 2
613
+ 605 70.0826 -3.109874 -2.293938 2 1
614
+ 606 70.1320 -3.207237 -2.928685 2 2
615
+ 607 70.0165 -2.951934 -2.569860 2 1
616
+ 608 70.0658 -3.158537 -3.637910 1 1
617
+ 609 69.9507 -7.459409 -7.102287 2 1
618
+ 610 70.0000 -3.064738 -3.367223 1 1
619
+ 611 70.0491 -4.519129 -4.580837 1 1
620
+ 612 70.0980 -5.176344 -6.754026 1 1
621
+ 613 70.1468 -3.081501 -2.221454 2 2
622
+ 614 70.1954 -5.705805 -5.664178 2 2
623
+ 615 70.2439 -3.166543 -2.851213 2 2
624
+ 616 70.2922 -5.305129 -5.272236 2 2
625
+ 617 70.3404 -13.412498 -5.834319 2 2
626
+ 618 70.2265 -3.971817 -4.228084 1 2
627
+ 619 70.2746 -3.869037 -3.438065 2 2
628
+ 620 70.1613 -5.069096 -4.065609 2 1
629
+ 621 70.2093 -3.368500 -2.145600 2 2
630
+ 622 70.2572 -3.323439 -3.745980 1 1
631
+ 623 70.3050 -2.959069 -3.211200 1 1
632
+ 624 70.3526 -4.671082 -3.883812 2 2
633
+ 625 70.4000 -4.682712 -6.352824 1 1
634
+ 626 70.2875 -1.893933 -1.179802 2 1
635
+ 627 70.3349 -3.705614 -4.091403 1 1
636
+ 628 70.3822 -3.322555 -2.551188 2 2
637
+ 629 70.4293 -1.986267 -2.515737 1 1
638
+ 630 70.4762 -1.239971 -1.751315 1 1
639
+ 631 70.5230 -2.644982 -4.004336 1 1
640
+ 632 70.5696 -1.874139 -1.597370 2 2
641
+ 633 70.6161 -2.000730 -1.925812 2 2
642
+ 634 70.6625 -2.757235 -3.772527 1 1
643
+ 635 70.5512 -4.730506 -4.905050 1 2
644
+ 636 70.5975 -1.461271 -2.797151 1 1
645
+ 637 70.6436 -3.140676 -5.462288 1 1
646
+ 638 70.6897 -5.902725 -6.698486 1 1
647
+ 639 70.7355 -2.360276 -2.645629 1 1
648
+ 640 70.6250 -2.115700 -2.676571 1 2
649
+ 641 70.5148 -6.667477 -5.635036 2 1
650
+ 642 70.5607 -5.035706 -3.674045 2 2
651
+ 643 70.6065 -2.265760 -1.434148 2 2
652
+ 644 70.6522 -3.084568 -4.064160 1 1
653
+ 645 70.6977 -0.978713 -2.443040 1 1
654
+ 646 70.5882 -3.902581 -3.592080 2 1
655
+ 647 70.4791 -3.119206 -2.467090 2 1
656
+ 648 70.5247 -3.201815 -3.084535 2 2
657
+ 649 70.5701 -3.475242 -3.838659 1 1
658
+ 650 70.6154 -3.967690 -4.603557 1 1
659
+ 651 70.5069 -5.120611 -5.925172 1 2
660
+ 652 70.5521 -5.501304 -5.460017 2 2
661
+ 653 70.5972 -5.170744 -3.642822 2 2
662
+ 654 70.4893 -3.404265 -3.887279 1 2
663
+ 655 70.3817 -2.497442 -2.650023 1 2
664
+ 656 70.4268 -4.245106 -3.179453 2 2
665
+ 657 70.3196 -5.292535 -5.547687 1 2
666
+ 658 70.3647 -4.009687 -1.647013 2 2
667
+ 659 70.4097 -2.531841 -1.770687 2 2
668
+ 660 70.3030 -3.254519 -4.011966 1 2
669
+ 661 70.3480 -3.508361 -3.879687 1 1
670
+ 662 70.3927 -3.022896 -3.879846 1 1
671
+ 663 70.4374 -6.479685 -6.214353 2 2
672
+ 664 70.3313 -3.972317 -3.925757 2 1
673
+ 665 70.3759 -3.371000 -3.247629 2 2
674
+ 666 70.4204 -3.505053 -4.889327 1 1
675
+ 667 70.4648 -7.939797 -7.248271 2 2
676
+ 668 70.3593 -2.403075 -2.268040 2 1
677
+ 669 70.4036 -2.422266 -3.133674 1 1
678
+ 670 70.2985 -3.557386 -3.386888 2 1
679
+ 671 70.3428 -3.340419 -4.709842 1 1
680
+ 672 70.3869 -3.390465 -3.111589 2 2
681
+ 673 70.4309 -1.783365 -1.718011 2 2
682
+ 674 70.3264 -4.936859 -4.726672 2 1
683
+ 675 70.2222 -4.011875 -3.556189 2 1
684
+ 676 70.1183 -2.215070 -2.592355 1 2
685
+ 677 70.1625 -3.498026 -3.779798 1 1
686
+ 678 70.0590 -5.623339 -5.950517 1 2
687
+ 679 70.1031 -5.540056 -4.463776 2 2
688
+ 680 70.1471 -3.580551 -4.018096 1 1
689
+ 681 70.1909 -2.978276 -3.427854 1 1
690
+ 682 70.0880 -2.345735 -2.451858 1 2
691
+ 683 70.1318 -2.065431 -3.497201 1 1
692
+ 684 70.1754 -5.121928 -7.306860 1 1
693
+ 685 70.2190 -4.795786 -2.373288 2 2
694
+ 686 70.1166 -5.564743 -5.955367 1 2
695
+ 687 70.1601 -4.651850 -3.487548 2 2
696
+ 688 70.2035 -8.138879 -8.205161 1 1
697
+ 689 70.2467 -2.879905 -2.422815 2 2
698
+ 690 70.1449 -2.473386 -2.648684 1 2
699
+ 691 70.0434 -3.042105 -2.551355 2 1
700
+ 692 70.0867 -1.947185 -2.619661 1 1
701
+ 693 70.1299 -4.773826 -5.401369 1 1
702
+ 694 70.0288 -3.488933 -3.532379 1 2
703
+ 695 70.0719 -4.050951 -4.940554 1 1
704
+ 696 70.1149 -4.518286 -5.673456 1 1
705
+ 697 70.1578 -2.273226 -2.608215 1 1
706
+ 698 70.2006 -4.416105 -4.069875 2 2
707
+ 699 70.2432 -3.534481 -4.534797 1 1
708
+ 700 70.2857 -4.325931 -3.995691 2 2
709
+ 701 70.1854 -2.837253 -2.593859 2 1
710
+ 702 70.0855 -4.228606 -5.137162 1 2
711
+ 703 70.1280 -3.695224 -3.949299 1 1
712
+ 704 70.1705 -0.941732 -1.422772 1 1
713
+ 705 70.0709 -3.161763 -3.490399 1 2
714
+ 706 69.9717 -3.470689 -3.480359 1 2
715
+ 707 69.8727 -4.841407 -4.574115 2 1
716
+ 708 69.9153 -4.262401 -3.367423 2 2
717
+ 709 69.9577 -3.382539 -2.783618 2 2
718
+ 710 70.0000 -3.690862 -4.623950 1 1
719
+ 711 69.9015 -1.782774 -2.238926 1 2
720
+ 712 69.8034 -2.054266 -3.884121 1 2
721
+ 713 69.7055 -3.454745 -3.493011 1 2
722
+ 714 69.6078 -3.242656 -3.389874 1 2
723
+ 715 69.6503 -4.106730 -3.283767 2 2
724
+ 716 69.5531 -4.765044 -4.835474 1 2
725
+ 717 69.5955 -1.786045 -0.265451 2 2
726
+ 718 69.4986 -4.529393 -4.288341 2 1
727
+ 719 69.4019 -4.868754 -4.530795 2 1
728
+ 720 69.3056 -2.464010 -5.392143 1 2
729
+ 721 69.2094 -5.102406 -5.696659 1 2
730
+ 722 69.1136 -2.040098 -1.989437 2 1
731
+ 723 69.1563 -2.328966 -2.857133 1 1
732
+ 724 69.1989 -4.073266 -3.965217 2 2
733
+ 725 69.1034 -2.110902 -2.540121 1 2
734
+ 726 69.1460 -4.758505 -4.416762 2 2
735
+ 727 69.1884 -2.994897 -2.532300 2 2
736
+ 728 69.0934 -3.439942 -3.398085 2 1
737
+ 729 69.1358 -1.391775 -2.805561 1 1
738
+ 730 69.1781 -4.153939 -4.862218 1 1
739
+ 731 69.2202 -4.638055 -4.729624 1 1
740
+ 732 69.2623 -2.935532 -2.089134 2 2
741
+ 733 69.3042 -7.074907 -6.510735 2 2
742
+ 734 69.3460 -1.200704 -11.672219 1 1
743
+ 735 69.2517 -3.871348 -3.264516 2 1
744
+ 736 69.2935 -2.331333 -3.716166 1 1
745
+ 737 69.3351 -4.618137 -5.446051 1 1
746
+ 738 69.2412 -4.298767 -4.643084 1 2
747
+ 739 69.2828 -0.731766 -5.938642 1 1
748
+ 740 69.3243 -4.791785 -3.101359 2 2
749
+ 741 69.2308 -4.584694 -4.519419 2 1
750
+ 742 69.2722 -2.263741 -1.998016 2 2
751
+ 743 69.3136 -4.115119 -3.972441 2 2
752
+ 744 69.3548 -4.728223 -4.097148 2 2
753
+ 745 69.3960 -2.760117 -3.423476 1 1
754
+ 746 69.4370 -7.386808 -7.633314 1 1
755
+ 747 69.4779 -5.410932 -6.608427 1 1
756
+ 748 69.5187 -4.661203 -3.783610 2 2
757
+ 749 69.5594 -4.262673 -5.315158 1 1
758
+ 750 69.6000 -2.247389 -7.314926 1 1
759
+
760
+ Final Winogrande score(750 tasks): 69.6000 +/- 1.6807
scores/Hammer2.1-7b-iq3_s.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 100.00000000
13
+ 3 100.00000000
14
+ 4 75.00000000
15
+ 5 80.00000000
16
+ 6 83.33333333
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 66.66666667
20
+ 10 70.00000000
21
+ 11 63.63636364
22
+ 12 66.66666667
23
+ 13 69.23076923
24
+ 14 71.42857143
25
+ 15 73.33333333
26
+ 16 68.75000000
27
+ 17 70.58823529
28
+ 18 66.66666667
29
+ 19 63.15789474
30
+ 20 60.00000000
31
+ 21 57.14285714
32
+ 22 59.09090909
33
+ 23 60.86956522
34
+ 24 62.50000000
35
+ 25 60.00000000
36
+ 26 57.69230769
37
+ 27 59.25925926
38
+ 28 57.14285714
39
+ 29 58.62068966
40
+ 30 60.00000000
41
+ 31 58.06451613
42
+ 32 56.25000000
43
+ 33 54.54545455
44
+ 34 52.94117647
45
+ 35 54.28571429
46
+ 36 52.77777778
47
+ 37 54.05405405
48
+ 38 52.63157895
49
+ 39 53.84615385
50
+ 40 55.00000000
51
+ 41 53.65853659
52
+ 42 52.38095238
53
+ 43 51.16279070
54
+ 44 50.00000000
55
+ 45 51.11111111
56
+ 46 52.17391304
57
+ 47 53.19148936
58
+ 48 54.16666667
59
+ 49 53.06122449
60
+ 50 52.00000000
61
+ 51 50.98039216
62
+ 52 51.92307692
63
+ 53 52.83018868
64
+ 54 51.85185185
65
+ 55 50.90909091
66
+ 56 51.78571429
67
+ 57 52.63157895
68
+ 58 53.44827586
69
+ 59 54.23728814
70
+ 60 55.00000000
71
+ 61 54.09836066
72
+ 62 54.83870968
73
+ 63 55.55555556
74
+ 64 56.25000000
75
+ 65 55.38461538
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 56.52173913
80
+ 70 55.71428571
81
+ 71 54.92957746
82
+ 72 54.16666667
83
+ 73 54.79452055
84
+ 74 55.40540541
85
+ 75 56.00000000
86
+ 76 55.26315789
87
+ 77 55.84415584
88
+ 78 56.41025641
89
+ 79 55.69620253
90
+ 80 56.25000000
91
+ 81 55.55555556
92
+ 82 56.09756098
93
+ 83 55.42168675
94
+ 84 54.76190476
95
+ 85 55.29411765
96
+ 86 55.81395349
97
+ 87 56.32183908
98
+ 88 56.81818182
99
+ 89 57.30337079
100
+ 90 56.66666667
101
+ 91 57.14285714
102
+ 92 57.60869565
103
+ 93 56.98924731
104
+ 94 56.38297872
105
+ 95 56.84210526
106
+ 96 57.29166667
107
+ 97 56.70103093
108
+ 98 56.12244898
109
+ 99 56.56565657
110
+ 100 57.00000000
111
+ 101 56.43564356
112
+ 102 56.86274510
113
+ 103 56.31067961
114
+ 104 55.76923077
115
+ 105 56.19047619
116
+ 106 55.66037736
117
+ 107 56.07476636
118
+ 108 55.55555556
119
+ 109 55.04587156
120
+ 110 55.45454545
121
+ 111 55.85585586
122
+ 112 56.25000000
123
+ 113 55.75221239
124
+ 114 56.14035088
125
+ 115 56.52173913
126
+ 116 56.03448276
127
+ 117 55.55555556
128
+ 118 55.93220339
129
+ 119 55.46218487
130
+ 120 55.83333333
131
+ 121 55.37190083
132
+ 122 55.73770492
133
+ 123 56.09756098
134
+ 124 55.64516129
135
+ 125 56.00000000
136
+ 126 56.34920635
137
+ 127 55.90551181
138
+ 128 56.25000000
139
+ 129 55.81395349
140
+ 130 55.38461538
141
+ 131 55.72519084
142
+ 132 56.06060606
143
+ 133 56.39097744
144
+ 134 56.71641791
145
+ 135 57.03703704
146
+ 136 56.61764706
147
+ 137 56.93430657
148
+ 138 57.24637681
149
+ 139 57.55395683
150
+ 140 57.85714286
151
+ 141 58.15602837
152
+ 142 57.74647887
153
+ 143 58.04195804
154
+ 144 58.33333333
155
+ 145 58.62068966
156
+ 146 58.21917808
157
+ 147 58.50340136
158
+ 148 58.10810811
159
+ 149 58.38926174
160
+ 150 58.66666667
161
+ 151 58.94039735
162
+ 152 59.21052632
163
+ 153 59.47712418
164
+ 154 59.09090909
165
+ 155 58.70967742
166
+ 156 58.97435897
167
+ 157 59.23566879
168
+ 158 58.86075949
169
+ 159 59.11949686
170
+ 160 58.75000000
171
+ 161 58.38509317
172
+ 162 58.64197531
173
+ 163 58.28220859
174
+ 164 57.92682927
175
+ 165 57.57575758
176
+ 166 57.83132530
177
+ 167 57.48502994
178
+ 168 57.73809524
179
+ 169 57.98816568
180
+ 170 58.23529412
181
+ 171 57.89473684
182
+ 172 58.13953488
183
+ 173 58.38150289
184
+ 174 58.62068966
185
+ 175 58.28571429
186
+ 176 57.95454545
187
+ 177 58.19209040
188
+ 178 57.86516854
189
+ 179 57.54189944
190
+ 180 57.22222222
191
+ 181 57.45856354
192
+ 182 57.69230769
193
+ 183 57.37704918
194
+ 184 57.60869565
195
+ 185 57.83783784
196
+ 186 58.06451613
197
+ 187 58.28877005
198
+ 188 57.97872340
199
+ 189 57.67195767
200
+ 190 57.36842105
201
+ 191 57.59162304
202
+ 192 57.29166667
203
+ 193 56.99481865
204
+ 194 57.21649485
205
+ 195 57.43589744
206
+ 196 57.65306122
207
+ 197 57.86802030
208
+ 198 57.57575758
209
+ 199 57.28643216
210
+ 200 57.50000000
211
+ 201 57.71144279
212
+ 202 57.92079208
213
+ 203 58.12807882
214
+ 204 58.33333333
215
+ 205 58.04878049
216
+ 206 57.76699029
217
+ 207 57.48792271
218
+ 208 57.69230769
219
+ 209 57.89473684
220
+ 210 58.09523810
221
+ 211 58.29383886
222
+ 212 58.01886792
223
+ 213 58.21596244
224
+ 214 58.41121495
225
+ 215 58.60465116
226
+ 216 58.33333333
227
+ 217 58.06451613
228
+ 218 57.79816514
229
+ 219 57.99086758
230
+ 220 57.72727273
231
+ 221 57.46606335
232
+ 222 57.65765766
233
+ 223 57.39910314
234
+ 224 57.58928571
235
+ 225 57.33333333
236
+ 226 57.07964602
237
+ 227 57.26872247
238
+ 228 57.45614035
239
+ 229 57.64192140
240
+ 230 57.82608696
241
+ 231 57.57575758
242
+ 232 57.32758621
243
+ 233 57.51072961
244
+ 234 57.26495726
245
+ 235 57.02127660
246
+ 236 57.20338983
247
+ 237 56.96202532
248
+ 238 57.14285714
249
+ 239 57.32217573
250
+ 240 57.50000000
251
+ 241 57.67634855
252
+ 242 57.43801653
253
+ 243 57.20164609
254
+ 244 56.96721311
255
+ 245 57.14285714
256
+ 246 57.31707317
257
+ 247 57.48987854
258
+ 248 57.66129032
259
+ 249 57.83132530
260
+ 250 58.00000000
261
+ 251 58.16733068
262
+ 252 58.33333333
263
+ 253 58.49802372
264
+ 254 58.26771654
265
+ 255 58.43137255
266
+ 256 58.59375000
267
+ 257 58.75486381
268
+ 258 58.52713178
269
+ 259 58.30115830
270
+ 260 58.46153846
271
+ 261 58.62068966
272
+ 262 58.77862595
273
+ 263 58.93536122
274
+ 264 59.09090909
275
+ 265 59.24528302
276
+ 266 59.02255639
277
+ 267 59.17602996
278
+ 268 58.95522388
279
+ 269 59.10780669
280
+ 270 59.25925926
281
+ 271 59.40959410
282
+ 272 59.19117647
283
+ 273 58.97435897
284
+ 274 58.75912409
285
+ 275 58.54545455
286
+ 276 58.69565217
287
+ 277 58.84476534
288
+ 278 58.63309353
289
+ 279 58.78136201
290
+ 280 58.92857143
291
+ 281 59.07473310
292
+ 282 59.21985816
293
+ 283 59.01060071
294
+ 284 59.15492958
295
+ 285 59.29824561
296
+ 286 59.44055944
297
+ 287 59.23344948
298
+ 288 59.37500000
299
+ 289 59.16955017
300
+ 290 59.31034483
301
+ 291 59.45017182
302
+ 292 59.24657534
303
+ 293 59.38566553
304
+ 294 59.18367347
305
+ 295 59.32203390
306
+ 296 59.45945946
307
+ 297 59.59595960
308
+ 298 59.73154362
309
+ 299 59.86622074
310
+ 300 59.66666667
311
+ 301 59.46843854
312
+ 302 59.60264901
313
+ 303 59.73597360
314
+ 304 59.86842105
315
+ 305 60.00000000
316
+ 306 59.80392157
317
+ 307 59.93485342
318
+ 308 60.06493506
319
+ 309 60.19417476
320
+ 310 60.00000000
321
+ 311 60.12861736
322
+ 312 59.93589744
323
+ 313 60.06389776
324
+ 314 59.87261146
325
+ 315 60.00000000
326
+ 316 60.12658228
327
+ 317 60.25236593
328
+ 318 60.06289308
329
+ 319 59.87460815
330
+ 320 59.68750000
331
+ 321 59.50155763
332
+ 322 59.62732919
333
+ 323 59.75232198
334
+ 324 59.56790123
335
+ 325 59.38461538
336
+ 326 59.50920245
337
+ 327 59.32721713
338
+ 328 59.45121951
339
+ 329 59.57446809
340
+ 330 59.69696970
341
+ 331 59.81873112
342
+ 332 59.93975904
343
+ 333 60.06006006
344
+ 334 59.88023952
345
+ 335 59.70149254
346
+ 336 59.82142857
347
+ 337 59.94065282
348
+ 338 59.76331361
349
+ 339 59.58702065
350
+ 340 59.70588235
351
+ 341 59.82404692
352
+ 342 59.94152047
353
+ 343 60.05830904
354
+ 344 60.17441860
355
+ 345 60.00000000
356
+ 346 60.11560694
357
+ 347 59.94236311
358
+ 348 59.77011494
359
+ 349 59.88538682
360
+ 350 59.71428571
361
+ 351 59.82905983
362
+ 352 59.94318182
363
+ 353 60.05665722
364
+ 354 59.88700565
365
+ 355 59.71830986
366
+ 356 59.55056180
367
+ 357 59.38375350
368
+ 358 59.49720670
369
+ 359 59.33147632
370
+ 360 59.44444444
371
+ 361 59.55678670
372
+ 362 59.66850829
373
+ 363 59.77961433
374
+ 364 59.61538462
375
+ 365 59.45205479
376
+ 366 59.28961749
377
+ 367 59.40054496
378
+ 368 59.51086957
379
+ 369 59.62059621
380
+ 370 59.72972973
381
+ 371 59.56873315
382
+ 372 59.40860215
383
+ 373 59.24932976
384
+ 374 59.35828877
385
+ 375 59.20000000
386
+ 376 59.04255319
387
+ 377 58.88594164
388
+ 378 58.99470899
389
+ 379 59.10290237
390
+ 380 59.21052632
391
+ 381 59.05511811
392
+ 382 58.90052356
393
+ 383 58.74673629
394
+ 384 58.85416667
395
+ 385 58.96103896
396
+ 386 59.06735751
397
+ 387 58.91472868
398
+ 388 58.76288660
399
+ 389 58.61182519
400
+ 390 58.46153846
401
+ 391 58.31202046
402
+ 392 58.41836735
403
+ 393 58.26972010
404
+ 394 58.37563452
405
+ 395 58.48101266
406
+ 396 58.33333333
407
+ 397 58.43828715
408
+ 398 58.29145729
409
+ 399 58.39598997
410
+ 400 58.50000000
411
+ 401 58.60349127
412
+ 402 58.70646766
413
+ 403 58.80893300
414
+ 404 58.91089109
415
+ 405 59.01234568
416
+ 406 59.11330049
417
+ 407 59.21375921
418
+ 408 59.31372549
419
+ 409 59.16870416
420
+ 410 59.26829268
421
+ 411 59.36739659
422
+ 412 59.46601942
423
+ 413 59.32203390
424
+ 414 59.42028986
425
+ 415 59.27710843
426
+ 416 59.37500000
427
+ 417 59.23261391
428
+ 418 59.09090909
429
+ 419 58.94988067
430
+ 420 58.80952381
431
+ 421 58.66983373
432
+ 422 58.76777251
433
+ 423 58.86524823
434
+ 424 58.72641509
435
+ 425 58.82352941
436
+ 426 58.68544601
437
+ 427 58.78220141
438
+ 428 58.64485981
439
+ 429 58.74125874
440
+ 430 58.60465116
441
+ 431 58.70069606
442
+ 432 58.79629630
443
+ 433 58.66050808
444
+ 434 58.75576037
445
+ 435 58.85057471
446
+ 436 58.94495413
447
+ 437 59.03890160
448
+ 438 58.90410959
449
+ 439 58.99772210
450
+ 440 58.86363636
451
+ 441 58.95691610
452
+ 442 58.82352941
453
+ 443 58.69074492
454
+ 444 58.78378378
455
+ 445 58.87640449
456
+ 446 58.96860987
457
+ 447 59.06040268
458
+ 448 58.92857143
459
+ 449 59.02004454
460
+ 450 58.88888889
461
+ 451 58.98004435
462
+ 452 59.07079646
463
+ 453 58.94039735
464
+ 454 58.81057269
465
+ 455 58.90109890
466
+ 456 58.77192982
467
+ 457 58.86214442
468
+ 458 58.95196507
469
+ 459 59.04139434
470
+ 460 58.91304348
471
+ 461 58.78524946
472
+ 462 58.87445887
473
+ 463 58.74730022
474
+ 464 58.62068966
475
+ 465 58.49462366
476
+ 466 58.58369099
477
+ 467 58.67237687
478
+ 468 58.76068376
479
+ 469 58.63539446
480
+ 470 58.72340426
481
+ 471 58.81104034
482
+ 472 58.68644068
483
+ 473 58.56236786
484
+ 474 58.64978903
485
+ 475 58.52631579
486
+ 476 58.61344538
487
+ 477 58.70020964
488
+ 478 58.78661088
489
+ 479 58.87265136
490
+ 480 58.95833333
491
+ 481 58.83575884
492
+ 482 58.92116183
493
+ 483 59.00621118
494
+ 484 58.88429752
495
+ 485 58.96907216
496
+ 486 59.05349794
497
+ 487 58.93223819
498
+ 488 59.01639344
499
+ 489 58.89570552
500
+ 490 58.77551020
501
+ 491 58.85947047
502
+ 492 58.94308943
503
+ 493 58.82352941
504
+ 494 58.90688259
505
+ 495 58.78787879
506
+ 496 58.87096774
507
+ 497 58.95372233
508
+ 498 59.03614458
509
+ 499 59.11823647
510
+ 500 59.20000000
511
+ 501 59.08183633
512
+ 502 59.16334661
513
+ 503 59.24453280
514
+ 504 59.12698413
515
+ 505 59.20792079
516
+ 506 59.09090909
517
+ 507 58.97435897
518
+ 508 58.85826772
519
+ 509 58.74263261
520
+ 510 58.82352941
521
+ 511 58.70841487
522
+ 512 58.78906250
523
+ 513 58.67446394
524
+ 514 58.75486381
525
+ 515 58.83495146
526
+ 516 58.72093023
527
+ 517 58.80077369
528
+ 518 58.88030888
529
+ 519 58.76685934
530
+ 520 58.65384615
531
+ 521 58.73320537
532
+ 522 58.62068966
533
+ 523 58.50860421
534
+ 524 58.39694656
535
+ 525 58.28571429
536
+ 526 58.17490494
537
+ 527 58.06451613
538
+ 528 57.95454545
539
+ 529 57.84499055
540
+ 530 57.92452830
541
+ 531 57.81544256
542
+ 532 57.89473684
543
+ 533 57.97373358
544
+ 534 57.86516854
545
+ 535 57.75700935
546
+ 536 57.83582090
547
+ 537 57.91433892
548
+ 538 57.80669145
549
+ 539 57.88497217
550
+ 540 57.77777778
551
+ 541 57.85582255
552
+ 542 57.74907749
553
+ 543 57.82688766
554
+ 544 57.72058824
555
+ 545 57.79816514
556
+ 546 57.69230769
557
+ 547 57.58683729
558
+ 548 57.66423358
559
+ 549 57.74134791
560
+ 550 57.81818182
561
+ 551 57.71324864
562
+ 552 57.60869565
563
+ 553 57.68535262
564
+ 554 57.76173285
565
+ 555 57.83783784
566
+ 556 57.73381295
567
+ 557 57.80969479
568
+ 558 57.70609319
569
+ 559 57.78175313
570
+ 560 57.85714286
571
+ 561 57.93226381
572
+ 562 58.00711744
573
+ 563 58.08170515
574
+ 564 57.97872340
575
+ 565 57.87610619
576
+ 566 57.95053004
577
+ 567 57.84832451
578
+ 568 57.92253521
579
+ 569 57.82073814
580
+ 570 57.71929825
581
+ 571 57.79334501
582
+ 572 57.69230769
583
+ 573 57.59162304
584
+ 574 57.66550523
585
+ 575 57.73913043
586
+ 576 57.81250000
587
+ 577 57.88561525
588
+ 578 57.95847751
589
+ 579 58.03108808
590
+ 580 57.93103448
591
+ 581 57.83132530
592
+ 582 57.73195876
593
+ 583 57.80445969
594
+ 584 57.87671233
595
+ 585 57.94871795
596
+ 586 58.02047782
597
+ 587 57.92163543
598
+ 588 57.82312925
599
+ 589 57.89473684
600
+ 590 57.96610169
601
+ 591 58.03722504
602
+ 592 57.93918919
603
+ 593 57.84148398
604
+ 594 57.91245791
605
+ 595 57.81512605
606
+ 596 57.71812081
607
+ 597 57.62144054
608
+ 598 57.52508361
609
+ 599 57.59599332
610
+ 600 57.66666667
611
+ 601 57.73710483
612
+ 602 57.64119601
613
+ 603 57.54560531
614
+ 604 57.45033113
615
+ 605 57.35537190
616
+ 606 57.42574257
617
+ 607 57.49588138
618
+ 608 57.56578947
619
+ 609 57.47126437
620
+ 610 57.54098361
621
+ 611 57.44680851
622
+ 612 57.35294118
623
+ 613 57.25938010
624
+ 614 57.16612378
625
+ 615 57.07317073
626
+ 616 56.98051948
627
+ 617 57.05024311
628
+ 618 56.95792880
629
+ 619 56.86591276
630
+ 620 56.93548387
631
+ 621 56.84380032
632
+ 622 56.75241158
633
+ 623 56.82182986
634
+ 624 56.73076923
635
+ 625 56.64000000
636
+ 626 56.54952077
637
+ 627 56.61881978
638
+ 628 56.52866242
639
+ 629 56.59777424
640
+ 630 56.66666667
641
+ 631 56.57686212
642
+ 632 56.48734177
643
+ 633 56.39810427
644
+ 634 56.46687697
645
+ 635 56.53543307
646
+ 636 56.60377358
647
+ 637 56.51491366
648
+ 638 56.58307210
649
+ 639 56.65101721
650
+ 640 56.56250000
651
+ 641 56.63026521
652
+ 642 56.54205607
653
+ 643 56.45412131
654
+ 644 56.52173913
655
+ 645 56.43410853
656
+ 646 56.50154799
657
+ 647 56.56877898
658
+ 648 56.48148148
659
+ 649 56.39445300
660
+ 650 56.30769231
661
+ 651 56.37480799
662
+ 652 56.28834356
663
+ 653 56.35528331
664
+ 654 56.42201835
665
+ 655 56.33587786
666
+ 656 56.40243902
667
+ 657 56.31659056
668
+ 658 56.23100304
669
+ 659 56.29742033
670
+ 660 56.36363636
671
+ 661 56.27836611
672
+ 662 56.34441088
673
+ 663 56.41025641
674
+ 664 56.47590361
675
+ 665 56.54135338
676
+ 666 56.45645646
677
+ 667 56.52173913
678
+ 668 56.43712575
679
+ 669 56.35276532
680
+ 670 56.41791045
681
+ 671 56.48286140
682
+ 672 56.54761905
683
+ 673 56.46359584
684
+ 674 56.52818991
685
+ 675 56.59259259
686
+ 676 56.65680473
687
+ 677 56.72082718
688
+ 678 56.78466077
689
+ 679 56.84830633
690
+ 680 56.91176471
691
+ 681 56.82819383
692
+ 682 56.74486804
693
+ 683 56.66178624
694
+ 684 56.72514620
695
+ 685 56.78832117
696
+ 686 56.70553936
697
+ 687 56.62299854
698
+ 688 56.68604651
699
+ 689 56.74891147
700
+ 690 56.81159420
701
+ 691 56.87409551
702
+ 692 56.93641618
703
+ 693 56.99855700
704
+ 694 57.06051873
705
+ 695 56.97841727
706
+ 696 56.89655172
707
+ 697 56.95839311
708
+ 698 56.87679083
709
+ 699 56.79542203
710
+ 700 56.85714286
711
+ 701 56.91868759
712
+ 702 56.98005698
713
+ 703 56.89900427
714
+ 704 56.96022727
715
+ 705 57.02127660
716
+ 706 57.08215297
717
+ 707 57.14285714
718
+ 708 57.20338983
719
+ 709 57.26375176
720
+ 710 57.32394366
721
+ 711 57.38396624
722
+ 712 57.44382022
723
+ 713 57.50350631
724
+ 714 57.42296919
725
+ 715 57.34265734
726
+ 716 57.40223464
727
+ 717 57.32217573
728
+ 718 57.24233983
729
+ 719 57.16272601
730
+ 720 57.22222222
731
+ 721 57.14285714
732
+ 722 57.20221607
733
+ 723 57.26141079
734
+ 724 57.32044199
735
+ 725 57.37931034
736
+ 726 57.30027548
737
+ 727 57.22145805
738
+ 728 57.28021978
739
+ 729 57.33882030
740
+ 730 57.39726027
741
+ 731 57.45554036
742
+ 732 57.51366120
743
+ 733 57.57162347
744
+ 734 57.62942779
745
+ 735 57.68707483
746
+ 736 57.74456522
747
+ 737 57.66621438
748
+ 738 57.58807588
749
+ 739 57.51014885
750
+ 740 57.43243243
751
+ 741 57.48987854
752
+ 742 57.54716981
753
+ 743 57.60430686
754
+ 744 57.52688172
755
+ 745 57.58389262
756
+ 746 57.64075067
757
+ 747 57.56358768
758
+ 748 57.48663102
759
+ 749 57.40987984
760
+ 750 57.46666667
761
+
762
+ Final result: 57.4667 +/- 1.8065
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-iq3_s.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 57.14285714
18
+ 8 62.50000000
19
+ 9 66.66666667
20
+ 10 70.00000000
21
+ 11 72.72727273
22
+ 12 66.66666667
23
+ 13 69.23076923
24
+ 14 71.42857143
25
+ 15 73.33333333
26
+ 16 75.00000000
27
+ 17 70.58823529
28
+ 18 66.66666667
29
+ 19 63.15789474
30
+ 20 65.00000000
31
+ 21 66.66666667
32
+ 22 68.18181818
33
+ 23 69.56521739
34
+ 24 66.66666667
35
+ 25 64.00000000
36
+ 26 61.53846154
37
+ 27 59.25925926
38
+ 28 60.71428571
39
+ 29 62.06896552
40
+ 30 60.00000000
41
+ 31 61.29032258
42
+ 32 59.37500000
43
+ 33 60.60606061
44
+ 34 61.76470588
45
+ 35 62.85714286
46
+ 36 61.11111111
47
+ 37 59.45945946
48
+ 38 60.52631579
49
+ 39 58.97435897
50
+ 40 60.00000000
51
+ 41 60.97560976
52
+ 42 59.52380952
53
+ 43 58.13953488
54
+ 44 59.09090909
55
+ 45 57.77777778
56
+ 46 58.69565217
57
+ 47 57.44680851
58
+ 48 56.25000000
59
+ 49 57.14285714
60
+ 50 58.00000000
61
+ 51 58.82352941
62
+ 52 57.69230769
63
+ 53 56.60377358
64
+ 54 55.55555556
65
+ 55 56.36363636
66
+ 56 55.35714286
67
+ 57 56.14035088
68
+ 58 55.17241379
69
+ 59 55.93220339
70
+ 60 56.66666667
71
+ 61 55.73770492
72
+ 62 54.83870968
73
+ 63 55.55555556
74
+ 64 56.25000000
75
+ 65 55.38461538
76
+ 66 54.54545455
77
+ 67 55.22388060
78
+ 68 55.88235294
79
+ 69 56.52173913
80
+ 70 57.14285714
81
+ 71 56.33802817
82
+ 72 56.94444444
83
+ 73 57.53424658
84
+ 74 58.10810811
85
+ 75 57.33333333
86
+ 76 57.89473684
87
+ 77 58.44155844
88
+ 78 58.97435897
89
+ 79 59.49367089
90
+ 80 60.00000000
91
+ 81 59.25925926
92
+ 82 59.75609756
93
+ 83 60.24096386
94
+ 84 60.71428571
95
+ 85 60.00000000
96
+ 86 60.46511628
97
+ 87 60.91954023
98
+ 88 61.36363636
99
+ 89 60.67415730
100
+ 90 61.11111111
101
+ 91 61.53846154
102
+ 92 60.86956522
103
+ 93 61.29032258
104
+ 94 61.70212766
105
+ 95 62.10526316
106
+ 96 62.50000000
107
+ 97 62.88659794
108
+ 98 63.26530612
109
+ 99 63.63636364
110
+ 100 63.00000000
111
+ 101 63.36633663
112
+ 102 62.74509804
113
+ 103 62.13592233
114
+ 104 62.50000000
115
+ 105 62.85714286
116
+ 106 63.20754717
117
+ 107 63.55140187
118
+ 108 63.88888889
119
+ 109 63.30275229
120
+ 110 63.63636364
121
+ 111 63.06306306
122
+ 112 63.39285714
123
+ 113 63.71681416
124
+ 114 64.03508772
125
+ 115 64.34782609
126
+ 116 63.79310345
127
+ 117 63.24786325
128
+ 118 63.55932203
129
+ 119 63.86554622
130
+ 120 63.33333333
131
+ 121 62.80991736
132
+ 122 63.11475410
133
+ 123 63.41463415
134
+ 124 62.90322581
135
+ 125 63.20000000
136
+ 126 63.49206349
137
+ 127 63.77952756
138
+ 128 63.28125000
139
+ 129 63.56589147
140
+ 130 63.07692308
141
+ 131 63.35877863
142
+ 132 62.87878788
143
+ 133 63.15789474
144
+ 134 63.43283582
145
+ 135 63.70370370
146
+ 136 63.97058824
147
+ 137 63.50364964
148
+ 138 63.76811594
149
+ 139 64.02877698
150
+ 140 64.28571429
151
+ 141 64.53900709
152
+ 142 64.78873239
153
+ 143 65.03496503
154
+ 144 65.27777778
155
+ 145 65.51724138
156
+ 146 65.75342466
157
+ 147 65.98639456
158
+ 148 66.21621622
159
+ 149 66.44295302
160
+ 150 66.66666667
161
+ 151 66.22516556
162
+ 152 65.78947368
163
+ 153 66.01307190
164
+ 154 66.23376623
165
+ 155 66.45161290
166
+ 156 66.02564103
167
+ 157 65.60509554
168
+ 158 65.82278481
169
+ 159 65.40880503
170
+ 160 65.62500000
171
+ 161 65.83850932
172
+ 162 66.04938272
173
+ 163 66.25766871
174
+ 164 66.46341463
175
+ 165 66.66666667
176
+ 166 66.86746988
177
+ 167 67.06586826
178
+ 168 67.26190476
179
+ 169 66.86390533
180
+ 170 67.05882353
181
+ 171 67.25146199
182
+ 172 67.44186047
183
+ 173 67.05202312
184
+ 174 66.66666667
185
+ 175 66.85714286
186
+ 176 67.04545455
187
+ 177 67.23163842
188
+ 178 67.41573034
189
+ 179 67.03910615
190
+ 180 67.22222222
191
+ 181 66.85082873
192
+ 182 67.03296703
193
+ 183 67.21311475
194
+ 184 66.84782609
195
+ 185 67.02702703
196
+ 186 66.66666667
197
+ 187 66.31016043
198
+ 188 65.95744681
199
+ 189 65.60846561
200
+ 190 65.78947368
201
+ 191 65.96858639
202
+ 192 66.14583333
203
+ 193 65.80310881
204
+ 194 65.97938144
205
+ 195 65.64102564
206
+ 196 65.30612245
207
+ 197 64.97461929
208
+ 198 64.64646465
209
+ 199 64.82412060
210
+ 200 65.00000000
211
+ 201 65.17412935
212
+ 202 65.34653465
213
+ 203 65.02463054
214
+ 204 65.19607843
215
+ 205 64.87804878
216
+ 206 65.04854369
217
+ 207 65.21739130
218
+ 208 64.90384615
219
+ 209 65.07177033
220
+ 210 65.23809524
221
+ 211 65.40284360
222
+ 212 65.56603774
223
+ 213 65.72769953
224
+ 214 65.88785047
225
+ 215 65.58139535
226
+ 216 65.74074074
227
+ 217 65.43778802
228
+ 218 65.13761468
229
+ 219 65.29680365
230
+ 220 65.45454545
231
+ 221 65.15837104
232
+ 222 65.31531532
233
+ 223 65.47085202
234
+ 224 65.62500000
235
+ 225 65.77777778
236
+ 226 65.92920354
237
+ 227 65.63876652
238
+ 228 65.78947368
239
+ 229 65.93886463
240
+ 230 65.65217391
241
+ 231 65.36796537
242
+ 232 65.08620690
243
+ 233 64.80686695
244
+ 234 64.95726496
245
+ 235 64.68085106
246
+ 236 64.40677966
247
+ 237 64.55696203
248
+ 238 64.70588235
249
+ 239 64.85355649
250
+ 240 65.00000000
251
+ 241 65.14522822
252
+ 242 65.28925620
253
+ 243 65.43209877
254
+ 244 65.16393443
255
+ 245 64.89795918
256
+ 246 65.04065041
257
+ 247 64.77732794
258
+ 248 64.91935484
259
+ 249 65.06024096
260
+ 250 65.20000000
261
+ 251 65.33864542
262
+ 252 65.47619048
263
+ 253 65.21739130
264
+ 254 65.35433071
265
+ 255 65.49019608
266
+ 256 65.62500000
267
+ 257 65.36964981
268
+ 258 65.50387597
269
+ 259 65.63706564
270
+ 260 65.76923077
271
+ 261 65.90038314
272
+ 262 65.64885496
273
+ 263 65.39923954
274
+ 264 65.53030303
275
+ 265 65.66037736
276
+ 266 65.41353383
277
+ 267 65.54307116
278
+ 268 65.67164179
279
+ 269 65.79925651
280
+ 270 65.92592593
281
+ 271 65.68265683
282
+ 272 65.80882353
283
+ 273 65.56776557
284
+ 274 65.69343066
285
+ 275 65.45454545
286
+ 276 65.57971014
287
+ 277 65.70397112
288
+ 278 65.82733813
289
+ 279 65.94982079
290
+ 280 66.07142857
291
+ 281 66.19217082
292
+ 282 66.31205674
293
+ 283 66.07773852
294
+ 284 66.19718310
295
+ 285 65.96491228
296
+ 286 65.73426573
297
+ 287 65.85365854
298
+ 288 65.97222222
299
+ 289 66.08996540
300
+ 290 66.20689655
301
+ 291 66.32302405
302
+ 292 66.09589041
303
+ 293 66.21160410
304
+ 294 66.32653061
305
+ 295 66.44067797
306
+ 296 66.55405405
307
+ 297 66.66666667
308
+ 298 66.77852349
309
+ 299 66.55518395
310
+ 300 66.33333333
311
+ 301 66.11295681
312
+ 302 66.22516556
313
+ 303 66.00660066
314
+ 304 65.78947368
315
+ 305 65.90163934
316
+ 306 66.01307190
317
+ 307 65.79804560
318
+ 308 65.90909091
319
+ 309 66.01941748
320
+ 310 66.12903226
321
+ 311 66.23794212
322
+ 312 66.34615385
323
+ 313 66.45367412
324
+ 314 66.56050955
325
+ 315 66.66666667
326
+ 316 66.77215190
327
+ 317 66.87697161
328
+ 318 66.66666667
329
+ 319 66.45768025
330
+ 320 66.56250000
331
+ 321 66.35514019
332
+ 322 66.14906832
333
+ 323 66.25386997
334
+ 324 66.35802469
335
+ 325 66.46153846
336
+ 326 66.25766871
337
+ 327 66.05504587
338
+ 328 66.15853659
339
+ 329 66.26139818
340
+ 330 66.36363636
341
+ 331 66.46525680
342
+ 332 66.26506024
343
+ 333 66.36636637
344
+ 334 66.16766467
345
+ 335 66.26865672
346
+ 336 66.36904762
347
+ 337 66.17210682
348
+ 338 65.97633136
349
+ 339 66.07669617
350
+ 340 66.17647059
351
+ 341 66.27565982
352
+ 342 66.37426901
353
+ 343 66.47230321
354
+ 344 66.27906977
355
+ 345 66.37681159
356
+ 346 66.18497110
357
+ 347 66.28242075
358
+ 348 66.09195402
359
+ 349 65.90257880
360
+ 350 66.00000000
361
+ 351 66.09686610
362
+ 352 66.19318182
363
+ 353 66.00566572
364
+ 354 66.10169492
365
+ 355 66.19718310
366
+ 356 66.29213483
367
+ 357 66.38655462
368
+ 358 66.48044693
369
+ 359 66.57381616
370
+ 360 66.38888889
371
+ 361 66.48199446
372
+ 362 66.57458564
373
+ 363 66.66666667
374
+ 364 66.75824176
375
+ 365 66.84931507
376
+ 366 66.93989071
377
+ 367 67.02997275
378
+ 368 67.11956522
379
+ 369 67.20867209
380
+ 370 67.02702703
381
+ 371 67.11590296
382
+ 372 67.20430108
383
+ 373 67.02412869
384
+ 374 66.84491979
385
+ 375 66.93333333
386
+ 376 67.02127660
387
+ 377 66.84350133
388
+ 378 66.93121693
389
+ 379 67.01846966
390
+ 380 67.10526316
391
+ 381 67.19160105
392
+ 382 67.01570681
393
+ 383 67.10182768
394
+ 384 67.18750000
395
+ 385 67.27272727
396
+ 386 67.35751295
397
+ 387 67.44186047
398
+ 388 67.26804124
399
+ 389 67.35218509
400
+ 390 67.43589744
401
+ 391 67.26342711
402
+ 392 67.34693878
403
+ 393 67.43002545
404
+ 394 67.51269036
405
+ 395 67.59493671
406
+ 396 67.67676768
407
+ 397 67.75818640
408
+ 398 67.83919598
409
+ 399 67.91979950
410
+ 400 67.75000000
411
+ 401 67.83042394
412
+ 402 67.66169154
413
+ 403 67.49379653
414
+ 404 67.57425743
415
+ 405 67.40740741
416
+ 406 67.48768473
417
+ 407 67.56756757
418
+ 408 67.64705882
419
+ 409 67.72616137
420
+ 410 67.56097561
421
+ 411 67.39659367
422
+ 412 67.23300971
423
+ 413 67.31234867
424
+ 414 67.39130435
425
+ 415 67.22891566
426
+ 416 67.30769231
427
+ 417 67.38609113
428
+ 418 67.46411483
429
+ 419 67.54176611
430
+ 420 67.61904762
431
+ 421 67.45843230
432
+ 422 67.53554502
433
+ 423 67.61229314
434
+ 424 67.68867925
435
+ 425 67.76470588
436
+ 426 67.84037559
437
+ 427 67.68149883
438
+ 428 67.75700935
439
+ 429 67.59906760
440
+ 430 67.67441860
441
+ 431 67.74941995
442
+ 432 67.82407407
443
+ 433 67.89838337
444
+ 434 67.97235023
445
+ 435 67.81609195
446
+ 436 67.88990826
447
+ 437 67.73455378
448
+ 438 67.80821918
449
+ 439 67.88154897
450
+ 440 67.95454545
451
+ 441 68.02721088
452
+ 442 68.09954751
453
+ 443 68.17155756
454
+ 444 68.24324324
455
+ 445 68.31460674
456
+ 446 68.38565022
457
+ 447 68.45637584
458
+ 448 68.52678571
459
+ 449 68.59688196
460
+ 450 68.66666667
461
+ 451 68.51441242
462
+ 452 68.58407080
463
+ 453 68.65342163
464
+ 454 68.72246696
465
+ 455 68.79120879
466
+ 456 68.85964912
467
+ 457 68.92778993
468
+ 458 68.99563319
469
+ 459 69.06318083
470
+ 460 69.13043478
471
+ 461 69.19739696
472
+ 462 69.26406926
473
+ 463 69.33045356
474
+ 464 69.39655172
475
+ 465 69.24731183
476
+ 466 69.31330472
477
+ 467 69.37901499
478
+ 468 69.23076923
479
+ 469 69.08315565
480
+ 470 68.93617021
481
+ 471 69.00212314
482
+ 472 69.06779661
483
+ 473 69.13319239
484
+ 474 69.19831224
485
+ 475 69.26315789
486
+ 476 69.11764706
487
+ 477 68.97274633
488
+ 478 69.03765690
489
+ 479 69.10229645
490
+ 480 69.16666667
491
+ 481 69.02286902
492
+ 482 68.87966805
493
+ 483 68.94409938
494
+ 484 69.00826446
495
+ 485 68.86597938
496
+ 486 68.93004115
497
+ 487 68.99383984
498
+ 488 68.85245902
499
+ 489 68.91615542
500
+ 490 68.97959184
501
+ 491 69.04276986
502
+ 492 69.10569106
503
+ 493 69.16835700
504
+ 494 69.23076923
505
+ 495 69.29292929
506
+ 496 69.35483871
507
+ 497 69.41649899
508
+ 498 69.47791165
509
+ 499 69.33867735
510
+ 500 69.40000000
511
+ 501 69.46107784
512
+ 502 69.52191235
513
+ 503 69.58250497
514
+ 504 69.64285714
515
+ 505 69.70297030
516
+ 506 69.76284585
517
+ 507 69.62524655
518
+ 508 69.68503937
519
+ 509 69.74459725
520
+ 510 69.80392157
521
+ 511 69.66731898
522
+ 512 69.53125000
523
+ 513 69.39571150
524
+ 514 69.45525292
525
+ 515 69.51456311
526
+ 516 69.37984496
527
+ 517 69.24564797
528
+ 518 69.30501931
529
+ 519 69.17148362
530
+ 520 69.23076923
531
+ 521 69.09788868
532
+ 522 68.96551724
533
+ 523 69.02485660
534
+ 524 69.08396947
535
+ 525 69.14285714
536
+ 526 69.01140684
537
+ 527 68.88045541
538
+ 528 68.93939394
539
+ 529 68.80907372
540
+ 530 68.86792453
541
+ 531 68.92655367
542
+ 532 68.98496241
543
+ 533 68.85553471
544
+ 534 68.91385768
545
+ 535 68.97196262
546
+ 536 69.02985075
547
+ 537 69.08752328
548
+ 538 69.14498141
549
+ 539 69.01669759
550
+ 540 69.07407407
551
+ 541 69.13123845
552
+ 542 69.18819188
553
+ 543 69.24493554
554
+ 544 69.30147059
555
+ 545 69.35779817
556
+ 546 69.41391941
557
+ 547 69.46983547
558
+ 548 69.52554745
559
+ 549 69.58105647
560
+ 550 69.45454545
561
+ 551 69.50998185
562
+ 552 69.38405797
563
+ 553 69.43942134
564
+ 554 69.49458484
565
+ 555 69.54954955
566
+ 556 69.60431655
567
+ 557 69.47935368
568
+ 558 69.35483871
569
+ 559 69.23076923
570
+ 560 69.28571429
571
+ 561 69.34046346
572
+ 562 69.21708185
573
+ 563 69.27175844
574
+ 564 69.14893617
575
+ 565 69.02654867
576
+ 566 68.90459364
577
+ 567 68.78306878
578
+ 568 68.83802817
579
+ 569 68.71704745
580
+ 570 68.77192982
581
+ 571 68.82661996
582
+ 572 68.88111888
583
+ 573 68.93542757
584
+ 574 68.98954704
585
+ 575 69.04347826
586
+ 576 69.09722222
587
+ 577 68.97746967
588
+ 578 69.03114187
589
+ 579 69.08462867
590
+ 580 68.96551724
591
+ 581 69.01893287
592
+ 582 69.07216495
593
+ 583 69.12521441
594
+ 584 69.17808219
595
+ 585 69.23076923
596
+ 586 69.11262799
597
+ 587 69.16524702
598
+ 588 69.04761905
599
+ 589 68.93039049
600
+ 590 68.81355932
601
+ 591 68.86632826
602
+ 592 68.91891892
603
+ 593 68.80269815
604
+ 594 68.68686869
605
+ 595 68.73949580
606
+ 596 68.79194631
607
+ 597 68.84422111
608
+ 598 68.89632107
609
+ 599 68.78130217
610
+ 600 68.83333333
611
+ 601 68.88519135
612
+ 602 68.93687708
613
+ 603 68.82255390
614
+ 604 68.87417219
615
+ 605 68.92561983
616
+ 606 68.81188119
617
+ 607 68.86326194
618
+ 608 68.75000000
619
+ 609 68.80131363
620
+ 610 68.85245902
621
+ 611 68.90343699
622
+ 612 68.79084967
623
+ 613 68.84176183
624
+ 614 68.72964169
625
+ 615 68.78048780
626
+ 616 68.66883117
627
+ 617 68.71961102
628
+ 618 68.77022654
629
+ 619 68.65912763
630
+ 620 68.70967742
631
+ 621 68.59903382
632
+ 622 68.64951768
633
+ 623 68.69983949
634
+ 624 68.75000000
635
+ 625 68.80000000
636
+ 626 68.69009585
637
+ 627 68.74003190
638
+ 628 68.78980892
639
+ 629 68.68044515
640
+ 630 68.73015873
641
+ 631 68.77971474
642
+ 632 68.67088608
643
+ 633 68.72037915
644
+ 634 68.76971609
645
+ 635 68.81889764
646
+ 636 68.86792453
647
+ 637 68.91679749
648
+ 638 68.96551724
649
+ 639 69.01408451
650
+ 640 68.90625000
651
+ 641 68.95475819
652
+ 642 69.00311526
653
+ 643 69.05132193
654
+ 644 68.94409938
655
+ 645 68.99224806
656
+ 646 69.04024768
657
+ 647 69.08809892
658
+ 648 69.13580247
659
+ 649 69.02927581
660
+ 650 69.07692308
661
+ 651 69.12442396
662
+ 652 69.17177914
663
+ 653 69.21898928
664
+ 654 69.26605505
665
+ 655 69.31297710
666
+ 656 69.35975610
667
+ 657 69.40639269
668
+ 658 69.30091185
669
+ 659 69.19575114
670
+ 660 69.24242424
671
+ 661 69.28895613
672
+ 662 69.33534743
673
+ 663 69.38159879
674
+ 664 69.42771084
675
+ 665 69.47368421
676
+ 666 69.51951952
677
+ 667 69.56521739
678
+ 668 69.61077844
679
+ 669 69.65620329
680
+ 670 69.70149254
681
+ 671 69.74664680
682
+ 672 69.79166667
683
+ 673 69.83655275
684
+ 674 69.73293769
685
+ 675 69.77777778
686
+ 676 69.82248521
687
+ 677 69.86706056
688
+ 678 69.91150442
689
+ 679 69.95581738
690
+ 680 70.00000000
691
+ 681 69.89720999
692
+ 682 69.94134897
693
+ 683 69.98535871
694
+ 684 70.02923977
695
+ 685 69.92700730
696
+ 686 69.97084548
697
+ 687 70.01455604
698
+ 688 70.05813953
699
+ 689 70.10159652
700
+ 690 70.14492754
701
+ 691 70.18813314
702
+ 692 70.23121387
703
+ 693 70.12987013
704
+ 694 70.17291066
705
+ 695 70.21582734
706
+ 696 70.11494253
707
+ 697 70.01434720
708
+ 698 70.05730659
709
+ 699 69.95708155
710
+ 700 70.00000000
711
+ 701 70.04279601
712
+ 702 70.08547009
713
+ 703 70.12802276
714
+ 704 70.17045455
715
+ 705 70.21276596
716
+ 706 70.25495751
717
+ 707 70.29702970
718
+ 708 70.33898305
719
+ 709 70.38081805
720
+ 710 70.42253521
721
+ 711 70.46413502
722
+ 712 70.50561798
723
+ 713 70.54698457
724
+ 714 70.58823529
725
+ 715 70.62937063
726
+ 716 70.67039106
727
+ 717 70.71129707
728
+ 718 70.75208914
729
+ 719 70.65368567
730
+ 720 70.69444444
731
+ 721 70.59639390
732
+ 722 70.63711911
733
+ 723 70.67773167
734
+ 724 70.71823204
735
+ 725 70.75862069
736
+ 726 70.79889807
737
+ 727 70.83906465
738
+ 728 70.87912088
739
+ 729 70.91906722
740
+ 730 70.82191781
741
+ 731 70.86183311
742
+ 732 70.76502732
743
+ 733 70.80491132
744
+ 734 70.84468665
745
+ 735 70.88435374
746
+ 736 70.78804348
747
+ 737 70.69199457
748
+ 738 70.59620596
749
+ 739 70.50067659
750
+ 740 70.54054054
751
+ 741 70.58029690
752
+ 742 70.61994609
753
+ 743 70.65948856
754
+ 744 70.69892473
755
+ 745 70.60402685
756
+ 746 70.64343164
757
+ 747 70.54886212
758
+ 748 70.58823529
759
+ 749 70.62750334
760
+ 750 70.66666667
scores/Hammer2.1-7b-iq3_s.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 0.00000000
13
+ 3 0.00000000
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 28.57142857
18
+ 8 37.50000000
19
+ 9 33.33333333
20
+ 10 30.00000000
21
+ 11 27.27272727
22
+ 12 33.33333333
23
+ 13 38.46153846
24
+ 14 35.71428571
25
+ 15 33.33333333
26
+ 16 37.50000000
27
+ 17 41.17647059
28
+ 18 38.88888889
29
+ 19 42.10526316
30
+ 20 40.00000000
31
+ 21 38.09523810
32
+ 22 36.36363636
33
+ 23 34.78260870
34
+ 24 37.50000000
35
+ 25 40.00000000
36
+ 26 38.46153846
37
+ 27 40.74074074
38
+ 28 39.28571429
39
+ 29 41.37931034
40
+ 30 40.00000000
41
+ 31 41.93548387
42
+ 32 43.75000000
43
+ 33 42.42424242
44
+ 34 44.11764706
45
+ 35 45.71428571
46
+ 36 44.44444444
47
+ 37 43.24324324
48
+ 38 42.10526316
49
+ 39 41.02564103
50
+ 40 40.00000000
51
+ 41 39.02439024
52
+ 42 38.09523810
53
+ 43 39.53488372
54
+ 44 38.63636364
55
+ 45 37.77777778
56
+ 46 36.95652174
57
+ 47 36.17021277
58
+ 48 37.50000000
59
+ 49 38.77551020
60
+ 50 38.00000000
61
+ 51 37.25490196
62
+ 52 36.53846154
63
+ 53 35.84905660
64
+ 54 35.18518519
65
+ 55 36.36363636
66
+ 56 37.50000000
67
+ 57 36.84210526
68
+ 58 37.93103448
69
+ 59 38.98305085
70
+ 60 38.33333333
71
+ 61 37.70491803
72
+ 62 37.09677419
73
+ 63 36.50793651
74
+ 64 35.93750000
75
+ 65 36.92307692
76
+ 66 37.87878788
77
+ 67 37.31343284
78
+ 68 36.76470588
79
+ 69 36.23188406
80
+ 70 37.14285714
81
+ 71 38.02816901
82
+ 72 37.50000000
83
+ 73 36.98630137
84
+ 74 37.83783784
85
+ 75 38.66666667
86
+ 76 39.47368421
87
+ 77 40.25974026
88
+ 78 41.02564103
89
+ 79 40.50632911
90
+ 80 41.25000000
91
+ 81 41.97530864
92
+ 82 41.46341463
93
+ 83 42.16867470
94
+ 84 42.85714286
95
+ 85 43.52941176
96
+ 86 43.02325581
97
+ 87 43.67816092
98
+ 88 44.31818182
99
+ 89 43.82022472
100
+ 90 43.33333333
101
+ 91 42.85714286
102
+ 92 42.39130435
103
+ 93 41.93548387
104
+ 94 42.55319149
105
+ 95 43.15789474
106
+ 96 43.75000000
107
+ 97 43.29896907
108
+ 98 42.85714286
109
+ 99 42.42424242
110
+ 100 42.00000000
111
+ 101 42.57425743
112
+ 102 42.15686275
113
+ 103 41.74757282
114
+ 104 42.30769231
115
+ 105 41.90476190
116
+ 106 42.45283019
117
+ 107 42.05607477
118
+ 108 42.59259259
119
+ 109 42.20183486
120
+ 110 41.81818182
121
+ 111 41.44144144
122
+ 112 41.96428571
123
+ 113 41.59292035
124
+ 114 42.10526316
125
+ 115 42.60869565
126
+ 116 42.24137931
127
+ 117 41.88034188
128
+ 118 41.52542373
129
+ 119 41.17647059
130
+ 120 40.83333333
131
+ 121 41.32231405
132
+ 122 40.98360656
133
+ 123 40.65040650
134
+ 124 40.32258065
135
+ 125 40.80000000
136
+ 126 40.47619048
137
+ 127 40.15748031
138
+ 128 39.84375000
139
+ 129 39.53488372
140
+ 130 39.23076923
141
+ 131 39.69465649
142
+ 132 39.39393939
143
+ 133 39.09774436
144
+ 134 38.80597015
145
+ 135 38.51851852
146
+ 136 38.97058824
147
+ 137 39.41605839
148
+ 138 39.13043478
149
+ 139 39.56834532
150
+ 140 40.00000000
151
+ 141 39.71631206
152
+ 142 40.14084507
153
+ 143 40.55944056
154
+ 144 40.27777778
155
+ 145 40.00000000
156
+ 146 40.41095890
157
+ 147 40.13605442
158
+ 148 40.54054054
159
+ 149 40.93959732
160
+ 150 40.66666667
161
+ 151 40.39735099
162
+ 152 40.78947368
163
+ 153 41.17647059
164
+ 154 40.90909091
165
+ 155 40.64516129
166
+ 156 40.38461538
167
+ 157 40.76433121
168
+ 158 41.13924051
169
+ 159 40.88050314
170
+ 160 41.25000000
171
+ 161 40.99378882
172
+ 162 40.74074074
173
+ 163 40.49079755
174
+ 164 40.24390244
175
+ 165 40.60606061
176
+ 166 40.36144578
177
+ 167 40.71856287
178
+ 168 40.47619048
179
+ 169 40.82840237
180
+ 170 40.58823529
181
+ 171 40.35087719
182
+ 172 40.11627907
183
+ 173 39.88439306
184
+ 174 40.22988506
185
+ 175 40.00000000
186
+ 176 39.77272727
187
+ 177 39.54802260
188
+ 178 39.32584270
189
+ 179 39.66480447
190
+ 180 40.00000000
191
+ 181 39.77900552
192
+ 182 39.56043956
193
+ 183 39.89071038
194
+ 184 39.67391304
195
+ 185 40.00000000
196
+ 186 40.32258065
197
+ 187 40.10695187
198
+ 188 40.42553191
199
+ 189 40.74074074
200
+ 190 40.52631579
201
+ 191 40.83769634
202
+ 192 40.62500000
203
+ 193 40.41450777
204
+ 194 40.72164948
205
+ 195 41.02564103
206
+ 196 40.81632653
207
+ 197 41.11675127
208
+ 198 40.90909091
209
+ 199 40.70351759
210
+ 200 40.50000000
211
+ 201 40.29850746
212
+ 202 40.09900990
213
+ 203 39.90147783
214
+ 204 39.70588235
215
+ 205 40.00000000
216
+ 206 39.80582524
217
+ 207 40.09661836
218
+ 208 39.90384615
219
+ 209 39.71291866
220
+ 210 39.52380952
221
+ 211 39.81042654
222
+ 212 39.62264151
223
+ 213 39.43661972
224
+ 214 39.25233645
225
+ 215 39.53488372
226
+ 216 39.81481481
227
+ 217 39.63133641
228
+ 218 39.90825688
229
+ 219 40.18264840
230
+ 220 40.45454545
231
+ 221 40.27149321
232
+ 222 40.54054054
233
+ 223 40.80717489
234
+ 224 41.07142857
235
+ 225 41.33333333
236
+ 226 41.59292035
237
+ 227 41.40969163
238
+ 228 41.66666667
239
+ 229 41.48471616
240
+ 230 41.30434783
241
+ 231 41.12554113
242
+ 232 40.94827586
243
+ 233 40.77253219
244
+ 234 41.02564103
245
+ 235 40.85106383
246
+ 236 40.67796610
247
+ 237 40.50632911
248
+ 238 40.33613445
249
+ 239 40.16736402
250
+ 240 40.00000000
251
+ 241 40.24896266
252
+ 242 40.49586777
253
+ 243 40.74074074
254
+ 244 40.57377049
255
+ 245 40.81632653
256
+ 246 40.65040650
257
+ 247 40.89068826
258
+ 248 41.12903226
259
+ 249 40.96385542
260
+ 250 40.80000000
261
+ 251 40.63745020
262
+ 252 40.47619048
263
+ 253 40.31620553
264
+ 254 40.15748031
265
+ 255 40.00000000
266
+ 256 39.84375000
267
+ 257 40.07782101
268
+ 258 39.92248062
269
+ 259 39.76833977
270
+ 260 39.61538462
271
+ 261 39.46360153
272
+ 262 39.31297710
273
+ 263 39.16349810
274
+ 264 39.39393939
275
+ 265 39.24528302
276
+ 266 39.47368421
277
+ 267 39.32584270
278
+ 268 39.17910448
279
+ 269 39.40520446
280
+ 270 39.62962963
281
+ 271 39.48339483
282
+ 272 39.33823529
283
+ 273 39.19413919
284
+ 274 39.05109489
285
+ 275 38.90909091
286
+ 276 39.13043478
287
+ 277 39.35018051
288
+ 278 39.56834532
289
+ 279 39.42652330
290
+ 280 39.64285714
291
+ 281 39.50177936
292
+ 282 39.36170213
293
+ 283 39.22261484
294
+ 284 39.08450704
295
+ 285 38.94736842
296
+ 286 38.81118881
297
+ 287 39.02439024
298
+ 288 38.88888889
299
+ 289 38.75432526
300
+ 290 38.96551724
301
+ 291 39.17525773
302
+ 292 39.38356164
303
+ 293 39.59044369
304
+ 294 39.45578231
305
+ 295 39.32203390
306
+ 296 39.18918919
307
+ 297 39.05723906
308
+ 298 39.26174497
309
+ 299 39.46488294
310
+ 300 39.66666667
311
+ 301 39.53488372
312
+ 302 39.40397351
313
+ 303 39.27392739
314
+ 304 39.14473684
315
+ 305 39.34426230
316
+ 306 39.21568627
317
+ 307 39.41368078
318
+ 308 39.28571429
319
+ 309 39.15857605
320
+ 310 39.03225806
321
+ 311 39.22829582
322
+ 312 39.42307692
323
+ 313 39.61661342
324
+ 314 39.80891720
325
+ 315 39.68253968
326
+ 316 39.55696203
327
+ 317 39.74763407
328
+ 318 39.62264151
329
+ 319 39.81191223
330
+ 320 39.68750000
331
+ 321 39.56386293
332
+ 322 39.44099379
333
+ 323 39.31888545
334
+ 324 39.19753086
335
+ 325 39.38461538
336
+ 326 39.26380368
337
+ 327 39.44954128
338
+ 328 39.63414634
339
+ 329 39.81762918
340
+ 330 39.69696970
341
+ 331 39.57703927
342
+ 332 39.75903614
343
+ 333 39.63963964
344
+ 334 39.52095808
345
+ 335 39.70149254
346
+ 336 39.88095238
347
+ 337 39.76261128
348
+ 338 39.64497041
349
+ 339 39.52802360
350
+ 340 39.70588235
351
+ 341 39.88269795
352
+ 342 40.05847953
353
+ 343 39.94169096
354
+ 344 39.82558140
355
+ 345 39.71014493
356
+ 346 39.88439306
357
+ 347 40.05763689
358
+ 348 39.94252874
359
+ 349 40.11461318
360
+ 350 40.28571429
361
+ 351 40.17094017
362
+ 352 40.05681818
363
+ 353 40.22662890
364
+ 354 40.39548023
365
+ 355 40.28169014
366
+ 356 40.44943820
367
+ 357 40.33613445
368
+ 358 40.22346369
369
+ 359 40.11142061
370
+ 360 40.27777778
371
+ 361 40.16620499
372
+ 362 40.33149171
373
+ 363 40.49586777
374
+ 364 40.38461538
375
+ 365 40.27397260
376
+ 366 40.43715847
377
+ 367 40.32697548
378
+ 368 40.21739130
379
+ 369 40.10840108
380
+ 370 40.27027027
381
+ 371 40.43126685
382
+ 372 40.32258065
383
+ 373 40.21447721
384
+ 374 40.37433155
385
+ 375 40.53333333
386
+ 376 40.42553191
387
+ 377 40.31830239
388
+ 378 40.47619048
389
+ 379 40.36939314
390
+ 380 40.26315789
391
+ 381 40.41994751
392
+ 382 40.57591623
393
+ 383 40.73107050
394
+ 384 40.62500000
395
+ 385 40.77922078
396
+ 386 40.67357513
397
+ 387 40.56847545
398
+ 388 40.46391753
399
+ 389 40.61696658
400
+ 390 40.76923077
401
+ 391 40.92071611
402
+ 392 41.07142857
403
+ 393 40.96692112
404
+ 394 40.86294416
405
+ 395 41.01265823
406
+ 396 41.16161616
407
+ 397 41.30982368
408
+ 398 41.20603015
409
+ 399 41.35338346
410
+ 400 41.25000000
411
+ 401 41.39650873
412
+ 402 41.29353234
413
+ 403 41.19106700
414
+ 404 41.08910891
415
+ 405 40.98765432
416
+ 406 40.88669951
417
+ 407 40.78624079
418
+ 408 40.68627451
419
+ 409 40.83129584
420
+ 410 40.73170732
421
+ 411 40.63260341
422
+ 412 40.53398058
423
+ 413 40.43583535
424
+ 414 40.33816425
425
+ 415 40.48192771
426
+ 416 40.38461538
427
+ 417 40.28776978
428
+ 418 40.19138756
429
+ 419 40.33412888
430
+ 420 40.47619048
431
+ 421 40.38004751
432
+ 422 40.28436019
433
+ 423 40.18912530
434
+ 424 40.33018868
435
+ 425 40.47058824
436
+ 426 40.61032864
437
+ 427 40.51522248
438
+ 428 40.42056075
439
+ 429 40.55944056
440
+ 430 40.46511628
441
+ 431 40.60324826
442
+ 432 40.74074074
443
+ 433 40.64665127
444
+ 434 40.55299539
445
+ 435 40.45977011
446
+ 436 40.36697248
447
+ 437 40.27459954
448
+ 438 40.41095890
449
+ 439 40.54669704
450
+ 440 40.45454545
451
+ 441 40.36281179
452
+ 442 40.27149321
453
+ 443 40.18058691
454
+ 444 40.09009009
455
+ 445 40.00000000
456
+ 446 39.91031390
457
+ 447 40.04474273
458
+ 448 39.95535714
459
+ 449 39.86636971
460
+ 450 40.00000000
461
+ 451 40.13303769
462
+ 452 40.26548673
463
+ 453 40.17660044
464
+ 454 40.08810573
465
+ 455 40.00000000
466
+ 456 40.13157895
467
+ 457 40.26258206
468
+ 458 40.17467249
469
+ 459 40.08714597
470
+ 460 40.21739130
471
+ 461 40.13015184
472
+ 462 40.04329004
473
+ 463 39.95680346
474
+ 464 40.08620690
475
+ 465 40.00000000
476
+ 466 39.91416309
477
+ 467 40.04282655
478
+ 468 39.95726496
479
+ 469 40.08528785
480
+ 470 40.00000000
481
+ 471 39.91507431
482
+ 472 39.83050847
483
+ 473 39.74630021
484
+ 474 39.66244726
485
+ 475 39.57894737
486
+ 476 39.70588235
487
+ 477 39.62264151
488
+ 478 39.74895397
489
+ 479 39.87473904
490
+ 480 40.00000000
491
+ 481 39.91683992
492
+ 482 39.83402490
493
+ 483 39.75155280
494
+ 484 39.66942149
495
+ 485 39.58762887
496
+ 486 39.71193416
497
+ 487 39.83572895
498
+ 488 39.75409836
499
+ 489 39.67280164
500
+ 490 39.79591837
501
+ 491 39.71486762
502
+ 492 39.63414634
503
+ 493 39.55375254
504
+ 494 39.67611336
505
+ 495 39.79797980
506
+ 496 39.91935484
507
+ 497 39.83903421
508
+ 498 39.75903614
509
+ 499 39.67935872
510
+ 500 39.60000000
511
+ 501 39.52095808
512
+ 502 39.44223108
513
+ 503 39.36381710
514
+ 504 39.48412698
515
+ 505 39.40594059
516
+ 506 39.52569170
517
+ 507 39.44773176
518
+ 508 39.56692913
519
+ 509 39.48919450
520
+ 510 39.41176471
521
+ 511 39.33463796
522
+ 512 39.25781250
523
+ 513 39.18128655
524
+ 514 39.10505837
525
+ 515 39.02912621
526
+ 516 39.14728682
527
+ 517 39.26499033
528
+ 518 39.38223938
529
+ 519 39.30635838
530
+ 520 39.42307692
531
+ 521 39.34740883
532
+ 522 39.46360153
533
+ 523 39.38814532
534
+ 524 39.50381679
535
+ 525 39.42857143
536
+ 526 39.35361217
537
+ 527 39.46869070
538
+ 528 39.39393939
539
+ 529 39.31947070
540
+ 530 39.43396226
541
+ 531 39.35969868
542
+ 532 39.28571429
543
+ 533 39.21200750
544
+ 534 39.13857678
545
+ 535 39.25233645
546
+ 536 39.17910448
547
+ 537 39.29236499
548
+ 538 39.40520446
549
+ 539 39.33209647
550
+ 540 39.25925926
551
+ 541 39.18669131
552
+ 542 39.29889299
553
+ 543 39.22651934
554
+ 544 39.15441176
555
+ 545 39.08256881
556
+ 546 39.01098901
557
+ 547 39.12248629
558
+ 548 39.23357664
559
+ 549 39.16211293
560
+ 550 39.09090909
561
+ 551 39.20145191
562
+ 552 39.13043478
563
+ 553 39.05967450
564
+ 554 39.16967509
565
+ 555 39.09909910
566
+ 556 39.20863309
567
+ 557 39.13824057
568
+ 558 39.24731183
569
+ 559 39.17710197
570
+ 560 39.10714286
571
+ 561 39.03743316
572
+ 562 39.14590747
573
+ 563 39.25399645
574
+ 564 39.18439716
575
+ 565 39.29203540
576
+ 566 39.39929329
577
+ 567 39.32980600
578
+ 568 39.26056338
579
+ 569 39.19156415
580
+ 570 39.29824561
581
+ 571 39.22942207
582
+ 572 39.33566434
583
+ 573 39.26701571
584
+ 574 39.37282230
585
+ 575 39.30434783
586
+ 576 39.40972222
587
+ 577 39.51473137
588
+ 578 39.44636678
589
+ 579 39.37823834
590
+ 580 39.31034483
591
+ 581 39.24268503
592
+ 582 39.17525773
593
+ 583 39.27958834
594
+ 584 39.38356164
595
+ 585 39.31623932
596
+ 586 39.24914676
597
+ 587 39.18228279
598
+ 588 39.11564626
599
+ 589 39.04923599
600
+ 590 38.98305085
601
+ 591 38.91708968
602
+ 592 38.85135135
603
+ 593 38.95446880
604
+ 594 39.05723906
605
+ 595 38.99159664
606
+ 596 39.09395973
607
+ 597 39.02847571
608
+ 598 38.96321070
609
+ 599 39.06510851
610
+ 600 39.00000000
611
+ 601 39.10149750
612
+ 602 39.03654485
613
+ 603 39.13764511
614
+ 604 39.07284768
615
+ 605 39.00826446
616
+ 606 38.94389439
617
+ 607 39.04448105
618
+ 608 38.98026316
619
+ 609 38.91625616
620
+ 610 38.85245902
621
+ 611 38.95253682
622
+ 612 38.88888889
623
+ 613 38.82544861
624
+ 614 38.76221498
625
+ 615 38.69918699
626
+ 616 38.79870130
627
+ 617 38.73581848
628
+ 618 38.67313916
629
+ 619 38.61066236
630
+ 620 38.54838710
631
+ 621 38.48631240
632
+ 622 38.42443730
633
+ 623 38.36276083
634
+ 624 38.30128205
635
+ 625 38.24000000
636
+ 626 38.33865815
637
+ 627 38.27751196
638
+ 628 38.21656051
639
+ 629 38.31478537
640
+ 630 38.41269841
641
+ 631 38.51030111
642
+ 632 38.60759494
643
+ 633 38.54660348
644
+ 634 38.48580442
645
+ 635 38.42519685
646
+ 636 38.36477987
647
+ 637 38.46153846
648
+ 638 38.40125392
649
+ 639 38.49765258
650
+ 640 38.59375000
651
+ 641 38.53354134
652
+ 642 38.62928349
653
+ 643 38.56920684
654
+ 644 38.66459627
655
+ 645 38.60465116
656
+ 646 38.54489164
657
+ 647 38.48531685
658
+ 648 38.42592593
659
+ 649 38.36671803
660
+ 650 38.30769231
661
+ 651 38.24884793
662
+ 652 38.19018405
663
+ 653 38.13169985
664
+ 654 38.22629969
665
+ 655 38.16793893
666
+ 656 38.10975610
667
+ 657 38.05175038
668
+ 658 38.14589666
669
+ 659 38.08801214
670
+ 660 38.03030303
671
+ 661 37.97276853
672
+ 662 37.91540785
673
+ 663 37.85822021
674
+ 664 37.95180723
675
+ 665 38.04511278
676
+ 666 37.98798799
677
+ 667 38.08095952
678
+ 668 38.17365269
679
+ 669 38.26606876
680
+ 670 38.20895522
681
+ 671 38.15201192
682
+ 672 38.09523810
683
+ 673 38.03863299
684
+ 674 37.98219585
685
+ 675 37.92592593
686
+ 676 37.86982249
687
+ 677 37.81388479
688
+ 678 37.75811209
689
+ 679 37.70250368
690
+ 680 37.79411765
691
+ 681 37.88546256
692
+ 682 37.82991202
693
+ 683 37.77452416
694
+ 684 37.71929825
695
+ 685 37.66423358
696
+ 686 37.60932945
697
+ 687 37.55458515
698
+ 688 37.50000000
699
+ 689 37.59071118
700
+ 690 37.53623188
701
+ 691 37.62662808
702
+ 692 37.57225434
703
+ 693 37.51803752
704
+ 694 37.46397695
705
+ 695 37.41007194
706
+ 696 37.35632184
707
+ 697 37.44619799
708
+ 698 37.39255014
709
+ 699 37.48211731
710
+ 700 37.42857143
711
+ 701 37.51783167
712
+ 702 37.60683761
713
+ 703 37.69559033
714
+ 704 37.64204545
715
+ 705 37.58865248
716
+ 706 37.53541076
717
+ 707 37.48231966
718
+ 708 37.42937853
719
+ 709 37.51763047
720
+ 710 37.60563380
721
+ 711 37.69338959
722
+ 712 37.64044944
723
+ 713 37.72791024
724
+ 714 37.81512605
725
+ 715 37.76223776
726
+ 716 37.84916201
727
+ 717 37.79637378
728
+ 718 37.74373259
729
+ 719 37.83031989
730
+ 720 37.91666667
731
+ 721 37.86407767
732
+ 722 37.95013850
733
+ 723 38.03596127
734
+ 724 37.98342541
735
+ 725 38.06896552
736
+ 726 38.01652893
737
+ 727 37.96423659
738
+ 728 37.91208791
739
+ 729 37.86008230
740
+ 730 37.80821918
741
+ 731 37.75649795
742
+ 732 37.84153005
743
+ 733 37.92633015
744
+ 734 38.01089918
745
+ 735 37.95918367
746
+ 736 37.90760870
747
+ 737 37.99185889
748
+ 738 37.94037940
749
+ 739 38.02435724
750
+ 740 37.97297297
751
+ 741 37.92172740
752
+ 742 38.00539084
753
+ 743 38.08882907
754
+ 744 38.17204301
755
+ 745 38.12080537
756
+ 746 38.06970509
757
+ 747 38.01874163
758
+ 748 37.96791444
759
+ 749 37.91722296
760
+ 750 38.00000000
761
+
762
+ Final result: 38.0000 +/- 1.7736
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-iq3_s.ppx ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ====== Perplexity statistics ======
2
+ Mean PPL(Q) : 10.536311 ± 0.075790
3
+ Mean PPL(base) : 9.366577 ± 0.066397
4
+ Cor(ln(PPL(Q)), ln(PPL(base))): 97.80%
5
+ Mean ln(PPL(Q)/PPL(base)) : 0.117680 ± 0.001502
6
+ Mean PPL(Q)/PPL(base) : 1.124884 ± 0.001690
7
+ Mean PPL(Q)-PPL(base) : 1.169734 ± 0.017604
8
+
9
+ ====== KL divergence statistics ======
10
+ Mean KLD: 0.139837 ± 0.000416
11
+ Maximum KLD: 6.695117
12
+ 99.9% KLD: 1.471549
13
+ 99.0% KLD: 0.739369
14
+ 99.0% KLD: 0.739369
15
+ Median KLD: 0.101636
16
+ 10.0% KLD: 0.002644
17
+ 5.0% KLD: 0.000563
18
+ 1.0% KLD: 0.000046
19
+ Minimum KLD: -0.000007
20
+
21
+ ====== Token probability statistics ======
22
+ Mean Δp: -2.184 ± 0.024 %
23
+ Maximum Δp: 91.689%
24
+ 99.9% Δp: 39.852%
25
+ 99.0% Δp: 22.355%
26
+ 95.0% Δp: 10.127%
27
+ 90.0% Δp: 5.126%
28
+ 75.0% Δp: 0.365%
29
+ Median Δp: -0.198%
30
+ 25.0% Δp: -4.009%
31
+ 10.0% Δp: -12.600%
32
+ 5.0% Δp: -19.219%
33
+ 1.0% Δp: -35.248%
34
+ 0.1% Δp: -59.692%
35
+ Minimum Δp: -95.077%
36
+ RMS Δp : 9.634 ± 0.040 %
37
+ Same top p: 80.455 ± 0.103 %
scores/Hammer2.1-7b-iq3_s.tqa ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 817 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 817 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 14.28571429
18
+ 8 12.50000000
19
+ 9 11.11111111
20
+ 10 10.00000000
21
+ 11 9.09090909
22
+ 12 16.66666667
23
+ 13 23.07692308
24
+ 14 21.42857143
25
+ 15 20.00000000
26
+ 16 18.75000000
27
+ 17 17.64705882
28
+ 18 16.66666667
29
+ 19 21.05263158
30
+ 20 20.00000000
31
+ 21 19.04761905
32
+ 22 18.18181818
33
+ 23 17.39130435
34
+ 24 20.83333333
35
+ 25 20.00000000
36
+ 26 19.23076923
37
+ 27 18.51851852
38
+ 28 17.85714286
39
+ 29 20.68965517
40
+ 30 20.00000000
41
+ 31 19.35483871
42
+ 32 21.87500000
43
+ 33 24.24242424
44
+ 34 23.52941176
45
+ 35 22.85714286
46
+ 36 22.22222222
47
+ 37 24.32432432
48
+ 38 23.68421053
49
+ 39 23.07692308
50
+ 40 22.50000000
51
+ 41 21.95121951
52
+ 42 21.42857143
53
+ 43 23.25581395
54
+ 44 22.72727273
55
+ 45 24.44444444
56
+ 46 26.08695652
57
+ 47 25.53191489
58
+ 48 27.08333333
59
+ 49 26.53061224
60
+ 50 28.00000000
61
+ 51 29.41176471
62
+ 52 28.84615385
63
+ 53 28.30188679
64
+ 54 29.62962963
65
+ 55 30.90909091
66
+ 56 30.35714286
67
+ 57 29.82456140
68
+ 58 29.31034483
69
+ 59 28.81355932
70
+ 60 28.33333333
71
+ 61 27.86885246
72
+ 62 29.03225806
73
+ 63 28.57142857
74
+ 64 29.68750000
75
+ 65 29.23076923
76
+ 66 28.78787879
77
+ 67 28.35820896
78
+ 68 27.94117647
79
+ 69 28.98550725
80
+ 70 30.00000000
81
+ 71 29.57746479
82
+ 72 29.16666667
83
+ 73 28.76712329
84
+ 74 28.37837838
85
+ 75 28.00000000
86
+ 76 27.63157895
87
+ 77 27.27272727
88
+ 78 26.92307692
89
+ 79 26.58227848
90
+ 80 26.25000000
91
+ 81 25.92592593
92
+ 82 26.82926829
93
+ 83 26.50602410
94
+ 84 26.19047619
95
+ 85 25.88235294
96
+ 86 25.58139535
97
+ 87 25.28735632
98
+ 88 25.00000000
99
+ 89 25.84269663
100
+ 90 25.55555556
101
+ 91 26.37362637
102
+ 92 26.08695652
103
+ 93 25.80645161
104
+ 94 25.53191489
105
+ 95 26.31578947
106
+ 96 27.08333333
107
+ 97 27.83505155
108
+ 98 27.55102041
109
+ 99 28.28282828
110
+ 100 29.00000000
111
+ 101 29.70297030
112
+ 102 29.41176471
113
+ 103 30.09708738
114
+ 104 29.80769231
115
+ 105 29.52380952
116
+ 106 29.24528302
117
+ 107 29.90654206
118
+ 108 30.55555556
119
+ 109 30.27522936
120
+ 110 30.90909091
121
+ 111 30.63063063
122
+ 112 30.35714286
123
+ 113 30.08849558
124
+ 114 30.70175439
125
+ 115 30.43478261
126
+ 116 30.17241379
127
+ 117 29.91452991
128
+ 118 29.66101695
129
+ 119 29.41176471
130
+ 120 30.00000000
131
+ 121 30.57851240
132
+ 122 30.32786885
133
+ 123 30.89430894
134
+ 124 31.45161290
135
+ 125 31.20000000
136
+ 126 30.95238095
137
+ 127 30.70866142
138
+ 128 30.46875000
139
+ 129 30.23255814
140
+ 130 30.00000000
141
+ 131 29.77099237
142
+ 132 29.54545455
143
+ 133 29.32330827
144
+ 134 29.85074627
145
+ 135 30.37037037
146
+ 136 30.14705882
147
+ 137 30.65693431
148
+ 138 31.15942029
149
+ 139 31.65467626
150
+ 140 31.42857143
151
+ 141 31.20567376
152
+ 142 30.98591549
153
+ 143 30.76923077
154
+ 144 31.25000000
155
+ 145 31.03448276
156
+ 146 30.82191781
157
+ 147 30.61224490
158
+ 148 30.40540541
159
+ 149 30.20134228
160
+ 150 30.00000000
161
+ 151 30.46357616
162
+ 152 30.26315789
163
+ 153 30.06535948
164
+ 154 29.87012987
165
+ 155 30.32258065
166
+ 156 30.76923077
167
+ 157 30.57324841
168
+ 158 31.01265823
169
+ 159 30.81761006
170
+ 160 30.62500000
171
+ 161 30.43478261
172
+ 162 30.24691358
173
+ 163 30.06134969
174
+ 164 29.87804878
175
+ 165 30.30303030
176
+ 166 30.72289157
177
+ 167 30.53892216
178
+ 168 30.35714286
179
+ 169 30.17751479
180
+ 170 30.00000000
181
+ 171 29.82456140
182
+ 172 29.65116279
183
+ 173 29.47976879
184
+ 174 29.88505747
185
+ 175 29.71428571
186
+ 176 29.54545455
187
+ 177 29.37853107
188
+ 178 29.21348315
189
+ 179 29.05027933
190
+ 180 28.88888889
191
+ 181 28.72928177
192
+ 182 28.57142857
193
+ 183 28.41530055
194
+ 184 28.26086957
195
+ 185 28.10810811
196
+ 186 28.49462366
197
+ 187 28.34224599
198
+ 188 28.72340426
199
+ 189 28.57142857
200
+ 190 28.42105263
201
+ 191 28.27225131
202
+ 192 28.64583333
203
+ 193 28.49740933
204
+ 194 28.86597938
205
+ 195 28.71794872
206
+ 196 28.57142857
207
+ 197 28.93401015
208
+ 198 29.29292929
209
+ 199 29.14572864
210
+ 200 29.00000000
211
+ 201 28.85572139
212
+ 202 29.20792079
213
+ 203 29.06403941
214
+ 204 28.92156863
215
+ 205 28.78048780
216
+ 206 28.64077670
217
+ 207 28.98550725
218
+ 208 28.84615385
219
+ 209 28.70813397
220
+ 210 28.57142857
221
+ 211 28.43601896
222
+ 212 28.30188679
223
+ 213 28.16901408
224
+ 214 28.03738318
225
+ 215 27.90697674
226
+ 216 28.24074074
227
+ 217 28.57142857
228
+ 218 28.89908257
229
+ 219 28.76712329
230
+ 220 28.63636364
231
+ 221 28.50678733
232
+ 222 28.37837838
233
+ 223 28.25112108
234
+ 224 28.57142857
235
+ 225 28.44444444
236
+ 226 28.31858407
237
+ 227 28.63436123
238
+ 228 28.50877193
239
+ 229 28.38427948
240
+ 230 28.26086957
241
+ 231 28.13852814
242
+ 232 28.44827586
243
+ 233 28.75536481
244
+ 234 28.63247863
245
+ 235 28.51063830
246
+ 236 28.81355932
247
+ 237 28.69198312
248
+ 238 28.99159664
249
+ 239 28.87029289
250
+ 240 28.75000000
251
+ 241 28.63070539
252
+ 242 28.51239669
253
+ 243 28.39506173
254
+ 244 28.27868852
255
+ 245 28.16326531
256
+ 246 28.04878049
257
+ 247 27.93522267
258
+ 248 27.82258065
259
+ 249 27.71084337
260
+ 250 27.60000000
261
+ 251 27.88844622
262
+ 252 27.77777778
263
+ 253 27.66798419
264
+ 254 27.55905512
265
+ 255 27.84313725
266
+ 256 28.12500000
267
+ 257 28.01556420
268
+ 258 27.90697674
269
+ 259 27.79922780
270
+ 260 27.69230769
271
+ 261 27.96934866
272
+ 262 27.86259542
273
+ 263 27.75665399
274
+ 264 28.03030303
275
+ 265 28.30188679
276
+ 266 28.19548872
277
+ 267 28.46441948
278
+ 268 28.35820896
279
+ 269 28.25278810
280
+ 270 28.14814815
281
+ 271 28.04428044
282
+ 272 27.94117647
283
+ 273 27.83882784
284
+ 274 28.10218978
285
+ 275 28.00000000
286
+ 276 27.89855072
287
+ 277 27.79783394
288
+ 278 27.69784173
289
+ 279 27.59856631
290
+ 280 27.85714286
291
+ 281 28.11387900
292
+ 282 28.01418440
293
+ 283 27.91519435
294
+ 284 28.16901408
295
+ 285 28.07017544
296
+ 286 28.32167832
297
+ 287 28.57142857
298
+ 288 28.47222222
299
+ 289 28.37370242
300
+ 290 28.27586207
301
+ 291 28.52233677
302
+ 292 28.42465753
303
+ 293 28.66894198
304
+ 294 28.57142857
305
+ 295 28.81355932
306
+ 296 29.05405405
307
+ 297 29.29292929
308
+ 298 29.19463087
309
+ 299 29.09698997
310
+ 300 29.00000000
311
+ 301 28.90365449
312
+ 302 28.80794702
313
+ 303 28.71287129
314
+ 304 28.61842105
315
+ 305 28.52459016
316
+ 306 28.43137255
317
+ 307 28.33876221
318
+ 308 28.57142857
319
+ 309 28.47896440
320
+ 310 28.38709677
321
+ 311 28.29581994
322
+ 312 28.52564103
323
+ 313 28.43450479
324
+ 314 28.66242038
325
+ 315 28.57142857
326
+ 316 28.48101266
327
+ 317 28.39116719
328
+ 318 28.30188679
329
+ 319 28.21316614
330
+ 320 28.12500000
331
+ 321 28.03738318
332
+ 322 28.26086957
333
+ 323 28.17337461
334
+ 324 28.08641975
335
+ 325 28.00000000
336
+ 326 27.91411043
337
+ 327 28.13455657
338
+ 328 28.35365854
339
+ 329 28.26747720
340
+ 330 28.18181818
341
+ 331 28.09667674
342
+ 332 28.31325301
343
+ 333 28.22822823
344
+ 334 28.14371257
345
+ 335 28.35820896
346
+ 336 28.57142857
347
+ 337 28.48664688
348
+ 338 28.40236686
349
+ 339 28.61356932
350
+ 340 28.52941176
351
+ 341 28.73900293
352
+ 342 28.94736842
353
+ 343 28.86297376
354
+ 344 29.06976744
355
+ 345 29.27536232
356
+ 346 29.47976879
357
+ 347 29.68299712
358
+ 348 29.59770115
359
+ 349 29.51289398
360
+ 350 29.42857143
361
+ 351 29.34472934
362
+ 352 29.26136364
363
+ 353 29.17847025
364
+ 354 29.09604520
365
+ 355 29.29577465
366
+ 356 29.21348315
367
+ 357 29.41176471
368
+ 358 29.60893855
369
+ 359 29.52646240
370
+ 360 29.44444444
371
+ 361 29.63988920
372
+ 362 29.55801105
373
+ 363 29.75206612
374
+ 364 29.67032967
375
+ 365 29.86301370
376
+ 366 29.78142077
377
+ 367 29.97275204
378
+ 368 29.89130435
379
+ 369 29.81029810
380
+ 370 29.72972973
381
+ 371 29.64959569
382
+ 372 29.83870968
383
+ 373 29.75871314
384
+ 374 29.94652406
385
+ 375 29.86666667
386
+ 376 29.78723404
387
+ 377 29.97347480
388
+ 378 29.89417989
389
+ 379 29.81530343
390
+ 380 29.73684211
391
+ 381 29.65879265
392
+ 382 29.84293194
393
+ 383 29.76501305
394
+ 384 29.68750000
395
+ 385 29.61038961
396
+ 386 29.53367876
397
+ 387 29.45736434
398
+ 388 29.63917526
399
+ 389 29.56298201
400
+ 390 29.48717949
401
+ 391 29.41176471
402
+ 392 29.33673469
403
+ 393 29.26208651
404
+ 394 29.44162437
405
+ 395 29.36708861
406
+ 396 29.29292929
407
+ 397 29.47103275
408
+ 398 29.39698492
409
+ 399 29.32330827
410
+ 400 29.25000000
411
+ 401 29.17705736
412
+ 402 29.35323383
413
+ 403 29.52853598
414
+ 404 29.45544554
415
+ 405 29.38271605
416
+ 406 29.31034483
417
+ 407 29.23832924
418
+ 408 29.16666667
419
+ 409 29.09535452
420
+ 410 29.02439024
421
+ 411 28.95377129
422
+ 412 29.12621359
423
+ 413 29.05569007
424
+ 414 29.22705314
425
+ 415 29.39759036
426
+ 416 29.32692308
427
+ 417 29.25659472
428
+ 418 29.18660287
429
+ 419 29.11694511
430
+ 420 29.28571429
431
+ 421 29.45368171
432
+ 422 29.38388626
433
+ 423 29.31442080
434
+ 424 29.24528302
435
+ 425 29.17647059
436
+ 426 29.10798122
437
+ 427 29.03981265
438
+ 428 28.97196262
439
+ 429 29.13752914
440
+ 430 29.06976744
441
+ 431 29.00232019
442
+ 432 29.16666667
443
+ 433 29.33025404
444
+ 434 29.49308756
445
+ 435 29.42528736
446
+ 436 29.35779817
447
+ 437 29.29061785
448
+ 438 29.22374429
449
+ 439 29.15717540
450
+ 440 29.09090909
451
+ 441 29.02494331
452
+ 442 28.95927602
453
+ 443 28.89390519
454
+ 444 29.05405405
455
+ 445 28.98876404
456
+ 446 28.92376682
457
+ 447 28.85906040
458
+ 448 28.79464286
459
+ 449 28.73051225
460
+ 450 28.66666667
461
+ 451 28.60310421
462
+ 452 28.53982301
463
+ 453 28.47682119
464
+ 454 28.63436123
465
+ 455 28.57142857
466
+ 456 28.50877193
467
+ 457 28.44638950
468
+ 458 28.38427948
469
+ 459 28.32244009
470
+ 460 28.26086957
471
+ 461 28.19956616
472
+ 462 28.13852814
473
+ 463 28.07775378
474
+ 464 28.23275862
475
+ 465 28.38709677
476
+ 466 28.32618026
477
+ 467 28.26552463
478
+ 468 28.20512821
479
+ 469 28.35820896
480
+ 470 28.29787234
481
+ 471 28.45010616
482
+ 472 28.38983051
483
+ 473 28.32980973
484
+ 474 28.27004219
485
+ 475 28.42105263
486
+ 476 28.36134454
487
+ 477 28.51153040
488
+ 478 28.45188285
489
+ 479 28.39248434
490
+ 480 28.33333333
491
+ 481 28.27442827
492
+ 482 28.42323651
493
+ 483 28.36438923
494
+ 484 28.51239669
495
+ 485 28.65979381
496
+ 486 28.60082305
497
+ 487 28.54209446
498
+ 488 28.48360656
499
+ 489 28.62985685
500
+ 490 28.77551020
501
+ 491 28.92057026
502
+ 492 29.06504065
503
+ 493 29.00608519
504
+ 494 28.94736842
505
+ 495 29.09090909
506
+ 496 29.03225806
507
+ 497 28.97384306
508
+ 498 28.91566265
509
+ 499 29.05811623
510
+ 500 29.00000000
511
+ 501 29.14171657
512
+ 502 29.08366534
513
+ 503 29.02584493
514
+ 504 29.16666667
515
+ 505 29.10891089
516
+ 506 29.05138340
517
+ 507 29.19132150
518
+ 508 29.13385827
519
+ 509 29.27308448
520
+ 510 29.41176471
521
+ 511 29.35420744
522
+ 512 29.29687500
523
+ 513 29.23976608
524
+ 514 29.18287938
525
+ 515 29.32038835
526
+ 516 29.26356589
527
+ 517 29.40038685
528
+ 518 29.34362934
529
+ 519 29.28709056
530
+ 520 29.23076923
531
+ 521 29.17466411
532
+ 522 29.11877395
533
+ 523 29.06309751
534
+ 524 29.00763359
535
+ 525 28.95238095
536
+ 526 28.89733840
537
+ 527 28.84250474
538
+ 528 28.78787879
539
+ 529 28.73345936
540
+ 530 28.67924528
541
+ 531 28.62523540
542
+ 532 28.75939850
543
+ 533 28.89305816
544
+ 534 28.83895131
545
+ 535 28.78504673
546
+ 536 28.73134328
547
+ 537 28.86405959
548
+ 538 28.81040892
549
+ 539 28.75695733
550
+ 540 28.88888889
551
+ 541 28.83548983
552
+ 542 28.78228782
553
+ 543 28.72928177
554
+ 544 28.67647059
555
+ 545 28.80733945
556
+ 546 28.93772894
557
+ 547 29.06764168
558
+ 548 29.19708029
559
+ 549 29.14389800
560
+ 550 29.09090909
561
+ 551 29.03811252
562
+ 552 29.16666667
563
+ 553 29.29475588
564
+ 554 29.24187726
565
+ 555 29.18918919
566
+ 556 29.13669065
567
+ 557 29.08438061
568
+ 558 29.03225806
569
+ 559 28.98032200
570
+ 560 29.10714286
571
+ 561 29.23351159
572
+ 562 29.18149466
573
+ 563 29.12966252
574
+ 564 29.25531915
575
+ 565 29.38053097
576
+ 566 29.50530035
577
+ 567 29.45326279
578
+ 568 29.40140845
579
+ 569 29.52548330
580
+ 570 29.47368421
581
+ 571 29.42206655
582
+ 572 29.37062937
583
+ 573 29.31937173
584
+ 574 29.44250871
585
+ 575 29.56521739
586
+ 576 29.68750000
587
+ 577 29.80935875
588
+ 578 29.93079585
589
+ 579 30.05181347
590
+ 580 30.17241379
591
+ 581 30.12048193
592
+ 582 30.06872852
593
+ 583 30.01715266
594
+ 584 30.13698630
595
+ 585 30.08547009
596
+ 586 30.20477816
597
+ 587 30.15332198
598
+ 588 30.10204082
599
+ 589 30.05093379
600
+ 590 30.16949153
601
+ 591 30.11844332
602
+ 592 30.06756757
603
+ 593 30.01686341
604
+ 594 29.96632997
605
+ 595 29.91596639
606
+ 596 29.86577181
607
+ 597 29.98324958
608
+ 598 29.93311037
609
+ 599 29.88313856
610
+ 600 29.83333333
611
+ 601 29.95008319
612
+ 602 29.90033223
613
+ 603 30.01658375
614
+ 604 29.96688742
615
+ 605 29.91735537
616
+ 606 30.03300330
617
+ 607 29.98352554
618
+ 608 29.93421053
619
+ 609 30.04926108
620
+ 610 30.00000000
621
+ 611 29.95090016
622
+ 612 30.06535948
623
+ 613 30.01631321
624
+ 614 29.96742671
625
+ 615 29.91869919
626
+ 616 30.03246753
627
+ 617 30.14586710
628
+ 618 30.09708738
629
+ 619 30.21001616
630
+ 620 30.16129032
631
+ 621 30.11272142
632
+ 622 30.22508039
633
+ 623 30.17656501
634
+ 624 30.12820513
635
+ 625 30.08000000
636
+ 626 30.03194888
637
+ 627 29.98405104
638
+ 628 29.93630573
639
+ 629 29.88871224
640
+ 630 29.84126984
641
+ 631 29.95245642
642
+ 632 29.90506329
643
+ 633 29.85781991
644
+ 634 29.96845426
645
+ 635 29.92125984
646
+ 636 29.87421384
647
+ 637 29.82731554
648
+ 638 29.78056426
649
+ 639 29.73395931
650
+ 640 29.84375000
651
+ 641 29.79719189
652
+ 642 29.90654206
653
+ 643 29.86003110
654
+ 644 29.81366460
655
+ 645 29.92248062
656
+ 646 29.87616099
657
+ 647 29.82998454
658
+ 648 29.93827160
659
+ 649 29.89214176
660
+ 650 30.00000000
661
+ 651 30.10752688
662
+ 652 30.06134969
663
+ 653 30.16845329
664
+ 654 30.12232416
665
+ 655 30.07633588
666
+ 656 30.18292683
667
+ 657 30.28919330
668
+ 658 30.24316109
669
+ 659 30.19726859
670
+ 660 30.15151515
671
+ 661 30.10590015
672
+ 662 30.06042296
673
+ 663 30.16591252
674
+ 664 30.12048193
675
+ 665 30.22556391
676
+ 666 30.33033033
677
+ 667 30.28485757
678
+ 668 30.23952096
679
+ 669 30.19431988
680
+ 670 30.29850746
681
+ 671 30.25335320
682
+ 672 30.20833333
683
+ 673 30.16344725
684
+ 674 30.11869436
685
+ 675 30.22222222
686
+ 676 30.32544379
687
+ 677 30.28064993
688
+ 678 30.23598820
689
+ 679 30.19145803
690
+ 680 30.14705882
691
+ 681 30.10279001
692
+ 682 30.05865103
693
+ 683 30.01464129
694
+ 684 30.11695906
695
+ 685 30.07299270
696
+ 686 30.02915452
697
+ 687 29.98544396
698
+ 688 29.94186047
699
+ 689 30.04354136
700
+ 690 30.00000000
701
+ 691 30.10130246
702
+ 692 30.20231214
703
+ 693 30.30303030
704
+ 694 30.25936599
705
+ 695 30.21582734
706
+ 696 30.31609195
707
+ 697 30.27259684
708
+ 698 30.22922636
709
+ 699 30.18597997
710
+ 700 30.28571429
711
+ 701 30.24251070
712
+ 702 30.19943020
713
+ 703 30.15647226
714
+ 704 30.11363636
715
+ 705 30.07092199
716
+ 706 30.02832861
717
+ 707 30.12729844
718
+ 708 30.08474576
719
+ 709 30.04231312
720
+ 710 30.00000000
721
+ 711 29.95780591
722
+ 712 29.91573034
723
+ 713 29.87377279
724
+ 714 29.97198880
725
+ 715 30.06993007
726
+ 716 30.02793296
727
+ 717 29.98605300
728
+ 718 29.94428969
729
+ 719 30.04172462
730
+ 720 30.00000000
731
+ 721 29.95839112
732
+ 722 29.91689751
733
+ 723 29.87551867
734
+ 724 29.83425414
735
+ 725 29.79310345
736
+ 726 29.88980716
737
+ 727 29.84869326
738
+ 728 29.80769231
739
+ 729 29.90397805
740
+ 730 29.86301370
741
+ 731 29.82216142
742
+ 732 29.91803279
743
+ 733 29.87721692
744
+ 734 29.83651226
745
+ 735 29.79591837
746
+ 736 29.75543478
747
+ 737 29.85074627
748
+ 738 29.81029810
749
+ 739 29.90527740
750
+ 740 29.86486486
751
+ 741 29.82456140
752
+ 742 29.78436658
753
+ 743 29.74427995
754
+ 744 29.83870968
755
+ 745 29.79865772
756
+ 746 29.89276139
757
+ 747 29.98661312
758
+ 748 29.94652406
759
+ 749 30.04005340
760
+ 750 30.13333333
761
+
762
+ Final result: 30.1333 +/- 1.6766
763
+ Random chance: 19.8992 +/- 1.4588
scores/Hammer2.1-7b-iq3_s.wng ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ winogrande_score : loaded 1267 tasks from prompt.
6
+ winogrande_score : selecting 750 random tasks
7
+ winogrande_score : tokenizing selected tasks
8
+ winogrande_score : calculating winogrande score over selected tasks.
9
+ 1 100.0000 -4.564049 -4.983895 1 1
10
+ 2 100.0000 -2.443548 -2.592391 1 1
11
+ 3 100.0000 -2.695466 -2.233610 2 2
12
+ 4 75.0000 -5.018048 -4.618641 2 1
13
+ 5 60.0000 -2.154083 -3.196243 1 2
14
+ 6 50.0000 -4.914985 -4.778591 2 1
15
+ 7 57.1429 -4.012952 -2.935628 2 2
16
+ 8 50.0000 -2.151992 -2.216527 1 2
17
+ 9 55.5556 -3.747523 -6.191468 1 1
18
+ 10 60.0000 -1.934170 -1.352474 2 2
19
+ 11 63.6364 -11.139496 -5.796354 2 2
20
+ 12 66.6667 -4.189046 -3.731196 2 2
21
+ 13 69.2308 -2.804996 -5.878899 1 1
22
+ 14 64.2857 -4.067920 -3.934711 2 1
23
+ 15 60.0000 -4.309402 -5.829717 1 2
24
+ 16 56.2500 -3.932866 -4.349301 1 2
25
+ 17 52.9412 -5.720638 -6.318021 1 2
26
+ 18 55.5556 -1.141072 -3.039824 1 1
27
+ 19 57.8947 -4.373578 -5.137520 1 1
28
+ 20 60.0000 -3.163618 -4.152479 1 1
29
+ 21 61.9048 -2.394028 -3.093969 1 1
30
+ 22 63.6364 -2.929190 -2.452881 2 2
31
+ 23 65.2174 -5.172765 -5.195069 1 1
32
+ 24 62.5000 -2.698766 -2.445210 2 1
33
+ 25 64.0000 -2.779315 -2.615278 2 2
34
+ 26 65.3846 -4.191188 -4.502489 1 1
35
+ 27 66.6667 -5.315368 -5.163735 2 2
36
+ 28 67.8571 -4.762212 -5.345638 1 1
37
+ 29 68.9655 -4.117605 -4.154055 1 1
38
+ 30 66.6667 -1.507990 -1.858474 1 2
39
+ 31 67.7419 -5.444788 -5.503157 1 1
40
+ 32 68.7500 -4.510567 -6.533535 1 1
41
+ 33 69.6970 -3.067488 -5.793545 1 1
42
+ 34 67.6471 -2.419710 -2.313942 2 1
43
+ 35 65.7143 -4.377050 -4.435561 1 2
44
+ 36 66.6667 -4.927518 -3.863583 2 2
45
+ 37 64.8649 -3.562357 -3.325751 2 1
46
+ 38 65.7895 -3.218704 -3.369295 1 1
47
+ 39 66.6667 -4.384321 -4.260691 2 2
48
+ 40 65.0000 -4.366860 -5.337930 1 2
49
+ 41 65.8537 -2.993298 -3.157077 1 1
50
+ 42 66.6667 -1.801891 -2.495732 1 1
51
+ 43 65.1163 -4.116697 -4.244745 1 2
52
+ 44 63.6364 -1.997564 -2.005370 1 2
53
+ 45 64.4444 -0.103019 -14.064075 1 1
54
+ 46 65.2174 -2.605427 -2.848113 1 1
55
+ 47 63.8298 -4.423427 -4.997763 1 2
56
+ 48 64.5833 -5.142738 -4.594782 2 2
57
+ 49 63.2653 -2.817792 -1.193462 2 1
58
+ 50 62.0000 -2.599653 -2.667218 1 2
59
+ 51 62.7451 -2.631130 -2.967622 1 1
60
+ 52 63.4615 -2.776400 -2.065787 2 2
61
+ 53 64.1509 -4.053586 -4.956534 1 1
62
+ 54 64.8148 -3.637321 -3.471377 2 2
63
+ 55 63.6364 -3.197219 -3.207480 1 2
64
+ 56 62.5000 -5.078475 -5.109688 1 2
65
+ 57 61.4035 -1.860222 -2.886575 1 2
66
+ 58 62.0690 -3.427972 -4.680273 1 1
67
+ 59 62.7119 -3.070632 -2.997353 2 2
68
+ 60 63.3333 -4.494180 -4.215181 2 2
69
+ 61 63.9344 -3.926221 -4.385199 1 1
70
+ 62 62.9032 -7.510101 -7.395861 2 1
71
+ 63 61.9048 -4.235316 -4.978574 1 2
72
+ 64 62.5000 -4.199964 -4.428668 1 1
73
+ 65 63.0769 -5.268451 -4.813458 2 2
74
+ 66 63.6364 -2.645530 -3.680751 1 1
75
+ 67 64.1791 -2.602703 -2.201235 2 2
76
+ 68 64.7059 -5.005672 -4.527631 2 2
77
+ 69 65.2174 -5.220839 -8.503501 1 1
78
+ 70 65.7143 -3.540717 -2.784654 2 2
79
+ 71 64.7887 -3.326887 -2.889338 2 1
80
+ 72 63.8889 -2.846037 -3.297407 1 2
81
+ 73 64.3836 -3.314669 -3.444314 1 1
82
+ 74 63.5135 -2.427323 -2.126325 2 1
83
+ 75 64.0000 -1.414952 -2.253034 1 1
84
+ 76 64.4737 -1.936907 -4.070088 1 1
85
+ 77 63.6364 -3.048207 -3.103973 1 2
86
+ 78 64.1026 -1.366719 -1.524945 1 1
87
+ 79 63.2911 -2.250650 -2.759335 1 2
88
+ 80 63.7500 -2.309438 -3.647755 1 1
89
+ 81 62.9630 -3.642732 -3.720715 1 2
90
+ 82 63.4146 -2.634964 -2.299270 2 2
91
+ 83 63.8554 -6.802913 -6.767538 2 2
92
+ 84 64.2857 -2.306052 -2.264986 2 2
93
+ 85 64.7059 -4.795518 -5.200772 1 1
94
+ 86 63.9535 -5.960296 -6.195642 1 2
95
+ 87 63.2184 -1.588663 -1.420082 2 1
96
+ 88 62.5000 -4.791022 -4.519946 2 1
97
+ 89 62.9213 -2.683556 -3.042119 1 1
98
+ 90 63.3333 -5.704089 -4.204198 2 2
99
+ 91 62.6374 -4.605670 -4.078777 2 1
100
+ 92 63.0435 -5.961967 -4.278804 2 2
101
+ 93 63.4409 -2.562942 -2.193210 2 2
102
+ 94 63.8298 -4.042863 -3.296520 2 2
103
+ 95 64.2105 -5.224189 -5.919940 1 1
104
+ 96 64.5833 -2.243029 -3.481765 1 1
105
+ 97 64.9485 -5.484175 -5.748281 1 1
106
+ 98 65.3061 -5.487869 -3.922006 2 2
107
+ 99 64.6465 -3.084383 -2.480156 2 1
108
+ 100 64.0000 -6.401377 -6.428550 1 2
109
+ 101 63.3663 -4.670393 -4.810647 1 2
110
+ 102 63.7255 -4.450538 -5.066936 1 1
111
+ 103 63.1068 -3.003145 -3.020189 1 2
112
+ 104 63.4615 -8.562346 -8.393780 2 2
113
+ 105 63.8095 -1.287700 -2.400275 1 1
114
+ 106 64.1509 -3.134441 -2.839276 2 2
115
+ 107 64.4860 -3.689794 -2.437676 2 2
116
+ 108 64.8148 -4.014494 -3.284989 2 2
117
+ 109 65.1376 -5.785357 -5.633955 2 2
118
+ 110 65.4545 -4.500303 -3.626762 2 2
119
+ 111 65.7658 -4.082104 -3.729338 2 2
120
+ 112 66.0714 -3.383434 -3.409619 1 1
121
+ 113 65.4867 -4.831551 -4.972342 1 2
122
+ 114 65.7895 -3.885744 -4.727722 1 1
123
+ 115 66.0870 -2.678518 -3.273825 1 1
124
+ 116 66.3793 -3.639898 -2.856767 2 2
125
+ 117 65.8120 -4.677776 -4.759790 1 2
126
+ 118 66.1017 -0.844216 -2.648723 1 1
127
+ 119 66.3866 -2.397732 -2.844238 1 1
128
+ 120 65.8333 -2.445095 -2.274779 2 1
129
+ 121 66.1157 -1.384618 -0.953072 2 2
130
+ 122 66.3934 -6.421969 -4.793217 2 2
131
+ 123 65.8537 -2.888857 -2.795301 2 1
132
+ 124 66.1290 -3.102995 -2.639677 2 2
133
+ 125 66.4000 -4.036875 -3.267375 2 2
134
+ 126 65.8730 -7.121170 -7.223282 1 2
135
+ 127 66.1417 -2.585284 -1.516536 2 2
136
+ 128 66.4062 -2.060086 -2.365982 1 1
137
+ 129 66.6667 -2.233005 -1.267313 2 2
138
+ 130 66.9231 -5.599187 -5.458332 2 2
139
+ 131 66.4122 -3.448964 -3.838318 1 2
140
+ 132 66.6667 -3.109529 -2.918217 2 2
141
+ 133 66.9173 -1.769546 -4.032725 1 1
142
+ 134 67.1642 -4.996045 -4.427151 2 2
143
+ 135 67.4074 -4.532968 -5.405071 1 1
144
+ 136 67.6471 -5.397173 -4.022269 2 2
145
+ 137 67.8832 -6.159969 -6.327109 1 1
146
+ 138 68.1159 -4.469512 -4.461008 2 2
147
+ 139 67.6259 -4.863276 -4.406518 2 1
148
+ 140 67.8571 -1.530824 -1.194042 2 2
149
+ 141 68.0851 -4.393756 -3.921801 2 2
150
+ 142 68.3099 -2.934736 -2.891924 2 2
151
+ 143 68.5315 -1.624732 -6.740814 1 1
152
+ 144 68.0556 -2.125757 -2.014686 2 1
153
+ 145 67.5862 -1.689354 -13.924341 1 2
154
+ 146 67.8082 -4.231803 -3.737949 2 2
155
+ 147 68.0272 -5.268581 -5.234885 2 2
156
+ 148 68.2432 -2.082044 -2.586201 1 1
157
+ 149 67.7852 -3.444744 -3.528258 1 2
158
+ 150 68.0000 -4.068851 -3.256299 2 2
159
+ 151 67.5497 -2.416000 -2.247530 2 1
160
+ 152 67.1053 -3.888351 -4.084313 1 2
161
+ 153 67.3203 -4.351574 -4.618624 1 1
162
+ 154 67.5325 -3.692674 -3.796716 1 1
163
+ 155 67.0968 -2.185930 -2.225910 1 2
164
+ 156 66.6667 -2.602103 -2.439901 2 1
165
+ 157 66.8790 -3.658721 -4.785900 1 1
166
+ 158 67.0886 -4.291919 -3.958784 2 2
167
+ 159 67.2956 -3.407285 -5.142665 1 1
168
+ 160 67.5000 -4.096056 -3.002550 2 2
169
+ 161 67.7019 -3.216218 -3.596185 1 1
170
+ 162 67.2840 -0.955891 -14.203045 1 2
171
+ 163 67.4847 -1.914506 -1.996020 1 1
172
+ 164 67.6829 -5.126400 -4.432253 2 2
173
+ 165 67.2727 -4.588373 -4.795654 1 2
174
+ 166 67.4699 -3.972745 -3.629868 2 2
175
+ 167 67.0659 -5.478478 -5.177651 2 1
176
+ 168 66.6667 -4.618664 -5.105081 1 2
177
+ 169 66.2722 -3.736616 -4.246635 1 2
178
+ 170 65.8824 -4.791512 -4.199450 2 1
179
+ 171 66.0819 -3.484462 -4.675864 1 1
180
+ 172 66.2791 -4.908723 -5.467742 1 1
181
+ 173 66.4740 -3.540135 -3.013892 2 2
182
+ 174 66.6667 -7.132929 -6.539358 2 2
183
+ 175 66.2857 -4.776808 -4.563461 2 1
184
+ 176 66.4773 -1.515423 -3.298390 1 1
185
+ 177 66.6667 -5.572706 -5.133069 2 2
186
+ 178 66.2921 -2.592910 -2.187837 2 1
187
+ 179 66.4804 -2.797251 -2.818381 1 1
188
+ 180 66.6667 -5.672459 -5.211691 2 2
189
+ 181 66.8508 -3.803691 -2.830240 2 2
190
+ 182 67.0330 -4.795970 -4.382248 2 2
191
+ 183 67.2131 -6.442017 -6.025576 2 2
192
+ 184 67.3913 -2.327852 -3.551183 1 1
193
+ 185 67.0270 -6.216490 -6.269615 1 2
194
+ 186 67.2043 -3.176242 -3.763793 1 1
195
+ 187 67.3797 -4.956929 -4.945092 2 2
196
+ 188 67.0213 -3.266803 -3.165568 2 1
197
+ 189 66.6667 -5.216561 -5.311581 1 2
198
+ 190 66.8421 -5.175112 -4.583391 2 2
199
+ 191 67.0157 -4.126638 -4.128599 1 1
200
+ 192 67.1875 -6.033358 -4.256539 2 2
201
+ 193 67.3575 -4.140415 -4.057361 2 2
202
+ 194 67.5258 -2.172437 -1.411652 2 2
203
+ 195 67.6923 -4.098838 -4.125308 1 1
204
+ 196 67.8571 -4.007442 -3.391981 2 2
205
+ 197 68.0203 -0.923275 -3.170093 1 1
206
+ 198 68.1818 -5.339823 -5.858106 1 1
207
+ 199 68.3417 -4.463627 -3.957210 2 2
208
+ 200 68.5000 -3.421924 -3.227380 2 2
209
+ 201 68.6567 -2.668170 -3.422923 1 1
210
+ 202 68.8119 -7.881493 -7.175443 2 2
211
+ 203 68.9655 -3.338203 -3.236547 2 2
212
+ 204 69.1176 -3.340639 -3.470735 1 1
213
+ 205 69.2683 -4.131991 -3.373822 2 2
214
+ 206 68.9320 -5.668425 -6.035740 1 2
215
+ 207 69.0821 -2.418905 -2.713939 1 1
216
+ 208 69.2308 -4.607456 -3.124019 2 2
217
+ 209 69.3780 -4.464358 -4.201508 2 2
218
+ 210 69.0476 -7.932397 -8.625921 1 2
219
+ 211 68.7204 -6.547441 -6.068893 2 1
220
+ 212 68.8679 -4.623966 -3.354893 2 2
221
+ 213 68.5446 -5.170671 -5.568387 1 2
222
+ 214 68.6916 -2.831274 -3.069042 1 1
223
+ 215 68.8372 -5.621704 -4.907341 2 2
224
+ 216 68.5185 -3.683287 -3.672051 2 1
225
+ 217 68.6636 -1.011282 -1.709117 1 1
226
+ 218 68.8073 -2.604565 -2.489355 2 2
227
+ 219 68.9498 -2.720432 -3.175932 1 1
228
+ 220 68.6364 -3.226118 -3.460047 1 2
229
+ 221 68.7783 -2.815430 -8.855265 1 1
230
+ 222 68.4685 -0.266883 -5.868640 1 2
231
+ 223 68.6099 -4.732946 -4.640093 2 2
232
+ 224 68.3036 -5.897593 -6.446714 1 2
233
+ 225 68.4444 -4.128978 -3.303990 2 2
234
+ 226 68.5841 -3.228155 -3.929738 1 1
235
+ 227 68.2819 -4.279943 -3.587930 2 1
236
+ 228 68.4211 -1.395466 -3.237124 1 1
237
+ 229 68.5590 -4.669386 -4.578946 2 2
238
+ 230 68.6957 -3.224326 -3.365432 1 1
239
+ 231 68.8312 -1.975234 -2.291529 1 1
240
+ 232 68.9655 -2.485839 -1.574110 2 2
241
+ 233 69.0987 -1.982712 -2.667977 1 1
242
+ 234 69.2308 -4.296283 -3.811112 2 2
243
+ 235 69.3617 -3.930235 -3.546640 2 2
244
+ 236 69.0678 -2.606013 -2.315397 2 1
245
+ 237 68.7764 -0.928193 -0.883929 2 1
246
+ 238 68.9076 -3.401902 -3.037450 2 2
247
+ 239 69.0377 -2.629865 -2.387428 2 2
248
+ 240 68.7500 -7.277557 -7.485116 1 2
249
+ 241 68.8797 -4.101169 -3.101131 2 2
250
+ 242 69.0083 -6.342686 -5.810444 2 2
251
+ 243 69.1358 -4.389893 -4.558333 1 1
252
+ 244 69.2623 -2.013572 -3.580997 1 1
253
+ 245 69.3878 -4.882235 -5.682190 1 1
254
+ 246 69.5122 -2.608390 -3.538750 1 1
255
+ 247 69.2308 -4.891563 -5.596321 1 2
256
+ 248 69.3548 -3.128388 -2.975576 2 2
257
+ 249 69.4779 -3.093555 -1.916871 2 2
258
+ 250 69.6000 -3.242072 -3.871322 1 1
259
+ 251 69.3227 -0.923310 -0.821718 2 1
260
+ 252 69.4444 -6.757301 -6.609255 2 2
261
+ 253 69.5652 -5.144303 -5.521512 1 1
262
+ 254 69.6850 -3.880278 -3.204643 2 2
263
+ 255 69.8039 -3.260842 -5.137737 1 1
264
+ 256 69.9219 -3.792717 -4.334153 1 1
265
+ 257 69.6498 -2.532104 -2.030293 2 1
266
+ 258 69.3798 -3.487507 -3.845524 1 2
267
+ 259 69.1120 -4.239560 -4.199788 2 1
268
+ 260 69.2308 -4.588445 -5.125564 1 1
269
+ 261 69.3487 -3.659524 -3.579194 2 2
270
+ 262 69.4656 -5.233979 -5.114939 2 2
271
+ 263 69.5817 -4.020043 -5.251196 1 1
272
+ 264 69.6970 -4.567102 -4.243685 2 2
273
+ 265 69.8113 -1.373382 -2.061783 1 1
274
+ 266 69.9248 -6.456631 -8.595693 1 1
275
+ 267 70.0375 -3.852945 -3.887883 1 1
276
+ 268 69.7761 -2.954614 -2.728430 2 1
277
+ 269 69.8885 -3.587453 -3.276672 2 2
278
+ 270 70.0000 -2.557718 -4.211267 1 1
279
+ 271 70.1107 -3.090135 -2.931706 2 2
280
+ 272 70.2206 -4.301549 -3.816646 2 2
281
+ 273 70.3297 -3.691031 -3.339726 2 2
282
+ 274 70.4380 -3.056009 -2.347582 2 2
283
+ 275 70.5455 -1.376791 -2.034060 1 1
284
+ 276 70.2899 -2.519263 -2.428876 2 1
285
+ 277 70.0361 -3.325269 -3.190564 2 1
286
+ 278 70.1439 -2.595628 -2.601255 1 1
287
+ 279 70.2509 -4.245304 -3.043910 2 2
288
+ 280 70.3571 -4.034679 -3.748404 2 2
289
+ 281 70.4626 -2.330715 -1.899545 2 2
290
+ 282 70.5674 -4.198890 -3.799667 2 2
291
+ 283 70.6714 -3.511445 -3.419577 2 2
292
+ 284 70.7746 -2.691241 -2.831009 1 1
293
+ 285 70.8772 -6.204762 -6.430551 1 1
294
+ 286 70.9790 -3.544106 -3.636078 1 1
295
+ 287 71.0801 -2.448456 -3.843027 1 1
296
+ 288 71.1806 -3.566141 -2.884309 2 2
297
+ 289 71.2803 -2.651893 -3.085881 1 1
298
+ 290 71.3793 -5.568446 -6.386772 1 1
299
+ 291 71.4777 -1.870223 -3.303023 1 1
300
+ 292 71.5753 -2.434567 -2.274573 2 2
301
+ 293 71.6724 -3.522840 -2.883008 2 2
302
+ 294 71.4286 -5.068295 -3.872059 2 1
303
+ 295 71.5254 -1.977107 -2.363389 1 1
304
+ 296 71.6216 -4.856613 -4.796801 2 2
305
+ 297 71.3805 -1.117338 -1.310553 1 2
306
+ 298 71.1409 -6.512083 -7.997723 1 2
307
+ 299 71.2375 -2.470865 -1.853013 2 2
308
+ 300 71.3333 -3.254343 -3.983928 1 1
309
+ 301 71.4286 -4.542827 -5.172909 1 1
310
+ 302 71.5232 -4.556103 -4.458071 2 2
311
+ 303 71.2871 -9.185862 -7.805008 2 1
312
+ 304 71.3816 -3.555370 -3.537802 2 2
313
+ 305 71.1475 -2.854463 -3.213757 1 2
314
+ 306 71.2418 -2.612833 -2.237679 2 2
315
+ 307 71.3355 -2.797689 -2.952635 1 1
316
+ 308 71.1039 -4.392760 -4.993405 1 2
317
+ 309 71.1974 -3.379035 -4.161753 1 1
318
+ 310 71.2903 -5.040904 -4.931065 2 2
319
+ 311 71.3826 -1.606420 -5.219582 1 1
320
+ 312 71.1538 -5.727122 -4.291016 2 1
321
+ 313 70.9265 -4.786856 -5.595756 1 2
322
+ 314 71.0191 -6.393810 -5.541536 2 2
323
+ 315 71.1111 -2.568756 -3.822536 1 1
324
+ 316 71.2025 -6.888626 -5.591304 2 2
325
+ 317 70.9779 -4.753928 -4.356578 2 1
326
+ 318 71.0692 -5.062870 -4.634061 2 2
327
+ 319 71.1599 -2.361433 -3.293596 1 1
328
+ 320 71.2500 -3.622120 -5.321121 1 1
329
+ 321 71.0280 -3.078336 -3.087999 1 2
330
+ 322 70.8075 -2.539727 -2.575897 1 2
331
+ 323 70.8978 -2.807223 -3.223767 1 1
332
+ 324 70.9877 -4.222728 -4.338785 1 1
333
+ 325 71.0769 -5.541180 -4.771935 2 2
334
+ 326 71.1656 -2.914106 -2.102217 2 2
335
+ 327 71.2538 -2.769662 -3.574931 1 1
336
+ 328 71.3415 -4.158017 -4.354668 1 1
337
+ 329 71.4286 -4.144868 -10.872568 1 1
338
+ 330 71.5152 -0.860563 -1.965008 1 1
339
+ 331 71.2991 -5.620200 -3.889178 2 1
340
+ 332 71.3855 -1.878913 -2.861038 1 1
341
+ 333 71.4715 -2.761050 -2.459287 2 2
342
+ 334 71.5569 -2.583628 -5.924510 1 1
343
+ 335 71.6418 -4.226890 -4.544033 1 1
344
+ 336 71.4286 -3.390356 -3.949334 1 2
345
+ 337 71.2166 -1.866804 -2.040254 1 2
346
+ 338 71.0059 -4.425655 -3.502093 2 1
347
+ 339 71.0914 -5.860173 -5.009783 2 2
348
+ 340 71.1765 -5.228643 -5.690258 1 1
349
+ 341 71.2610 -2.774680 -4.404914 1 1
350
+ 342 71.3450 -6.080098 -5.911227 2 2
351
+ 343 71.4286 -3.922923 -4.372533 1 1
352
+ 344 71.2209 -2.919039 -2.811806 2 1
353
+ 345 71.3043 -1.329802 -3.187450 1 1
354
+ 346 71.0983 -4.670353 -6.695405 1 2
355
+ 347 71.1816 -4.273296 -3.738665 2 2
356
+ 348 71.2644 -1.319347 -2.582942 1 1
357
+ 349 71.0602 -3.617547 -5.551081 1 2
358
+ 350 70.8571 -1.364061 -4.248843 1 2
359
+ 351 70.9402 -2.235479 -3.292579 1 1
360
+ 352 71.0227 -3.897244 -2.875620 2 2
361
+ 353 70.8215 -2.251068 -1.924899 2 1
362
+ 354 70.9040 -5.348984 -3.769576 2 2
363
+ 355 70.9859 -3.050266 -2.523066 2 2
364
+ 356 71.0674 -3.746408 -3.275239 2 2
365
+ 357 71.1485 -4.260843 -3.881188 2 2
366
+ 358 71.2291 -2.666503 -2.545099 2 2
367
+ 359 71.3092 -6.269689 -3.965765 2 2
368
+ 360 71.1111 -2.654946 -3.038751 1 2
369
+ 361 71.1911 -3.691508 -1.626953 2 2
370
+ 362 71.2707 -5.480256 -5.199198 2 2
371
+ 363 71.0744 -2.501657 -2.459157 2 1
372
+ 364 71.1538 -6.048635 -6.112708 1 1
373
+ 365 71.2329 -5.805447 -6.266085 1 1
374
+ 366 71.0383 -4.722961 -10.664696 1 2
375
+ 367 71.1172 -5.474047 -5.816067 1 1
376
+ 368 71.1957 -3.350766 -1.420245 2 2
377
+ 369 71.0027 -3.176488 -3.400327 1 2
378
+ 370 71.0811 -4.569815 -4.873927 1 1
379
+ 371 71.1590 -4.704939 -5.325861 1 1
380
+ 372 70.9677 -2.685040 -2.335501 2 1
381
+ 373 71.0456 -5.437434 -5.841838 1 1
382
+ 374 71.1230 -2.017469 -2.935250 1 1
383
+ 375 71.2000 -3.206317 -2.673229 2 2
384
+ 376 71.2766 -5.443141 -3.605138 2 2
385
+ 377 71.0875 -3.964123 -4.110891 1 2
386
+ 378 70.8995 -3.016566 -3.340574 1 2
387
+ 379 70.7124 -1.698172 -2.706369 1 2
388
+ 380 70.5263 -7.952806 -7.392488 2 1
389
+ 381 70.6037 -10.916053 -9.264097 2 2
390
+ 382 70.4188 -2.366687 -2.439604 1 2
391
+ 383 70.4961 -2.029263 -0.867887 2 2
392
+ 384 70.3125 -4.084346 -3.088594 2 1
393
+ 385 70.1299 -1.364663 -1.106582 2 1
394
+ 386 70.2073 -2.844437 -9.548964 1 1
395
+ 387 70.2842 -1.518799 -1.948244 1 1
396
+ 388 70.1031 -5.229100 -6.003133 1 2
397
+ 389 70.1799 -3.063151 -2.231602 2 2
398
+ 390 70.2564 -2.623027 -3.826552 1 1
399
+ 391 70.0767 -3.662146 -5.157445 1 2
400
+ 392 69.8980 -5.738766 -4.531609 2 1
401
+ 393 69.9746 -7.608169 -6.958110 2 2
402
+ 394 70.0508 -1.982103 -4.968424 1 1
403
+ 395 70.1266 -4.373048 -4.315045 2 2
404
+ 396 70.2020 -5.285030 -5.518341 1 1
405
+ 397 70.0252 -7.407098 -6.770819 2 1
406
+ 398 70.1005 -5.287242 -8.393716 1 1
407
+ 399 70.1754 -5.197666 -5.354281 1 1
408
+ 400 70.2500 -1.899929 -4.965192 1 1
409
+ 401 70.3242 -2.784137 -3.005195 1 1
410
+ 402 70.1493 -3.950481 -4.171018 1 2
411
+ 403 69.9752 -6.666692 -6.869461 1 2
412
+ 404 70.0495 -3.705375 -2.380541 2 2
413
+ 405 70.1235 -3.299920 -4.267696 1 1
414
+ 406 70.1970 -8.341837 -9.434620 1 1
415
+ 407 70.0246 -5.355197 -5.398414 1 2
416
+ 408 70.0980 -3.373358 -3.119762 2 2
417
+ 409 70.1711 -4.123866 -4.315636 1 1
418
+ 410 70.2439 -5.170630 -5.677562 1 1
419
+ 411 70.3163 -5.576116 -6.080760 1 1
420
+ 412 70.3883 -4.135100 -3.112332 2 2
421
+ 413 70.4600 -4.103818 -2.740623 2 2
422
+ 414 70.2899 -4.904391 -5.255712 1 2
423
+ 415 70.3614 -2.725536 -2.538948 2 2
424
+ 416 70.4327 -5.684915 -3.569008 2 2
425
+ 417 70.5036 -4.870584 -5.518087 1 1
426
+ 418 70.5742 -5.088659 -7.802388 1 1
427
+ 419 70.4057 -4.446722 -5.066386 1 2
428
+ 420 70.2381 -3.363349 -3.484886 1 2
429
+ 421 70.3088 -4.150953 -3.559892 2 2
430
+ 422 70.1422 -2.624155 -2.324282 2 1
431
+ 423 69.9764 -3.542715 -3.303008 2 1
432
+ 424 70.0472 -5.379400 -5.945479 1 1
433
+ 425 70.1176 -4.048955 -3.124197 2 2
434
+ 426 69.9531 -1.850860 -2.648764 1 2
435
+ 427 70.0234 -3.551065 -4.734993 1 1
436
+ 428 69.8598 -3.921132 -3.189312 2 1
437
+ 429 69.9301 -4.091774 -4.001524 2 2
438
+ 430 69.7674 -6.244325 -5.660656 2 1
439
+ 431 69.8376 -4.840743 -5.852038 1 1
440
+ 432 69.6759 -4.358678 -4.811829 1 2
441
+ 433 69.7460 -2.080915 -1.735903 2 2
442
+ 434 69.8157 -7.379761 -7.963377 1 1
443
+ 435 69.8851 -3.202125 -3.805578 1 1
444
+ 436 69.9541 -0.750618 -2.244745 1 1
445
+ 437 70.0229 -3.663185 -3.520825 2 2
446
+ 438 70.0913 -3.979266 -3.861609 2 2
447
+ 439 69.9317 -3.825572 -2.954645 2 1
448
+ 440 70.0000 -4.562495 -5.200001 1 1
449
+ 441 70.0680 -4.127037 -5.332975 1 1
450
+ 442 69.9095 -4.959048 -4.725274 2 1
451
+ 443 69.9774 -1.556460 -3.383786 1 1
452
+ 444 69.8198 -5.935822 -7.447041 1 2
453
+ 445 69.8876 -1.820385 -3.143990 1 1
454
+ 446 69.9552 -5.895039 -4.932636 2 2
455
+ 447 70.0224 -1.803761 -1.873815 1 1
456
+ 448 70.0893 -4.252747 -3.634388 2 2
457
+ 449 70.1559 -2.927382 -2.872544 2 2
458
+ 450 70.2222 -4.081425 -3.708212 2 2
459
+ 451 70.2882 -4.334718 -3.334052 2 2
460
+ 452 70.3540 -5.016918 -5.338756 1 1
461
+ 453 70.4194 -5.328543 -4.651102 2 2
462
+ 454 70.4846 -3.177649 -3.638199 1 1
463
+ 455 70.3297 -2.284919 -9.508350 1 2
464
+ 456 70.3947 -4.537247 -4.512091 2 2
465
+ 457 70.2407 -10.091568 -9.675279 2 1
466
+ 458 70.3057 -4.070455 -3.879054 2 2
467
+ 459 70.3704 -4.616488 -4.388407 2 2
468
+ 460 70.4348 -1.119833 -7.968352 1 1
469
+ 461 70.4989 -2.786390 -3.860716 1 1
470
+ 462 70.5628 -4.891298 -5.117933 1 1
471
+ 463 70.6263 -0.902819 -1.138866 1 1
472
+ 464 70.4741 -2.882805 -2.758255 2 1
473
+ 465 70.5376 -3.241472 -3.226279 2 2
474
+ 466 70.6009 -5.377877 -4.973397 2 2
475
+ 467 70.6638 -4.900952 -5.317527 1 1
476
+ 468 70.7265 -5.608815 -4.230068 2 2
477
+ 469 70.5757 -2.037160 -1.777424 2 1
478
+ 470 70.6383 -1.665616 -1.345727 2 2
479
+ 471 70.7006 -2.264172 -2.442881 1 1
480
+ 472 70.7627 -4.296752 -3.732426 2 2
481
+ 473 70.8245 -4.400584 -3.096070 2 2
482
+ 474 70.8861 -5.081336 -5.701107 1 1
483
+ 475 70.9474 -1.703756 -1.231129 2 2
484
+ 476 70.7983 -4.229283 -4.639537 1 2
485
+ 477 70.6499 -4.787489 -5.357100 1 2
486
+ 478 70.5021 -4.483231 -3.957478 2 1
487
+ 479 70.5637 -3.293798 -3.605517 1 1
488
+ 480 70.4167 -5.243164 -5.366000 1 2
489
+ 481 70.4782 -2.187320 -2.588389 1 1
490
+ 482 70.3320 -4.514816 -5.421480 1 2
491
+ 483 70.1863 -5.701480 -5.043275 2 1
492
+ 484 70.0413 -2.534904 -1.676203 2 1
493
+ 485 70.1031 -1.830878 -2.584952 1 1
494
+ 486 70.1646 -2.649925 -3.323503 1 1
495
+ 487 70.2259 -2.822470 -1.968600 2 2
496
+ 488 70.2869 -3.255502 -2.363632 2 2
497
+ 489 70.3476 -2.339654 -2.498749 1 1
498
+ 490 70.4082 -3.196512 -3.733702 1 1
499
+ 491 70.4684 -5.654249 -4.734473 2 2
500
+ 492 70.5285 -5.648119 -5.288558 2 2
501
+ 493 70.5882 -1.661824 -3.309512 1 1
502
+ 494 70.6478 -4.794731 -4.682871 2 2
503
+ 495 70.7071 -4.230577 -4.352903 1 1
504
+ 496 70.5645 -3.541327 -3.785727 1 2
505
+ 497 70.4225 -5.885485 -5.395486 2 1
506
+ 498 70.2811 -1.416558 -1.879955 1 2
507
+ 499 70.3407 -2.698055 -3.185922 1 1
508
+ 500 70.4000 -5.490611 -4.531842 2 2
509
+ 501 70.4591 -1.514790 -2.667901 1 1
510
+ 502 70.5179 -1.434827 -2.095304 1 1
511
+ 503 70.5765 -2.995279 -2.637357 2 2
512
+ 504 70.6349 -2.802677 -2.845852 1 1
513
+ 505 70.4950 -6.897599 -7.657016 1 2
514
+ 506 70.3557 -6.787336 -6.142086 2 1
515
+ 507 70.4142 -4.434003 -4.783604 1 1
516
+ 508 70.4724 -6.494354 -2.793794 2 2
517
+ 509 70.5305 -3.409007 -7.970373 1 1
518
+ 510 70.5882 -3.396550 -3.589788 1 1
519
+ 511 70.6458 -5.240699 -4.006346 2 2
520
+ 512 70.7031 -3.603536 -4.171160 1 1
521
+ 513 70.7602 -3.976140 -3.751311 2 2
522
+ 514 70.8171 -5.494944 -5.740723 1 1
523
+ 515 70.8738 -6.342858 -6.586165 1 1
524
+ 516 70.9302 -4.410367 -4.544744 1 1
525
+ 517 70.7930 -1.710368 -1.520782 2 1
526
+ 518 70.8494 -2.912271 -1.786355 2 2
527
+ 519 70.9056 -9.724302 -9.675817 2 2
528
+ 520 70.9615 -2.666356 -2.062732 2 2
529
+ 521 71.0173 -1.097292 -1.315424 1 1
530
+ 522 71.0728 -3.965566 -3.190285 2 2
531
+ 523 70.9369 -3.948887 -4.039154 1 2
532
+ 524 70.8015 -7.575801 -7.356222 2 1
533
+ 525 70.8571 -5.185006 -1.856690 2 2
534
+ 526 70.9125 -0.660760 -16.515976 1 1
535
+ 527 70.9677 -6.980890 -5.160274 2 2
536
+ 528 71.0227 -6.387946 -6.326921 2 2
537
+ 529 71.0775 -4.761805 -6.141434 1 1
538
+ 530 71.1321 -5.460374 -4.453095 2 2
539
+ 531 70.9981 -4.103496 -3.734915 2 1
540
+ 532 71.0526 -2.327525 -3.133998 1 1
541
+ 533 71.1069 -3.448150 -2.783705 2 2
542
+ 534 71.1610 -2.968098 -3.607601 1 1
543
+ 535 71.0280 -3.350432 -3.379534 1 2
544
+ 536 70.8955 -2.512584 -2.249976 2 1
545
+ 537 70.9497 -5.818592 -6.358843 1 1
546
+ 538 71.0037 -2.565659 -3.217110 1 1
547
+ 539 71.0575 -4.447995 -3.106114 2 2
548
+ 540 70.9259 -2.406775 -2.992070 1 2
549
+ 541 70.9797 -2.429869 -3.041593 1 1
550
+ 542 70.8487 -3.960595 -5.142495 1 2
551
+ 543 70.9024 -5.436193 -4.923785 2 2
552
+ 544 70.9559 -2.622061 -4.502477 1 1
553
+ 545 70.8257 -2.075444 -5.739558 1 2
554
+ 546 70.8791 -2.753712 -3.910147 1 1
555
+ 547 70.9324 -6.035166 -4.476553 2 2
556
+ 548 70.8029 -4.651083 -4.512567 2 1
557
+ 549 70.6740 -5.523981 -5.510857 2 1
558
+ 550 70.5455 -3.403569 -3.404960 1 2
559
+ 551 70.4174 -2.398785 -7.484997 1 2
560
+ 552 70.4710 -2.632620 -3.381036 1 1
561
+ 553 70.5244 -6.248417 -5.648043 2 2
562
+ 554 70.3971 -4.708705 -4.902507 1 2
563
+ 555 70.4505 -3.029551 -2.115620 2 2
564
+ 556 70.5036 -5.938469 -5.162807 2 2
565
+ 557 70.5566 -4.693294 -5.504109 1 1
566
+ 558 70.6093 -2.032475 -3.389102 1 1
567
+ 559 70.6619 -2.554197 -2.432564 2 2
568
+ 560 70.5357 -2.801763 -1.924608 2 1
569
+ 561 70.5882 -2.131085 -2.582068 1 1
570
+ 562 70.6406 -1.112470 -1.851653 1 1
571
+ 563 70.5151 -1.544102 -2.900197 1 2
572
+ 564 70.5674 -3.381135 -3.775825 1 1
573
+ 565 70.6195 -3.806712 -3.578685 2 2
574
+ 566 70.6714 -4.534478 -5.076048 1 1
575
+ 567 70.7231 -2.965701 -2.771533 2 2
576
+ 568 70.7746 -7.085715 -4.485383 2 2
577
+ 569 70.6503 -5.769938 -4.915669 2 1
578
+ 570 70.7018 -4.499670 -6.547434 1 1
579
+ 571 70.7531 -4.884565 -3.534300 2 2
580
+ 572 70.8042 -2.766551 -2.592849 2 2
581
+ 573 70.6806 -4.288250 -4.670691 1 2
582
+ 574 70.7317 -3.760988 -2.515644 2 2
583
+ 575 70.7826 -4.849058 -4.539058 2 2
584
+ 576 70.8333 -2.893359 -3.526457 1 1
585
+ 577 70.8839 -4.595818 -4.356552 2 2
586
+ 578 70.9343 -3.338332 -3.099706 2 2
587
+ 579 70.8117 -5.451514 -5.566599 1 2
588
+ 580 70.8621 -1.670993 -3.221905 1 1
589
+ 581 70.9122 -5.841762 -8.182266 1 1
590
+ 582 70.7904 -5.298597 -5.706238 1 2
591
+ 583 70.8405 -4.486544 -4.319669 2 2
592
+ 584 70.7192 -4.506101 -4.448192 2 1
593
+ 585 70.7692 -0.340273 -1.264459 1 1
594
+ 586 70.8191 -3.824473 -4.161646 1 1
595
+ 587 70.6985 -5.447095 -5.158605 2 1
596
+ 588 70.7483 -4.205687 -2.498649 2 2
597
+ 589 70.6282 -3.211741 -3.554926 1 2
598
+ 590 70.6780 -4.165066 -3.761560 2 2
599
+ 591 70.7276 -3.986691 -3.479631 2 2
600
+ 592 70.7770 -3.380139 -4.205868 1 1
601
+ 593 70.8263 -4.693285 -4.586618 2 2
602
+ 594 70.7071 -5.236917 -4.371696 2 1
603
+ 595 70.7563 -2.009322 -1.956967 2 2
604
+ 596 70.8054 -4.920583 -4.518074 2 2
605
+ 597 70.6868 -2.344572 -2.340869 2 1
606
+ 598 70.7358 -2.990325 -3.512002 1 1
607
+ 599 70.7846 -2.955184 -4.071931 1 1
608
+ 600 70.8333 -3.725276 -3.479023 2 2
609
+ 601 70.8819 -3.345482 -2.774332 2 2
610
+ 602 70.9302 -3.120574 -5.259781 1 1
611
+ 603 70.9784 -6.821024 -5.401530 2 2
612
+ 604 70.8609 -3.072906 -3.091824 1 2
613
+ 605 70.7438 -3.101952 -2.222045 2 1
614
+ 606 70.7921 -3.271429 -2.954625 2 2
615
+ 607 70.6755 -2.993876 -2.582807 2 1
616
+ 608 70.7237 -3.274199 -3.863012 1 1
617
+ 609 70.6076 -7.414813 -6.972252 2 1
618
+ 610 70.6557 -3.047330 -3.309754 1 1
619
+ 611 70.5401 -4.648418 -4.597436 2 1
620
+ 612 70.5882 -4.957109 -6.748741 1 1
621
+ 613 70.6362 -2.885153 -2.082744 2 2
622
+ 614 70.5212 -5.817165 -5.857469 1 2
623
+ 615 70.5691 -3.118172 -2.839146 2 2
624
+ 616 70.6169 -5.240674 -5.197654 2 2
625
+ 617 70.6645 -13.518783 -5.767840 2 2
626
+ 618 70.5502 -3.947356 -4.208788 1 2
627
+ 619 70.5977 -3.844186 -3.427033 2 2
628
+ 620 70.4839 -5.084860 -4.134609 2 1
629
+ 621 70.5314 -3.304082 -2.182160 2 2
630
+ 622 70.5788 -3.348047 -3.765050 1 1
631
+ 623 70.6260 -3.029715 -3.214550 1 1
632
+ 624 70.6731 -4.646512 -3.868998 2 2
633
+ 625 70.7200 -4.658016 -6.392656 1 1
634
+ 626 70.6070 -1.922071 -1.188342 2 1
635
+ 627 70.6539 -3.615727 -3.987148 1 1
636
+ 628 70.7006 -3.374762 -2.593869 2 2
637
+ 629 70.7472 -2.040991 -2.590285 1 1
638
+ 630 70.7937 -1.228370 -1.742215 1 1
639
+ 631 70.8399 -2.629596 -4.010056 1 1
640
+ 632 70.8861 -1.902431 -1.661982 2 2
641
+ 633 70.7741 -2.123907 -2.159136 1 2
642
+ 634 70.8202 -2.749724 -3.731588 1 1
643
+ 635 70.7087 -4.641824 -4.755588 1 2
644
+ 636 70.7547 -1.459129 -3.026493 1 1
645
+ 637 70.8006 -3.109084 -5.443490 1 1
646
+ 638 70.8464 -5.834625 -6.763369 1 1
647
+ 639 70.8920 -2.053847 -2.563142 1 1
648
+ 640 70.7812 -2.094434 -2.631055 1 2
649
+ 641 70.6708 -6.828894 -5.673113 2 1
650
+ 642 70.7165 -5.198835 -3.722330 2 2
651
+ 643 70.7621 -2.078644 -1.353701 2 2
652
+ 644 70.8075 -2.858294 -4.172447 1 1
653
+ 645 70.8527 -1.002175 -2.339788 1 1
654
+ 646 70.7430 -3.923726 -3.636564 2 1
655
+ 647 70.6337 -3.297742 -2.430887 2 1
656
+ 648 70.6790 -3.148511 -2.967418 2 2
657
+ 649 70.7242 -3.489189 -3.831672 1 1
658
+ 650 70.7692 -4.060736 -4.706410 1 1
659
+ 651 70.6605 -5.170542 -6.020569 1 2
660
+ 652 70.7055 -5.701991 -5.517997 2 2
661
+ 653 70.7504 -5.117662 -3.670259 2 2
662
+ 654 70.6422 -3.444314 -3.881610 1 2
663
+ 655 70.5344 -2.473348 -2.649559 1 2
664
+ 656 70.5793 -4.149153 -3.096317 2 2
665
+ 657 70.4718 -5.299390 -5.648738 1 2
666
+ 658 70.5167 -4.242005 -1.638720 2 2
667
+ 659 70.5615 -2.664886 -1.742135 2 2
668
+ 660 70.4545 -3.329002 -4.026456 1 2
669
+ 661 70.4992 -3.622036 -4.022227 1 1
670
+ 662 70.5438 -2.947179 -3.756589 1 1
671
+ 663 70.5882 -6.716118 -6.587849 2 2
672
+ 664 70.4819 -4.089277 -3.941725 2 1
673
+ 665 70.5263 -3.370888 -3.253590 2 2
674
+ 666 70.5706 -3.476680 -4.962936 1 1
675
+ 667 70.6147 -8.010424 -6.939659 2 2
676
+ 668 70.5090 -2.305406 -2.200042 2 1
677
+ 669 70.5531 -2.315434 -3.122011 1 1
678
+ 670 70.4478 -3.551354 -3.387468 2 1
679
+ 671 70.4918 -3.482282 -4.814491 1 1
680
+ 672 70.5357 -3.203128 -3.107714 2 2
681
+ 673 70.5795 -1.749583 -1.660100 2 2
682
+ 674 70.4748 -4.851263 -4.776650 2 1
683
+ 675 70.3704 -3.920711 -3.530686 2 1
684
+ 676 70.2663 -2.203540 -2.575792 1 2
685
+ 677 70.3102 -3.520045 -3.805712 1 1
686
+ 678 70.2065 -5.684715 -5.953469 1 2
687
+ 679 70.2504 -5.558782 -4.597155 2 2
688
+ 680 70.2941 -3.306087 -4.027033 1 1
689
+ 681 70.3377 -3.038645 -3.549582 1 1
690
+ 682 70.2346 -2.430589 -2.433201 1 2
691
+ 683 70.2782 -2.007927 -3.543745 1 1
692
+ 684 70.3216 -4.993980 -7.056942 1 1
693
+ 685 70.3650 -4.765462 -2.212383 2 2
694
+ 686 70.2624 -5.349525 -5.823514 1 2
695
+ 687 70.3057 -4.599239 -3.467577 2 2
696
+ 688 70.3488 -8.094491 -8.233539 1 1
697
+ 689 70.3919 -2.828684 -2.402968 2 2
698
+ 690 70.2899 -2.409560 -2.598195 1 2
699
+ 691 70.1881 -2.926509 -2.491089 2 1
700
+ 692 70.2312 -2.007058 -2.644291 1 1
701
+ 693 70.2742 -4.856884 -5.512000 1 1
702
+ 694 70.1729 -3.441837 -3.472530 1 2
703
+ 695 70.2158 -3.695871 -5.036958 1 1
704
+ 696 70.2586 -4.238877 -5.484143 1 1
705
+ 697 70.3013 -2.265965 -2.533708 1 1
706
+ 698 70.3438 -4.333498 -4.023142 2 2
707
+ 699 70.3863 -3.460687 -4.524829 1 1
708
+ 700 70.4286 -4.268239 -3.849057 2 2
709
+ 701 70.3281 -2.714237 -2.495793 2 1
710
+ 702 70.2279 -4.269673 -5.149114 1 2
711
+ 703 70.2703 -3.752163 -3.829759 1 1
712
+ 704 70.3125 -1.023005 -1.594468 1 1
713
+ 705 70.2128 -3.164422 -3.361421 1 2
714
+ 706 70.1133 -3.443037 -3.659791 1 2
715
+ 707 70.0141 -4.805991 -4.553332 2 1
716
+ 708 70.0565 -4.263180 -3.302231 2 2
717
+ 709 70.0987 -3.187586 -2.622000 2 2
718
+ 710 70.1408 -3.782768 -4.684071 1 1
719
+ 711 70.0422 -1.684146 -2.185022 1 2
720
+ 712 69.9438 -2.076753 -3.827259 1 2
721
+ 713 69.8457 -3.459714 -3.532904 1 2
722
+ 714 69.7479 -3.204956 -3.358673 1 2
723
+ 715 69.7902 -4.069281 -3.244724 2 2
724
+ 716 69.8324 -4.830584 -4.751895 2 2
725
+ 717 69.8745 -1.664510 -0.276591 2 2
726
+ 718 69.7772 -4.598638 -4.349403 2 1
727
+ 719 69.6801 -4.889815 -4.699295 2 1
728
+ 720 69.5833 -2.463095 -5.407917 1 2
729
+ 721 69.4868 -5.040697 -5.750609 1 2
730
+ 722 69.3906 -1.949793 -1.922879 2 1
731
+ 723 69.4329 -2.353966 -2.961028 1 1
732
+ 724 69.4751 -4.149101 -3.963925 2 2
733
+ 725 69.3793 -2.145051 -2.891193 1 2
734
+ 726 69.4215 -4.654620 -4.360547 2 2
735
+ 727 69.4635 -3.118649 -2.715597 2 2
736
+ 728 69.5055 -3.454545 -3.466176 1 1
737
+ 729 69.5473 -1.394623 -2.717707 1 1
738
+ 730 69.5890 -4.386215 -5.016888 1 1
739
+ 731 69.6306 -4.852068 -4.932930 1 1
740
+ 732 69.6721 -3.050046 -2.201308 2 2
741
+ 733 69.7135 -6.942048 -6.522440 2 2
742
+ 734 69.7548 -1.000131 -11.445993 1 1
743
+ 735 69.6599 -3.796953 -3.352941 2 1
744
+ 736 69.7011 -2.372715 -3.788441 1 1
745
+ 737 69.7422 -4.622221 -5.429656 1 1
746
+ 738 69.6477 -4.373654 -4.699358 1 2
747
+ 739 69.6888 -0.608700 -5.884484 1 1
748
+ 740 69.7297 -4.713471 -3.285275 2 2
749
+ 741 69.6356 -4.796085 -4.531281 2 1
750
+ 742 69.6765 -2.247282 -1.977303 2 2
751
+ 743 69.7174 -4.045102 -4.009606 2 2
752
+ 744 69.7581 -4.703612 -4.107315 2 2
753
+ 745 69.7987 -2.862856 -3.553505 1 1
754
+ 746 69.8391 -7.450499 -7.797253 1 1
755
+ 747 69.8795 -5.526339 -6.663438 1 1
756
+ 748 69.9198 -4.481537 -3.677316 2 2
757
+ 749 69.9599 -4.241408 -5.343355 1 1
758
+ 750 70.0000 -3.153725 -7.031760 1 1
759
+
760
+ Final Winogrande score(750 tasks): 70.0000 +/- 1.6744
scores/Hammer2.1-7b-iq4_nl.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 57.14285714
18
+ 8 62.50000000
19
+ 9 55.55555556
20
+ 10 60.00000000
21
+ 11 54.54545455
22
+ 12 58.33333333
23
+ 13 61.53846154
24
+ 14 64.28571429
25
+ 15 66.66666667
26
+ 16 62.50000000
27
+ 17 64.70588235
28
+ 18 61.11111111
29
+ 19 57.89473684
30
+ 20 55.00000000
31
+ 21 52.38095238
32
+ 22 54.54545455
33
+ 23 52.17391304
34
+ 24 54.16666667
35
+ 25 52.00000000
36
+ 26 50.00000000
37
+ 27 51.85185185
38
+ 28 50.00000000
39
+ 29 51.72413793
40
+ 30 50.00000000
41
+ 31 48.38709677
42
+ 32 46.87500000
43
+ 33 45.45454545
44
+ 34 44.11764706
45
+ 35 45.71428571
46
+ 36 44.44444444
47
+ 37 45.94594595
48
+ 38 47.36842105
49
+ 39 48.71794872
50
+ 40 50.00000000
51
+ 41 51.21951220
52
+ 42 52.38095238
53
+ 43 51.16279070
54
+ 44 50.00000000
55
+ 45 51.11111111
56
+ 46 52.17391304
57
+ 47 53.19148936
58
+ 48 54.16666667
59
+ 49 53.06122449
60
+ 50 54.00000000
61
+ 51 52.94117647
62
+ 52 53.84615385
63
+ 53 54.71698113
64
+ 54 53.70370370
65
+ 55 52.72727273
66
+ 56 53.57142857
67
+ 57 54.38596491
68
+ 58 55.17241379
69
+ 59 55.93220339
70
+ 60 56.66666667
71
+ 61 57.37704918
72
+ 62 56.45161290
73
+ 63 57.14285714
74
+ 64 57.81250000
75
+ 65 56.92307692
76
+ 66 57.57575758
77
+ 67 58.20895522
78
+ 68 58.82352941
79
+ 69 57.97101449
80
+ 70 57.14285714
81
+ 71 56.33802817
82
+ 72 55.55555556
83
+ 73 56.16438356
84
+ 74 56.75675676
85
+ 75 57.33333333
86
+ 76 57.89473684
87
+ 77 58.44155844
88
+ 78 58.97435897
89
+ 79 58.22784810
90
+ 80 58.75000000
91
+ 81 58.02469136
92
+ 82 58.53658537
93
+ 83 57.83132530
94
+ 84 57.14285714
95
+ 85 57.64705882
96
+ 86 58.13953488
97
+ 87 58.62068966
98
+ 88 59.09090909
99
+ 89 59.55056180
100
+ 90 58.88888889
101
+ 91 59.34065934
102
+ 92 59.78260870
103
+ 93 59.13978495
104
+ 94 58.51063830
105
+ 95 58.94736842
106
+ 96 59.37500000
107
+ 97 58.76288660
108
+ 98 58.16326531
109
+ 99 58.58585859
110
+ 100 59.00000000
111
+ 101 58.41584158
112
+ 102 58.82352941
113
+ 103 58.25242718
114
+ 104 57.69230769
115
+ 105 58.09523810
116
+ 106 57.54716981
117
+ 107 57.94392523
118
+ 108 57.40740741
119
+ 109 56.88073394
120
+ 110 57.27272727
121
+ 111 57.65765766
122
+ 112 58.03571429
123
+ 113 57.52212389
124
+ 114 57.89473684
125
+ 115 58.26086957
126
+ 116 57.75862069
127
+ 117 57.26495726
128
+ 118 57.62711864
129
+ 119 57.14285714
130
+ 120 57.50000000
131
+ 121 57.02479339
132
+ 122 57.37704918
133
+ 123 57.72357724
134
+ 124 57.25806452
135
+ 125 57.60000000
136
+ 126 57.93650794
137
+ 127 57.48031496
138
+ 128 57.81250000
139
+ 129 57.36434109
140
+ 130 56.92307692
141
+ 131 56.48854962
142
+ 132 56.81818182
143
+ 133 56.39097744
144
+ 134 55.97014925
145
+ 135 56.29629630
146
+ 136 55.88235294
147
+ 137 56.20437956
148
+ 138 56.52173913
149
+ 139 56.83453237
150
+ 140 57.14285714
151
+ 141 57.44680851
152
+ 142 57.04225352
153
+ 143 57.34265734
154
+ 144 56.94444444
155
+ 145 57.24137931
156
+ 146 56.84931507
157
+ 147 57.14285714
158
+ 148 57.43243243
159
+ 149 57.71812081
160
+ 150 58.00000000
161
+ 151 58.27814570
162
+ 152 58.55263158
163
+ 153 58.82352941
164
+ 154 58.44155844
165
+ 155 58.06451613
166
+ 156 58.33333333
167
+ 157 58.59872611
168
+ 158 58.86075949
169
+ 159 59.11949686
170
+ 160 58.75000000
171
+ 161 58.38509317
172
+ 162 58.64197531
173
+ 163 58.28220859
174
+ 164 57.92682927
175
+ 165 58.18181818
176
+ 166 58.43373494
177
+ 167 58.08383234
178
+ 168 58.33333333
179
+ 169 58.57988166
180
+ 170 58.82352941
181
+ 171 58.47953216
182
+ 172 58.72093023
183
+ 173 58.95953757
184
+ 174 59.19540230
185
+ 175 58.85714286
186
+ 176 58.52272727
187
+ 177 58.75706215
188
+ 178 58.42696629
189
+ 179 58.65921788
190
+ 180 58.33333333
191
+ 181 58.56353591
192
+ 182 58.79120879
193
+ 183 58.46994536
194
+ 184 58.15217391
195
+ 185 58.37837838
196
+ 186 58.60215054
197
+ 187 58.82352941
198
+ 188 58.51063830
199
+ 189 58.20105820
200
+ 190 57.89473684
201
+ 191 58.11518325
202
+ 192 57.81250000
203
+ 193 57.51295337
204
+ 194 57.73195876
205
+ 195 57.94871795
206
+ 196 57.65306122
207
+ 197 57.86802030
208
+ 198 57.57575758
209
+ 199 57.28643216
210
+ 200 57.50000000
211
+ 201 57.71144279
212
+ 202 57.92079208
213
+ 203 57.63546798
214
+ 204 57.84313725
215
+ 205 57.56097561
216
+ 206 57.28155340
217
+ 207 57.48792271
218
+ 208 57.69230769
219
+ 209 57.89473684
220
+ 210 58.09523810
221
+ 211 58.29383886
222
+ 212 58.01886792
223
+ 213 57.74647887
224
+ 214 57.94392523
225
+ 215 58.13953488
226
+ 216 57.87037037
227
+ 217 57.60368664
228
+ 218 57.33944954
229
+ 219 57.53424658
230
+ 220 57.27272727
231
+ 221 57.01357466
232
+ 222 57.20720721
233
+ 223 56.95067265
234
+ 224 57.14285714
235
+ 225 56.88888889
236
+ 226 56.63716814
237
+ 227 56.82819383
238
+ 228 57.01754386
239
+ 229 57.20524017
240
+ 230 57.39130435
241
+ 231 57.14285714
242
+ 232 56.89655172
243
+ 233 57.08154506
244
+ 234 57.26495726
245
+ 235 57.02127660
246
+ 236 57.20338983
247
+ 237 57.38396624
248
+ 238 57.56302521
249
+ 239 57.74058577
250
+ 240 57.91666667
251
+ 241 58.09128631
252
+ 242 57.85123967
253
+ 243 58.02469136
254
+ 244 57.78688525
255
+ 245 57.95918367
256
+ 246 58.13008130
257
+ 247 58.29959514
258
+ 248 58.46774194
259
+ 249 58.23293173
260
+ 250 58.40000000
261
+ 251 58.56573705
262
+ 252 58.73015873
263
+ 253 58.89328063
264
+ 254 58.66141732
265
+ 255 58.82352941
266
+ 256 58.98437500
267
+ 257 59.14396887
268
+ 258 59.30232558
269
+ 259 59.07335907
270
+ 260 59.23076923
271
+ 261 59.38697318
272
+ 262 59.54198473
273
+ 263 59.69581749
274
+ 264 59.84848485
275
+ 265 60.00000000
276
+ 266 59.77443609
277
+ 267 59.92509363
278
+ 268 59.70149254
279
+ 269 59.85130112
280
+ 270 60.00000000
281
+ 271 59.77859779
282
+ 272 59.55882353
283
+ 273 59.34065934
284
+ 274 59.12408759
285
+ 275 58.90909091
286
+ 276 59.05797101
287
+ 277 59.20577617
288
+ 278 58.99280576
289
+ 279 59.13978495
290
+ 280 59.28571429
291
+ 281 59.43060498
292
+ 282 59.57446809
293
+ 283 59.36395760
294
+ 284 59.50704225
295
+ 285 59.64912281
296
+ 286 59.79020979
297
+ 287 59.58188153
298
+ 288 59.72222222
299
+ 289 59.51557093
300
+ 290 59.65517241
301
+ 291 59.79381443
302
+ 292 59.58904110
303
+ 293 59.72696246
304
+ 294 59.52380952
305
+ 295 59.66101695
306
+ 296 59.79729730
307
+ 297 59.93265993
308
+ 298 60.06711409
309
+ 299 60.20066890
310
+ 300 60.00000000
311
+ 301 59.80066445
312
+ 302 59.93377483
313
+ 303 60.06600660
314
+ 304 60.19736842
315
+ 305 60.32786885
316
+ 306 60.13071895
317
+ 307 59.93485342
318
+ 308 60.06493506
319
+ 309 60.19417476
320
+ 310 60.00000000
321
+ 311 60.12861736
322
+ 312 59.93589744
323
+ 313 60.06389776
324
+ 314 59.87261146
325
+ 315 60.00000000
326
+ 316 60.12658228
327
+ 317 60.25236593
328
+ 318 60.06289308
329
+ 319 59.87460815
330
+ 320 59.68750000
331
+ 321 59.50155763
332
+ 322 59.62732919
333
+ 323 59.44272446
334
+ 324 59.25925926
335
+ 325 59.07692308
336
+ 326 59.20245399
337
+ 327 59.02140673
338
+ 328 59.14634146
339
+ 329 59.27051672
340
+ 330 59.39393939
341
+ 331 59.51661631
342
+ 332 59.63855422
343
+ 333 59.75975976
344
+ 334 59.58083832
345
+ 335 59.40298507
346
+ 336 59.22619048
347
+ 337 59.05044510
348
+ 338 58.87573964
349
+ 339 58.99705015
350
+ 340 59.11764706
351
+ 341 59.23753666
352
+ 342 59.35672515
353
+ 343 59.47521866
354
+ 344 59.59302326
355
+ 345 59.42028986
356
+ 346 59.53757225
357
+ 347 59.36599424
358
+ 348 59.19540230
359
+ 349 59.31232092
360
+ 350 59.14285714
361
+ 351 59.25925926
362
+ 352 59.37500000
363
+ 353 59.49008499
364
+ 354 59.32203390
365
+ 355 59.15492958
366
+ 356 58.98876404
367
+ 357 58.82352941
368
+ 358 58.93854749
369
+ 359 59.05292479
370
+ 360 59.16666667
371
+ 361 59.27977839
372
+ 362 59.39226519
373
+ 363 59.50413223
374
+ 364 59.34065934
375
+ 365 59.45205479
376
+ 366 59.28961749
377
+ 367 59.40054496
378
+ 368 59.51086957
379
+ 369 59.34959350
380
+ 370 59.18918919
381
+ 371 59.02964960
382
+ 372 58.87096774
383
+ 373 58.71313673
384
+ 374 58.82352941
385
+ 375 58.93333333
386
+ 376 59.04255319
387
+ 377 58.88594164
388
+ 378 58.99470899
389
+ 379 59.10290237
390
+ 380 59.21052632
391
+ 381 59.05511811
392
+ 382 58.90052356
393
+ 383 58.74673629
394
+ 384 58.85416667
395
+ 385 58.96103896
396
+ 386 59.06735751
397
+ 387 58.91472868
398
+ 388 58.76288660
399
+ 389 58.61182519
400
+ 390 58.46153846
401
+ 391 58.31202046
402
+ 392 58.41836735
403
+ 393 58.26972010
404
+ 394 58.37563452
405
+ 395 58.48101266
406
+ 396 58.33333333
407
+ 397 58.43828715
408
+ 398 58.29145729
409
+ 399 58.14536341
410
+ 400 58.25000000
411
+ 401 58.35411471
412
+ 402 58.20895522
413
+ 403 58.31265509
414
+ 404 58.41584158
415
+ 405 58.51851852
416
+ 406 58.62068966
417
+ 407 58.72235872
418
+ 408 58.82352941
419
+ 409 58.67970660
420
+ 410 58.78048780
421
+ 411 58.88077859
422
+ 412 58.98058252
423
+ 413 58.83777240
424
+ 414 58.93719807
425
+ 415 58.79518072
426
+ 416 58.65384615
427
+ 417 58.51318945
428
+ 418 58.61244019
429
+ 419 58.47255370
430
+ 420 58.33333333
431
+ 421 58.19477435
432
+ 422 58.29383886
433
+ 423 58.39243499
434
+ 424 58.25471698
435
+ 425 58.11764706
436
+ 426 57.98122066
437
+ 427 58.07962529
438
+ 428 57.94392523
439
+ 429 58.04195804
440
+ 430 57.90697674
441
+ 431 58.00464037
442
+ 432 58.10185185
443
+ 433 57.96766744
444
+ 434 58.06451613
445
+ 435 58.16091954
446
+ 436 58.25688073
447
+ 437 58.12356979
448
+ 438 57.99086758
449
+ 439 58.08656036
450
+ 440 57.95454545
451
+ 441 57.82312925
452
+ 442 57.69230769
453
+ 443 57.56207675
454
+ 444 57.65765766
455
+ 445 57.75280899
456
+ 446 57.84753363
457
+ 447 57.94183445
458
+ 448 57.81250000
459
+ 449 57.90645880
460
+ 450 58.00000000
461
+ 451 58.09312639
462
+ 452 58.18584071
463
+ 453 58.05739514
464
+ 454 57.92951542
465
+ 455 58.02197802
466
+ 456 57.89473684
467
+ 457 57.98687090
468
+ 458 58.07860262
469
+ 459 58.16993464
470
+ 460 58.04347826
471
+ 461 57.91757050
472
+ 462 58.00865801
473
+ 463 57.88336933
474
+ 464 57.75862069
475
+ 465 57.63440860
476
+ 466 57.72532189
477
+ 467 57.81584582
478
+ 468 57.90598291
479
+ 469 57.78251599
480
+ 470 57.87234043
481
+ 471 57.96178344
482
+ 472 58.05084746
483
+ 473 57.92811839
484
+ 474 58.01687764
485
+ 475 57.89473684
486
+ 476 57.77310924
487
+ 477 57.86163522
488
+ 478 57.94979079
489
+ 479 58.03757829
490
+ 480 58.12500000
491
+ 481 58.00415800
492
+ 482 58.09128631
493
+ 483 58.17805383
494
+ 484 58.26446281
495
+ 485 58.35051546
496
+ 486 58.43621399
497
+ 487 58.31622177
498
+ 488 58.40163934
499
+ 489 58.28220859
500
+ 490 58.16326531
501
+ 491 58.04480652
502
+ 492 58.13008130
503
+ 493 58.01217039
504
+ 494 58.09716599
505
+ 495 57.97979798
506
+ 496 58.06451613
507
+ 497 58.14889336
508
+ 498 58.23293173
509
+ 499 58.31663327
510
+ 500 58.40000000
511
+ 501 58.28343313
512
+ 502 58.36653386
513
+ 503 58.44930417
514
+ 504 58.33333333
515
+ 505 58.41584158
516
+ 506 58.30039526
517
+ 507 58.18540434
518
+ 508 58.07086614
519
+ 509 57.95677800
520
+ 510 58.03921569
521
+ 511 57.92563601
522
+ 512 58.00781250
523
+ 513 57.89473684
524
+ 514 57.97665370
525
+ 515 58.05825243
526
+ 516 57.94573643
527
+ 517 58.02707930
528
+ 518 58.10810811
529
+ 519 57.99614644
530
+ 520 57.88461538
531
+ 521 57.96545106
532
+ 522 57.85440613
533
+ 523 57.74378585
534
+ 524 57.63358779
535
+ 525 57.52380952
536
+ 526 57.41444867
537
+ 527 57.30550285
538
+ 528 57.19696970
539
+ 529 57.08884688
540
+ 530 57.16981132
541
+ 531 57.25047081
542
+ 532 57.33082707
543
+ 533 57.41088180
544
+ 534 57.30337079
545
+ 535 57.19626168
546
+ 536 57.27611940
547
+ 537 57.35567970
548
+ 538 57.24907063
549
+ 539 57.32838590
550
+ 540 57.22222222
551
+ 541 57.30129390
552
+ 542 57.38007380
553
+ 543 57.45856354
554
+ 544 57.35294118
555
+ 545 57.43119266
556
+ 546 57.32600733
557
+ 547 57.22120658
558
+ 548 57.29927007
559
+ 549 57.37704918
560
+ 550 57.45454545
561
+ 551 57.35027223
562
+ 552 57.42753623
563
+ 553 57.50452080
564
+ 554 57.58122744
565
+ 555 57.65765766
566
+ 556 57.55395683
567
+ 557 57.63016158
568
+ 558 57.52688172
569
+ 559 57.60286225
570
+ 560 57.67857143
571
+ 561 57.75401070
572
+ 562 57.82918149
573
+ 563 57.90408526
574
+ 564 57.80141844
575
+ 565 57.69911504
576
+ 566 57.77385159
577
+ 567 57.67195767
578
+ 568 57.74647887
579
+ 569 57.64499121
580
+ 570 57.54385965
581
+ 571 57.61821366
582
+ 572 57.51748252
583
+ 573 57.41710297
584
+ 574 57.49128920
585
+ 575 57.56521739
586
+ 576 57.63888889
587
+ 577 57.71230503
588
+ 578 57.78546713
589
+ 579 57.85837651
590
+ 580 57.75862069
591
+ 581 57.65920826
592
+ 582 57.73195876
593
+ 583 57.80445969
594
+ 584 57.87671233
595
+ 585 57.94871795
596
+ 586 58.02047782
597
+ 587 57.92163543
598
+ 588 57.82312925
599
+ 589 57.89473684
600
+ 590 57.96610169
601
+ 591 58.03722504
602
+ 592 57.93918919
603
+ 593 58.01011804
604
+ 594 58.08080808
605
+ 595 57.98319328
606
+ 596 57.88590604
607
+ 597 57.78894472
608
+ 598 57.69230769
609
+ 599 57.76293823
610
+ 600 57.83333333
611
+ 601 57.90349418
612
+ 602 57.80730897
613
+ 603 57.71144279
614
+ 604 57.61589404
615
+ 605 57.52066116
616
+ 606 57.59075908
617
+ 607 57.66062603
618
+ 608 57.73026316
619
+ 609 57.63546798
620
+ 610 57.70491803
621
+ 611 57.61047463
622
+ 612 57.51633987
623
+ 613 57.42251223
624
+ 614 57.32899023
625
+ 615 57.23577236
626
+ 616 57.14285714
627
+ 617 57.21231767
628
+ 618 57.11974110
629
+ 619 57.02746365
630
+ 620 57.09677419
631
+ 621 57.16586151
632
+ 622 57.07395498
633
+ 623 57.14285714
634
+ 624 57.05128205
635
+ 625 56.96000000
636
+ 626 56.86900958
637
+ 627 56.93779904
638
+ 628 56.84713376
639
+ 629 56.91573927
640
+ 630 56.98412698
641
+ 631 56.89381933
642
+ 632 56.80379747
643
+ 633 56.71406003
644
+ 634 56.78233438
645
+ 635 56.85039370
646
+ 636 56.91823899
647
+ 637 56.82888540
648
+ 638 56.89655172
649
+ 639 56.80751174
650
+ 640 56.71875000
651
+ 641 56.63026521
652
+ 642 56.54205607
653
+ 643 56.60964230
654
+ 644 56.67701863
655
+ 645 56.58914729
656
+ 646 56.65634675
657
+ 647 56.72333849
658
+ 648 56.63580247
659
+ 649 56.54853621
660
+ 650 56.46153846
661
+ 651 56.52841782
662
+ 652 56.44171779
663
+ 653 56.50842266
664
+ 654 56.57492355
665
+ 655 56.48854962
666
+ 656 56.55487805
667
+ 657 56.62100457
668
+ 658 56.53495441
669
+ 659 56.44916540
670
+ 660 56.51515152
671
+ 661 56.42965204
672
+ 662 56.49546828
673
+ 663 56.56108597
674
+ 664 56.62650602
675
+ 665 56.69172932
676
+ 666 56.60660661
677
+ 667 56.67166417
678
+ 668 56.58682635
679
+ 669 56.50224215
680
+ 670 56.56716418
681
+ 671 56.63189270
682
+ 672 56.54761905
683
+ 673 56.46359584
684
+ 674 56.37982196
685
+ 675 56.44444444
686
+ 676 56.50887574
687
+ 677 56.57311669
688
+ 678 56.63716814
689
+ 679 56.70103093
690
+ 680 56.76470588
691
+ 681 56.68135095
692
+ 682 56.59824047
693
+ 683 56.51537335
694
+ 684 56.57894737
695
+ 685 56.64233577
696
+ 686 56.55976676
697
+ 687 56.62299854
698
+ 688 56.68604651
699
+ 689 56.74891147
700
+ 690 56.81159420
701
+ 691 56.87409551
702
+ 692 56.93641618
703
+ 693 56.99855700
704
+ 694 57.06051873
705
+ 695 56.97841727
706
+ 696 56.89655172
707
+ 697 56.81492109
708
+ 698 56.73352436
709
+ 699 56.65236052
710
+ 700 56.71428571
711
+ 701 56.63338088
712
+ 702 56.69515670
713
+ 703 56.75675676
714
+ 704 56.81818182
715
+ 705 56.87943262
716
+ 706 56.94050992
717
+ 707 57.00141443
718
+ 708 57.06214689
719
+ 709 57.12270804
720
+ 710 57.04225352
721
+ 711 57.10267229
722
+ 712 57.16292135
723
+ 713 57.22300140
724
+ 714 57.14285714
725
+ 715 57.06293706
726
+ 716 57.12290503
727
+ 717 57.04323570
728
+ 718 56.96378830
729
+ 719 56.88456189
730
+ 720 56.94444444
731
+ 721 56.86546463
732
+ 722 56.92520776
733
+ 723 56.98478562
734
+ 724 57.04419890
735
+ 725 57.10344828
736
+ 726 57.02479339
737
+ 727 56.94635488
738
+ 728 57.00549451
739
+ 729 57.06447188
740
+ 730 57.12328767
741
+ 731 57.18194254
742
+ 732 57.24043716
743
+ 733 57.29877217
744
+ 734 57.35694823
745
+ 735 57.27891156
746
+ 736 57.33695652
747
+ 737 57.25915875
748
+ 738 57.18157182
749
+ 739 57.10419486
750
+ 740 57.02702703
751
+ 741 57.08502024
752
+ 742 57.14285714
753
+ 743 57.20053836
754
+ 744 57.12365591
755
+ 745 57.18120805
756
+ 746 57.10455764
757
+ 747 57.02811245
758
+ 748 56.95187166
759
+ 749 57.00934579
760
+ 750 57.06666667
761
+
762
+ Final result: 57.0667 +/- 1.8086
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-iq4_nl.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 75.00000000
15
+ 5 80.00000000
16
+ 6 83.33333333
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 77.77777778
20
+ 10 80.00000000
21
+ 11 81.81818182
22
+ 12 75.00000000
23
+ 13 76.92307692
24
+ 14 78.57142857
25
+ 15 80.00000000
26
+ 16 81.25000000
27
+ 17 76.47058824
28
+ 18 72.22222222
29
+ 19 68.42105263
30
+ 20 70.00000000
31
+ 21 71.42857143
32
+ 22 72.72727273
33
+ 23 73.91304348
34
+ 24 70.83333333
35
+ 25 68.00000000
36
+ 26 65.38461538
37
+ 27 62.96296296
38
+ 28 64.28571429
39
+ 29 65.51724138
40
+ 30 63.33333333
41
+ 31 64.51612903
42
+ 32 62.50000000
43
+ 33 63.63636364
44
+ 34 64.70588235
45
+ 35 65.71428571
46
+ 36 63.88888889
47
+ 37 64.86486486
48
+ 38 65.78947368
49
+ 39 64.10256410
50
+ 40 65.00000000
51
+ 41 65.85365854
52
+ 42 64.28571429
53
+ 43 62.79069767
54
+ 44 63.63636364
55
+ 45 64.44444444
56
+ 46 63.04347826
57
+ 47 61.70212766
58
+ 48 62.50000000
59
+ 49 63.26530612
60
+ 50 62.00000000
61
+ 51 62.74509804
62
+ 52 61.53846154
63
+ 53 60.37735849
64
+ 54 59.25925926
65
+ 55 58.18181818
66
+ 56 57.14285714
67
+ 57 57.89473684
68
+ 58 56.89655172
69
+ 59 57.62711864
70
+ 60 58.33333333
71
+ 61 57.37704918
72
+ 62 56.45161290
73
+ 63 57.14285714
74
+ 64 57.81250000
75
+ 65 56.92307692
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 57.97101449
80
+ 70 58.57142857
81
+ 71 57.74647887
82
+ 72 58.33333333
83
+ 73 58.90410959
84
+ 74 59.45945946
85
+ 75 58.66666667
86
+ 76 59.21052632
87
+ 77 59.74025974
88
+ 78 60.25641026
89
+ 79 60.75949367
90
+ 80 61.25000000
91
+ 81 61.72839506
92
+ 82 62.19512195
93
+ 83 62.65060241
94
+ 84 63.09523810
95
+ 85 62.35294118
96
+ 86 62.79069767
97
+ 87 63.21839080
98
+ 88 63.63636364
99
+ 89 62.92134831
100
+ 90 63.33333333
101
+ 91 63.73626374
102
+ 92 63.04347826
103
+ 93 63.44086022
104
+ 94 63.82978723
105
+ 95 64.21052632
106
+ 96 64.58333333
107
+ 97 64.94845361
108
+ 98 65.30612245
109
+ 99 65.65656566
110
+ 100 65.00000000
111
+ 101 65.34653465
112
+ 102 64.70588235
113
+ 103 64.07766990
114
+ 104 64.42307692
115
+ 105 64.76190476
116
+ 106 65.09433962
117
+ 107 65.42056075
118
+ 108 65.74074074
119
+ 109 65.13761468
120
+ 110 65.45454545
121
+ 111 65.76576577
122
+ 112 66.07142857
123
+ 113 66.37168142
124
+ 114 66.66666667
125
+ 115 66.95652174
126
+ 116 66.37931034
127
+ 117 65.81196581
128
+ 118 66.10169492
129
+ 119 66.38655462
130
+ 120 65.83333333
131
+ 121 65.28925620
132
+ 122 65.57377049
133
+ 123 65.85365854
134
+ 124 65.32258065
135
+ 125 65.60000000
136
+ 126 65.87301587
137
+ 127 66.14173228
138
+ 128 65.62500000
139
+ 129 65.89147287
140
+ 130 65.38461538
141
+ 131 65.64885496
142
+ 132 65.15151515
143
+ 133 65.41353383
144
+ 134 65.67164179
145
+ 135 65.92592593
146
+ 136 66.17647059
147
+ 137 66.42335766
148
+ 138 66.66666667
149
+ 139 66.90647482
150
+ 140 67.14285714
151
+ 141 67.37588652
152
+ 142 67.60563380
153
+ 143 67.83216783
154
+ 144 68.05555556
155
+ 145 68.27586207
156
+ 146 68.49315068
157
+ 147 68.70748299
158
+ 148 68.91891892
159
+ 149 69.12751678
160
+ 150 69.33333333
161
+ 151 68.87417219
162
+ 152 68.42105263
163
+ 153 68.62745098
164
+ 154 68.83116883
165
+ 155 69.03225806
166
+ 156 68.58974359
167
+ 157 68.15286624
168
+ 158 68.35443038
169
+ 159 67.92452830
170
+ 160 68.12500000
171
+ 161 68.32298137
172
+ 162 68.51851852
173
+ 163 68.71165644
174
+ 164 68.90243902
175
+ 165 69.09090909
176
+ 166 69.27710843
177
+ 167 69.46107784
178
+ 168 69.64285714
179
+ 169 69.23076923
180
+ 170 69.41176471
181
+ 171 69.59064327
182
+ 172 69.18604651
183
+ 173 68.78612717
184
+ 174 68.39080460
185
+ 175 68.57142857
186
+ 176 68.75000000
187
+ 177 68.92655367
188
+ 178 69.10112360
189
+ 179 68.71508380
190
+ 180 68.88888889
191
+ 181 68.50828729
192
+ 182 68.68131868
193
+ 183 68.85245902
194
+ 184 68.47826087
195
+ 185 68.64864865
196
+ 186 68.27956989
197
+ 187 67.91443850
198
+ 188 67.55319149
199
+ 189 67.19576720
200
+ 190 67.36842105
201
+ 191 67.53926702
202
+ 192 67.70833333
203
+ 193 67.35751295
204
+ 194 67.52577320
205
+ 195 67.17948718
206
+ 196 66.83673469
207
+ 197 66.49746193
208
+ 198 66.16161616
209
+ 199 66.33165829
210
+ 200 66.50000000
211
+ 201 66.66666667
212
+ 202 66.33663366
213
+ 203 66.00985222
214
+ 204 66.17647059
215
+ 205 65.85365854
216
+ 206 65.53398058
217
+ 207 65.70048309
218
+ 208 65.38461538
219
+ 209 65.55023923
220
+ 210 65.71428571
221
+ 211 65.87677725
222
+ 212 66.03773585
223
+ 213 66.19718310
224
+ 214 66.35514019
225
+ 215 66.04651163
226
+ 216 66.20370370
227
+ 217 65.89861751
228
+ 218 65.59633028
229
+ 219 65.75342466
230
+ 220 65.90909091
231
+ 221 65.61085973
232
+ 222 65.76576577
233
+ 223 65.91928251
234
+ 224 66.07142857
235
+ 225 66.22222222
236
+ 226 66.37168142
237
+ 227 66.07929515
238
+ 228 66.22807018
239
+ 229 66.37554585
240
+ 230 66.08695652
241
+ 231 65.80086580
242
+ 232 65.51724138
243
+ 233 65.23605150
244
+ 234 65.38461538
245
+ 235 65.10638298
246
+ 236 64.83050847
247
+ 237 64.97890295
248
+ 238 65.12605042
249
+ 239 65.27196653
250
+ 240 65.41666667
251
+ 241 65.56016598
252
+ 242 65.70247934
253
+ 243 65.84362140
254
+ 244 65.57377049
255
+ 245 65.30612245
256
+ 246 65.44715447
257
+ 247 65.18218623
258
+ 248 65.32258065
259
+ 249 65.46184739
260
+ 250 65.60000000
261
+ 251 65.73705179
262
+ 252 65.87301587
263
+ 253 65.61264822
264
+ 254 65.35433071
265
+ 255 65.49019608
266
+ 256 65.62500000
267
+ 257 65.36964981
268
+ 258 65.50387597
269
+ 259 65.63706564
270
+ 260 65.76923077
271
+ 261 65.90038314
272
+ 262 65.64885496
273
+ 263 65.39923954
274
+ 264 65.53030303
275
+ 265 65.66037736
276
+ 266 65.41353383
277
+ 267 65.54307116
278
+ 268 65.67164179
279
+ 269 65.79925651
280
+ 270 65.92592593
281
+ 271 65.68265683
282
+ 272 65.80882353
283
+ 273 65.56776557
284
+ 274 65.69343066
285
+ 275 65.45454545
286
+ 276 65.57971014
287
+ 277 65.70397112
288
+ 278 65.82733813
289
+ 279 65.94982079
290
+ 280 66.07142857
291
+ 281 66.19217082
292
+ 282 66.31205674
293
+ 283 66.07773852
294
+ 284 66.19718310
295
+ 285 65.96491228
296
+ 286 66.08391608
297
+ 287 66.20209059
298
+ 288 66.31944444
299
+ 289 66.43598616
300
+ 290 66.55172414
301
+ 291 66.66666667
302
+ 292 66.43835616
303
+ 293 66.55290102
304
+ 294 66.66666667
305
+ 295 66.77966102
306
+ 296 66.89189189
307
+ 297 67.00336700
308
+ 298 67.11409396
309
+ 299 66.88963211
310
+ 300 66.66666667
311
+ 301 66.44518272
312
+ 302 66.55629139
313
+ 303 66.33663366
314
+ 304 66.11842105
315
+ 305 66.22950820
316
+ 306 66.33986928
317
+ 307 66.12377850
318
+ 308 66.23376623
319
+ 309 66.34304207
320
+ 310 66.45161290
321
+ 311 66.55948553
322
+ 312 66.66666667
323
+ 313 66.77316294
324
+ 314 66.56050955
325
+ 315 66.66666667
326
+ 316 66.77215190
327
+ 317 66.87697161
328
+ 318 66.66666667
329
+ 319 66.45768025
330
+ 320 66.56250000
331
+ 321 66.35514019
332
+ 322 66.14906832
333
+ 323 66.25386997
334
+ 324 66.35802469
335
+ 325 66.46153846
336
+ 326 66.25766871
337
+ 327 66.05504587
338
+ 328 66.15853659
339
+ 329 66.26139818
340
+ 330 66.36363636
341
+ 331 66.46525680
342
+ 332 66.26506024
343
+ 333 66.36636637
344
+ 334 66.16766467
345
+ 335 66.26865672
346
+ 336 66.36904762
347
+ 337 66.17210682
348
+ 338 65.97633136
349
+ 339 66.07669617
350
+ 340 66.17647059
351
+ 341 66.27565982
352
+ 342 66.37426901
353
+ 343 66.47230321
354
+ 344 66.27906977
355
+ 345 66.37681159
356
+ 346 66.18497110
357
+ 347 66.28242075
358
+ 348 66.09195402
359
+ 349 65.90257880
360
+ 350 66.00000000
361
+ 351 66.09686610
362
+ 352 66.19318182
363
+ 353 66.00566572
364
+ 354 66.10169492
365
+ 355 66.19718310
366
+ 356 66.29213483
367
+ 357 66.10644258
368
+ 358 66.20111732
369
+ 359 66.29526462
370
+ 360 66.11111111
371
+ 361 66.20498615
372
+ 362 66.29834254
373
+ 363 66.39118457
374
+ 364 66.48351648
375
+ 365 66.57534247
376
+ 366 66.66666667
377
+ 367 66.75749319
378
+ 368 66.84782609
379
+ 369 66.93766938
380
+ 370 66.75675676
381
+ 371 66.84636119
382
+ 372 66.93548387
383
+ 373 66.75603217
384
+ 374 66.84491979
385
+ 375 66.93333333
386
+ 376 67.02127660
387
+ 377 66.84350133
388
+ 378 66.93121693
389
+ 379 67.01846966
390
+ 380 67.10526316
391
+ 381 67.19160105
392
+ 382 67.01570681
393
+ 383 67.10182768
394
+ 384 67.18750000
395
+ 385 67.27272727
396
+ 386 67.35751295
397
+ 387 67.18346253
398
+ 388 67.01030928
399
+ 389 67.09511568
400
+ 390 67.17948718
401
+ 391 67.00767263
402
+ 392 67.09183673
403
+ 393 67.17557252
404
+ 394 67.25888325
405
+ 395 67.34177215
406
+ 396 67.42424242
407
+ 397 67.50629723
408
+ 398 67.58793970
409
+ 399 67.66917293
410
+ 400 67.75000000
411
+ 401 67.83042394
412
+ 402 67.91044776
413
+ 403 67.74193548
414
+ 404 67.82178218
415
+ 405 67.65432099
416
+ 406 67.73399015
417
+ 407 67.81326781
418
+ 408 67.89215686
419
+ 409 67.97066015
420
+ 410 67.80487805
421
+ 411 67.63990268
422
+ 412 67.47572816
423
+ 413 67.55447942
424
+ 414 67.63285024
425
+ 415 67.46987952
426
+ 416 67.54807692
427
+ 417 67.62589928
428
+ 418 67.70334928
429
+ 419 67.78042959
430
+ 420 67.85714286
431
+ 421 67.69596200
432
+ 422 67.77251185
433
+ 423 67.84869976
434
+ 424 67.92452830
435
+ 425 68.00000000
436
+ 426 68.07511737
437
+ 427 67.91569087
438
+ 428 67.99065421
439
+ 429 67.83216783
440
+ 430 67.90697674
441
+ 431 67.98143852
442
+ 432 68.05555556
443
+ 433 68.12933025
444
+ 434 68.20276498
445
+ 435 68.04597701
446
+ 436 68.11926606
447
+ 437 67.96338673
448
+ 438 67.80821918
449
+ 439 67.88154897
450
+ 440 67.95454545
451
+ 441 68.02721088
452
+ 442 68.09954751
453
+ 443 68.17155756
454
+ 444 68.24324324
455
+ 445 68.31460674
456
+ 446 68.38565022
457
+ 447 68.45637584
458
+ 448 68.52678571
459
+ 449 68.59688196
460
+ 450 68.66666667
461
+ 451 68.51441242
462
+ 452 68.58407080
463
+ 453 68.43267108
464
+ 454 68.50220264
465
+ 455 68.35164835
466
+ 456 68.42105263
467
+ 457 68.49015317
468
+ 458 68.55895197
469
+ 459 68.62745098
470
+ 460 68.69565217
471
+ 461 68.76355748
472
+ 462 68.83116883
473
+ 463 68.89848812
474
+ 464 68.96551724
475
+ 465 68.81720430
476
+ 466 68.88412017
477
+ 467 68.95074946
478
+ 468 68.80341880
479
+ 469 68.65671642
480
+ 470 68.51063830
481
+ 471 68.36518047
482
+ 472 68.43220339
483
+ 473 68.49894292
484
+ 474 68.56540084
485
+ 475 68.63157895
486
+ 476 68.48739496
487
+ 477 68.34381551
488
+ 478 68.41004184
489
+ 479 68.47599165
490
+ 480 68.54166667
491
+ 481 68.60706861
492
+ 482 68.46473029
493
+ 483 68.53002070
494
+ 484 68.59504132
495
+ 485 68.45360825
496
+ 486 68.51851852
497
+ 487 68.58316222
498
+ 488 68.44262295
499
+ 489 68.50715746
500
+ 490 68.57142857
501
+ 491 68.63543788
502
+ 492 68.69918699
503
+ 493 68.76267748
504
+ 494 68.82591093
505
+ 495 68.88888889
506
+ 496 68.95161290
507
+ 497 69.01408451
508
+ 498 69.07630522
509
+ 499 68.93787575
510
+ 500 69.00000000
511
+ 501 69.06187625
512
+ 502 69.12350598
513
+ 503 69.18489066
514
+ 504 69.24603175
515
+ 505 69.30693069
516
+ 506 69.36758893
517
+ 507 69.42800789
518
+ 508 69.48818898
519
+ 509 69.54813360
520
+ 510 69.60784314
521
+ 511 69.66731898
522
+ 512 69.53125000
523
+ 513 69.39571150
524
+ 514 69.45525292
525
+ 515 69.51456311
526
+ 516 69.37984496
527
+ 517 69.24564797
528
+ 518 69.30501931
529
+ 519 69.17148362
530
+ 520 69.23076923
531
+ 521 69.09788868
532
+ 522 68.96551724
533
+ 523 69.02485660
534
+ 524 69.08396947
535
+ 525 69.14285714
536
+ 526 69.01140684
537
+ 527 68.88045541
538
+ 528 68.93939394
539
+ 529 68.99810964
540
+ 530 69.05660377
541
+ 531 69.11487759
542
+ 532 69.17293233
543
+ 533 69.04315197
544
+ 534 69.10112360
545
+ 535 69.15887850
546
+ 536 69.21641791
547
+ 537 69.27374302
548
+ 538 69.33085502
549
+ 539 69.20222635
550
+ 540 69.25925926
551
+ 541 69.31608133
552
+ 542 69.37269373
553
+ 543 69.42909761
554
+ 544 69.48529412
555
+ 545 69.54128440
556
+ 546 69.59706960
557
+ 547 69.65265082
558
+ 548 69.70802920
559
+ 549 69.76320583
560
+ 550 69.63636364
561
+ 551 69.69147005
562
+ 552 69.56521739
563
+ 553 69.62025316
564
+ 554 69.67509025
565
+ 555 69.54954955
566
+ 556 69.60431655
567
+ 557 69.47935368
568
+ 558 69.35483871
569
+ 559 69.23076923
570
+ 560 69.28571429
571
+ 561 69.34046346
572
+ 562 69.21708185
573
+ 563 69.27175844
574
+ 564 69.32624113
575
+ 565 69.20353982
576
+ 566 69.08127208
577
+ 567 69.13580247
578
+ 568 69.19014085
579
+ 569 69.06854130
580
+ 570 69.12280702
581
+ 571 69.17688266
582
+ 572 69.23076923
583
+ 573 69.28446771
584
+ 574 69.33797909
585
+ 575 69.39130435
586
+ 576 69.44444444
587
+ 577 69.32409012
588
+ 578 69.37716263
589
+ 579 69.43005181
590
+ 580 69.31034483
591
+ 581 69.19104991
592
+ 582 69.24398625
593
+ 583 69.29674099
594
+ 584 69.34931507
595
+ 585 69.40170940
596
+ 586 69.45392491
597
+ 587 69.50596252
598
+ 588 69.38775510
599
+ 589 69.26994907
600
+ 590 69.15254237
601
+ 591 69.20473773
602
+ 592 69.25675676
603
+ 593 69.13996627
604
+ 594 69.02356902
605
+ 595 69.07563025
606
+ 596 69.12751678
607
+ 597 69.17922948
608
+ 598 69.23076923
609
+ 599 69.11519199
610
+ 600 69.16666667
611
+ 601 69.21797005
612
+ 602 69.26910299
613
+ 603 69.15422886
614
+ 604 69.20529801
615
+ 605 69.25619835
616
+ 606 69.14191419
617
+ 607 69.19275124
618
+ 608 69.07894737
619
+ 609 69.12972085
620
+ 610 69.18032787
621
+ 611 69.23076923
622
+ 612 69.28104575
623
+ 613 69.33115824
624
+ 614 69.21824104
625
+ 615 69.26829268
626
+ 616 69.15584416
627
+ 617 69.04376013
628
+ 618 69.09385113
629
+ 619 68.98222940
630
+ 620 69.03225806
631
+ 621 69.08212560
632
+ 622 69.13183280
633
+ 623 69.18138042
634
+ 624 69.23076923
635
+ 625 69.28000000
636
+ 626 69.16932907
637
+ 627 69.21850080
638
+ 628 69.26751592
639
+ 629 69.15739269
640
+ 630 69.04761905
641
+ 631 69.09667195
642
+ 632 68.98734177
643
+ 633 69.03633491
644
+ 634 69.08517350
645
+ 635 69.13385827
646
+ 636 69.18238994
647
+ 637 69.23076923
648
+ 638 69.27899687
649
+ 639 69.32707355
650
+ 640 69.21875000
651
+ 641 69.26677067
652
+ 642 69.15887850
653
+ 643 69.20684292
654
+ 644 69.25465839
655
+ 645 69.14728682
656
+ 646 69.19504644
657
+ 647 69.24265842
658
+ 648 69.29012346
659
+ 649 69.18335901
660
+ 650 69.23076923
661
+ 651 69.27803379
662
+ 652 69.32515337
663
+ 653 69.37212864
664
+ 654 69.41896024
665
+ 655 69.46564885
666
+ 656 69.51219512
667
+ 657 69.55859970
668
+ 658 69.45288754
669
+ 659 69.34749621
670
+ 660 69.39393939
671
+ 661 69.44024206
672
+ 662 69.48640483
673
+ 663 69.53242836
674
+ 664 69.57831325
675
+ 665 69.62406015
676
+ 666 69.66966967
677
+ 667 69.71514243
678
+ 668 69.76047904
679
+ 669 69.80568012
680
+ 670 69.85074627
681
+ 671 69.89567809
682
+ 672 69.94047619
683
+ 673 69.98514116
684
+ 674 70.02967359
685
+ 675 70.07407407
686
+ 676 69.97041420
687
+ 677 70.01477105
688
+ 678 70.05899705
689
+ 679 70.10309278
690
+ 680 70.14705882
691
+ 681 70.04405286
692
+ 682 70.08797654
693
+ 683 70.13177160
694
+ 684 70.17543860
695
+ 685 70.07299270
696
+ 686 69.97084548
697
+ 687 70.01455604
698
+ 688 70.05813953
699
+ 689 70.10159652
700
+ 690 70.14492754
701
+ 691 70.18813314
702
+ 692 70.23121387
703
+ 693 70.12987013
704
+ 694 70.17291066
705
+ 695 70.21582734
706
+ 696 70.25862069
707
+ 697 70.30129125
708
+ 698 70.34383954
709
+ 699 70.38626609
710
+ 700 70.42857143
711
+ 701 70.47075606
712
+ 702 70.51282051
713
+ 703 70.55476529
714
+ 704 70.59659091
715
+ 705 70.63829787
716
+ 706 70.67988669
717
+ 707 70.72135785
718
+ 708 70.76271186
719
+ 709 70.80394922
720
+ 710 70.84507042
721
+ 711 70.74542897
722
+ 712 70.78651685
723
+ 713 70.82748948
724
+ 714 70.86834734
725
+ 715 70.76923077
726
+ 716 70.81005587
727
+ 717 70.85076709
728
+ 718 70.89136490
729
+ 719 70.79276773
730
+ 720 70.83333333
731
+ 721 70.73509015
732
+ 722 70.77562327
733
+ 723 70.81604426
734
+ 724 70.85635359
735
+ 725 70.89655172
736
+ 726 70.93663912
737
+ 727 70.97661623
738
+ 728 71.01648352
739
+ 729 71.05624143
740
+ 730 70.95890411
741
+ 731 70.99863201
742
+ 732 70.90163934
743
+ 733 70.94133697
744
+ 734 70.98092643
745
+ 735 71.02040816
746
+ 736 71.05978261
747
+ 737 70.96336499
748
+ 738 70.86720867
749
+ 739 70.77131258
750
+ 740 70.81081081
751
+ 741 70.85020243
752
+ 742 70.88948787
753
+ 743 70.92866756
754
+ 744 70.96774194
755
+ 745 70.87248322
756
+ 746 70.91152815
757
+ 747 70.81659973
758
+ 748 70.85561497
759
+ 749 70.89452603
760
+ 750 70.93333333
scores/Hammer2.1-7b-iq4_nl.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 50.00000000
15
+ 5 40.00000000
16
+ 6 33.33333333
17
+ 7 42.85714286
18
+ 8 50.00000000
19
+ 9 44.44444444
20
+ 10 40.00000000
21
+ 11 36.36363636
22
+ 12 33.33333333
23
+ 13 38.46153846
24
+ 14 35.71428571
25
+ 15 33.33333333
26
+ 16 37.50000000
27
+ 17 41.17647059
28
+ 18 38.88888889
29
+ 19 42.10526316
30
+ 20 45.00000000
31
+ 21 42.85714286
32
+ 22 40.90909091
33
+ 23 39.13043478
34
+ 24 37.50000000
35
+ 25 40.00000000
36
+ 26 38.46153846
37
+ 27 40.74074074
38
+ 28 39.28571429
39
+ 29 41.37931034
40
+ 30 43.33333333
41
+ 31 45.16129032
42
+ 32 46.87500000
43
+ 33 45.45454545
44
+ 34 47.05882353
45
+ 35 48.57142857
46
+ 36 47.22222222
47
+ 37 45.94594595
48
+ 38 47.36842105
49
+ 39 46.15384615
50
+ 40 45.00000000
51
+ 41 46.34146341
52
+ 42 45.23809524
53
+ 43 46.51162791
54
+ 44 45.45454545
55
+ 45 46.66666667
56
+ 46 45.65217391
57
+ 47 44.68085106
58
+ 48 45.83333333
59
+ 49 46.93877551
60
+ 50 46.00000000
61
+ 51 45.09803922
62
+ 52 44.23076923
63
+ 53 43.39622642
64
+ 54 42.59259259
65
+ 55 43.63636364
66
+ 56 44.64285714
67
+ 57 43.85964912
68
+ 58 44.82758621
69
+ 59 45.76271186
70
+ 60 45.00000000
71
+ 61 44.26229508
72
+ 62 43.54838710
73
+ 63 42.85714286
74
+ 64 42.18750000
75
+ 65 41.53846154
76
+ 66 42.42424242
77
+ 67 41.79104478
78
+ 68 41.17647059
79
+ 69 40.57971014
80
+ 70 41.42857143
81
+ 71 42.25352113
82
+ 72 41.66666667
83
+ 73 41.09589041
84
+ 74 41.89189189
85
+ 75 42.66666667
86
+ 76 43.42105263
87
+ 77 44.15584416
88
+ 78 44.87179487
89
+ 79 44.30379747
90
+ 80 45.00000000
91
+ 81 45.67901235
92
+ 82 45.12195122
93
+ 83 45.78313253
94
+ 84 46.42857143
95
+ 85 47.05882353
96
+ 86 46.51162791
97
+ 87 47.12643678
98
+ 88 47.72727273
99
+ 89 47.19101124
100
+ 90 46.66666667
101
+ 91 46.15384615
102
+ 92 45.65217391
103
+ 93 45.16129032
104
+ 94 45.74468085
105
+ 95 45.26315789
106
+ 96 45.83333333
107
+ 97 45.36082474
108
+ 98 44.89795918
109
+ 99 45.45454545
110
+ 100 45.00000000
111
+ 101 45.54455446
112
+ 102 45.09803922
113
+ 103 44.66019417
114
+ 104 45.19230769
115
+ 105 45.71428571
116
+ 106 46.22641509
117
+ 107 45.79439252
118
+ 108 46.29629630
119
+ 109 46.78899083
120
+ 110 46.36363636
121
+ 111 45.94594595
122
+ 112 46.42857143
123
+ 113 46.01769912
124
+ 114 46.49122807
125
+ 115 46.95652174
126
+ 116 46.55172414
127
+ 117 46.15384615
128
+ 118 45.76271186
129
+ 119 45.37815126
130
+ 120 45.00000000
131
+ 121 45.45454545
132
+ 122 45.90163934
133
+ 123 45.52845528
134
+ 124 45.16129032
135
+ 125 45.60000000
136
+ 126 45.23809524
137
+ 127 44.88188976
138
+ 128 44.53125000
139
+ 129 44.18604651
140
+ 130 43.84615385
141
+ 131 44.27480916
142
+ 132 43.93939394
143
+ 133 43.60902256
144
+ 134 43.28358209
145
+ 135 42.96296296
146
+ 136 43.38235294
147
+ 137 43.79562044
148
+ 138 43.47826087
149
+ 139 43.88489209
150
+ 140 44.28571429
151
+ 141 43.97163121
152
+ 142 44.36619718
153
+ 143 44.75524476
154
+ 144 44.44444444
155
+ 145 44.13793103
156
+ 146 44.52054795
157
+ 147 44.21768707
158
+ 148 44.59459459
159
+ 149 44.96644295
160
+ 150 44.66666667
161
+ 151 44.37086093
162
+ 152 44.73684211
163
+ 153 44.44444444
164
+ 154 44.15584416
165
+ 155 43.87096774
166
+ 156 43.58974359
167
+ 157 43.94904459
168
+ 158 44.30379747
169
+ 159 44.02515723
170
+ 160 44.37500000
171
+ 161 44.09937888
172
+ 162 43.82716049
173
+ 163 43.55828221
174
+ 164 43.29268293
175
+ 165 43.63636364
176
+ 166 43.37349398
177
+ 167 43.71257485
178
+ 168 43.45238095
179
+ 169 43.78698225
180
+ 170 43.52941176
181
+ 171 43.85964912
182
+ 172 43.60465116
183
+ 173 43.35260116
184
+ 174 43.10344828
185
+ 175 42.85714286
186
+ 176 42.61363636
187
+ 177 42.37288136
188
+ 178 42.13483146
189
+ 179 42.45810056
190
+ 180 42.22222222
191
+ 181 41.98895028
192
+ 182 41.75824176
193
+ 183 42.07650273
194
+ 184 41.84782609
195
+ 185 42.16216216
196
+ 186 42.47311828
197
+ 187 42.24598930
198
+ 188 42.02127660
199
+ 189 42.32804233
200
+ 190 42.10526316
201
+ 191 41.88481675
202
+ 192 41.66666667
203
+ 193 41.45077720
204
+ 194 41.23711340
205
+ 195 41.53846154
206
+ 196 41.32653061
207
+ 197 41.62436548
208
+ 198 41.41414141
209
+ 199 41.20603015
210
+ 200 41.00000000
211
+ 201 40.79601990
212
+ 202 40.59405941
213
+ 203 40.39408867
214
+ 204 40.19607843
215
+ 205 40.48780488
216
+ 206 40.29126214
217
+ 207 40.57971014
218
+ 208 40.38461538
219
+ 209 40.19138756
220
+ 210 40.00000000
221
+ 211 40.28436019
222
+ 212 40.09433962
223
+ 213 39.90610329
224
+ 214 39.71962617
225
+ 215 40.00000000
226
+ 216 40.27777778
227
+ 217 40.09216590
228
+ 218 40.36697248
229
+ 219 40.63926941
230
+ 220 40.90909091
231
+ 221 41.17647059
232
+ 222 40.99099099
233
+ 223 41.25560538
234
+ 224 41.51785714
235
+ 225 41.77777778
236
+ 226 42.03539823
237
+ 227 41.85022026
238
+ 228 42.10526316
239
+ 229 41.92139738
240
+ 230 41.73913043
241
+ 231 41.55844156
242
+ 232 41.81034483
243
+ 233 41.63090129
244
+ 234 41.88034188
245
+ 235 41.70212766
246
+ 236 41.52542373
247
+ 237 41.35021097
248
+ 238 41.17647059
249
+ 239 41.00418410
250
+ 240 41.25000000
251
+ 241 41.07883817
252
+ 242 41.32231405
253
+ 243 41.56378601
254
+ 244 41.39344262
255
+ 245 41.63265306
256
+ 246 41.46341463
257
+ 247 41.70040486
258
+ 248 41.93548387
259
+ 249 41.76706827
260
+ 250 41.60000000
261
+ 251 41.43426295
262
+ 252 41.26984127
263
+ 253 41.10671937
264
+ 254 40.94488189
265
+ 255 40.78431373
266
+ 256 40.62500000
267
+ 257 40.85603113
268
+ 258 40.69767442
269
+ 259 40.54054054
270
+ 260 40.38461538
271
+ 261 40.22988506
272
+ 262 40.07633588
273
+ 263 39.92395437
274
+ 264 40.15151515
275
+ 265 40.37735849
276
+ 266 40.22556391
277
+ 267 40.07490637
278
+ 268 39.92537313
279
+ 269 40.14869888
280
+ 270 40.37037037
281
+ 271 40.22140221
282
+ 272 40.44117647
283
+ 273 40.29304029
284
+ 274 40.14598540
285
+ 275 40.00000000
286
+ 276 40.21739130
287
+ 277 40.43321300
288
+ 278 40.64748201
289
+ 279 40.50179211
290
+ 280 40.71428571
291
+ 281 40.56939502
292
+ 282 40.42553191
293
+ 283 40.28268551
294
+ 284 40.14084507
295
+ 285 40.00000000
296
+ 286 39.86013986
297
+ 287 40.06968641
298
+ 288 39.93055556
299
+ 289 39.79238754
300
+ 290 40.00000000
301
+ 291 40.20618557
302
+ 292 40.41095890
303
+ 293 40.61433447
304
+ 294 40.47619048
305
+ 295 40.33898305
306
+ 296 40.20270270
307
+ 297 40.06734007
308
+ 298 40.26845638
309
+ 299 40.46822742
310
+ 300 40.66666667
311
+ 301 40.53156146
312
+ 302 40.72847682
313
+ 303 40.59405941
314
+ 304 40.46052632
315
+ 305 40.65573770
316
+ 306 40.52287582
317
+ 307 40.71661238
318
+ 308 40.58441558
319
+ 309 40.45307443
320
+ 310 40.64516129
321
+ 311 40.51446945
322
+ 312 40.70512821
323
+ 313 40.89456869
324
+ 314 41.08280255
325
+ 315 41.26984127
326
+ 316 41.13924051
327
+ 317 41.32492114
328
+ 318 41.19496855
329
+ 319 41.37931034
330
+ 320 41.25000000
331
+ 321 41.12149533
332
+ 322 40.99378882
333
+ 323 40.86687307
334
+ 324 40.74074074
335
+ 325 40.92307692
336
+ 326 40.79754601
337
+ 327 40.97859327
338
+ 328 40.85365854
339
+ 329 40.72948328
340
+ 330 40.60606061
341
+ 331 40.48338369
342
+ 332 40.66265060
343
+ 333 40.54054054
344
+ 334 40.41916168
345
+ 335 40.59701493
346
+ 336 40.77380952
347
+ 337 40.65281899
348
+ 338 40.53254438
349
+ 339 40.41297935
350
+ 340 40.58823529
351
+ 341 40.76246334
352
+ 342 40.93567251
353
+ 343 40.81632653
354
+ 344 40.69767442
355
+ 345 40.57971014
356
+ 346 40.75144509
357
+ 347 40.92219020
358
+ 348 40.80459770
359
+ 349 40.97421203
360
+ 350 41.14285714
361
+ 351 41.02564103
362
+ 352 40.90909091
363
+ 353 41.07648725
364
+ 354 41.24293785
365
+ 355 41.12676056
366
+ 356 41.29213483
367
+ 357 41.45658263
368
+ 358 41.62011173
369
+ 359 41.50417827
370
+ 360 41.66666667
371
+ 361 41.55124654
372
+ 362 41.71270718
373
+ 363 41.87327824
374
+ 364 41.75824176
375
+ 365 41.64383562
376
+ 366 41.80327869
377
+ 367 41.68937330
378
+ 368 41.57608696
379
+ 369 41.46341463
380
+ 370 41.62162162
381
+ 371 41.77897574
382
+ 372 41.66666667
383
+ 373 41.55495979
384
+ 374 41.71122995
385
+ 375 41.86666667
386
+ 376 41.75531915
387
+ 377 41.64456233
388
+ 378 41.79894180
389
+ 379 41.68865435
390
+ 380 41.57894737
391
+ 381 41.73228346
392
+ 382 41.88481675
393
+ 383 42.03655352
394
+ 384 41.92708333
395
+ 385 42.07792208
396
+ 386 41.96891192
397
+ 387 41.86046512
398
+ 388 41.75257732
399
+ 389 41.90231362
400
+ 390 42.05128205
401
+ 391 42.19948849
402
+ 392 42.34693878
403
+ 393 42.23918575
404
+ 394 42.13197970
405
+ 395 42.27848101
406
+ 396 42.42424242
407
+ 397 42.56926952
408
+ 398 42.46231156
409
+ 399 42.60651629
410
+ 400 42.50000000
411
+ 401 42.64339152
412
+ 402 42.53731343
413
+ 403 42.43176179
414
+ 404 42.32673267
415
+ 405 42.22222222
416
+ 406 42.11822660
417
+ 407 42.01474201
418
+ 408 41.91176471
419
+ 409 42.05378973
420
+ 410 41.95121951
421
+ 411 41.84914842
422
+ 412 41.74757282
423
+ 413 41.64648910
424
+ 414 41.54589372
425
+ 415 41.68674699
426
+ 416 41.82692308
427
+ 417 41.72661871
428
+ 418 41.62679426
429
+ 419 41.76610979
430
+ 420 41.90476190
431
+ 421 41.80522565
432
+ 422 41.70616114
433
+ 423 41.60756501
434
+ 424 41.74528302
435
+ 425 41.88235294
436
+ 426 42.01877934
437
+ 427 41.92037471
438
+ 428 41.82242991
439
+ 429 41.95804196
440
+ 430 41.86046512
441
+ 431 41.99535963
442
+ 432 42.12962963
443
+ 433 42.03233256
444
+ 434 41.93548387
445
+ 435 41.83908046
446
+ 436 41.74311927
447
+ 437 41.64759725
448
+ 438 41.78082192
449
+ 439 41.91343964
450
+ 440 41.81818182
451
+ 441 41.72335601
452
+ 442 41.62895928
453
+ 443 41.53498871
454
+ 444 41.44144144
455
+ 445 41.34831461
456
+ 446 41.25560538
457
+ 447 41.38702461
458
+ 448 41.29464286
459
+ 449 41.20267261
460
+ 450 41.33333333
461
+ 451 41.24168514
462
+ 452 41.37168142
463
+ 453 41.28035320
464
+ 454 41.18942731
465
+ 455 41.09890110
466
+ 456 41.22807018
467
+ 457 41.35667396
468
+ 458 41.26637555
469
+ 459 41.17647059
470
+ 460 41.30434783
471
+ 461 41.21475054
472
+ 462 41.12554113
473
+ 463 41.03671706
474
+ 464 41.16379310
475
+ 465 41.07526882
476
+ 466 40.98712446
477
+ 467 41.11349036
478
+ 468 41.02564103
479
+ 469 41.15138593
480
+ 470 41.06382979
481
+ 471 40.97664544
482
+ 472 40.88983051
483
+ 473 40.80338266
484
+ 474 40.71729958
485
+ 475 40.63157895
486
+ 476 40.75630252
487
+ 477 40.67085954
488
+ 478 40.79497908
489
+ 479 40.91858038
490
+ 480 41.04166667
491
+ 481 40.95634096
492
+ 482 40.87136929
493
+ 483 40.78674948
494
+ 484 40.70247934
495
+ 485 40.61855670
496
+ 486 40.53497942
497
+ 487 40.65708419
498
+ 488 40.57377049
499
+ 489 40.49079755
500
+ 490 40.61224490
501
+ 491 40.52953157
502
+ 492 40.44715447
503
+ 493 40.36511156
504
+ 494 40.48582996
505
+ 495 40.60606061
506
+ 496 40.72580645
507
+ 497 40.64386318
508
+ 498 40.56224900
509
+ 499 40.48096192
510
+ 500 40.40000000
511
+ 501 40.31936128
512
+ 502 40.23904382
513
+ 503 40.15904573
514
+ 504 40.27777778
515
+ 505 40.19801980
516
+ 506 40.31620553
517
+ 507 40.23668639
518
+ 508 40.35433071
519
+ 509 40.27504912
520
+ 510 40.39215686
521
+ 511 40.31311155
522
+ 512 40.23437500
523
+ 513 40.15594542
524
+ 514 40.07782101
525
+ 515 40.00000000
526
+ 516 40.11627907
527
+ 517 40.23210832
528
+ 518 40.34749035
529
+ 519 40.46242775
530
+ 520 40.57692308
531
+ 521 40.49904031
532
+ 522 40.42145594
533
+ 523 40.34416826
534
+ 524 40.45801527
535
+ 525 40.38095238
536
+ 526 40.30418251
537
+ 527 40.41745731
538
+ 528 40.34090909
539
+ 529 40.26465028
540
+ 530 40.37735849
541
+ 531 40.30131827
542
+ 532 40.22556391
543
+ 533 40.15009381
544
+ 534 40.07490637
545
+ 535 40.18691589
546
+ 536 40.11194030
547
+ 537 40.22346369
548
+ 538 40.33457249
549
+ 539 40.25974026
550
+ 540 40.18518519
551
+ 541 40.11090573
552
+ 542 40.22140221
553
+ 543 40.14732965
554
+ 544 40.07352941
555
+ 545 40.00000000
556
+ 546 39.92673993
557
+ 547 40.03656307
558
+ 548 40.14598540
559
+ 549 40.07285974
560
+ 550 40.00000000
561
+ 551 40.10889292
562
+ 552 40.03623188
563
+ 553 39.96383363
564
+ 554 40.07220217
565
+ 555 40.00000000
566
+ 556 40.10791367
567
+ 557 40.03590664
568
+ 558 40.14336918
569
+ 559 40.07155635
570
+ 560 40.00000000
571
+ 561 39.92869875
572
+ 562 40.03558719
573
+ 563 40.14209591
574
+ 564 40.07092199
575
+ 565 40.17699115
576
+ 566 40.10600707
577
+ 567 40.03527337
578
+ 568 39.96478873
579
+ 569 39.89455185
580
+ 570 40.00000000
581
+ 571 39.92994746
582
+ 572 40.03496503
583
+ 573 39.96509599
584
+ 574 39.89547038
585
+ 575 39.82608696
586
+ 576 39.93055556
587
+ 577 40.03466205
588
+ 578 39.96539792
589
+ 579 39.89637306
590
+ 580 40.00000000
591
+ 581 40.10327022
592
+ 582 40.03436426
593
+ 583 40.13722127
594
+ 584 40.23972603
595
+ 585 40.17094017
596
+ 586 40.27303754
597
+ 587 40.20442930
598
+ 588 40.13605442
599
+ 589 40.06791171
600
+ 590 40.00000000
601
+ 591 39.93231810
602
+ 592 39.86486486
603
+ 593 39.79763912
604
+ 594 39.89898990
605
+ 595 39.83193277
606
+ 596 39.93288591
607
+ 597 39.86599665
608
+ 598 39.79933110
609
+ 599 39.89983306
610
+ 600 39.83333333
611
+ 601 39.93344426
612
+ 602 39.86710963
613
+ 603 39.96683250
614
+ 604 39.90066225
615
+ 605 40.00000000
616
+ 606 39.93399340
617
+ 607 40.03294893
618
+ 608 39.96710526
619
+ 609 39.90147783
620
+ 610 39.83606557
621
+ 611 39.93453355
622
+ 612 39.86928105
623
+ 613 39.80424144
624
+ 614 39.90228013
625
+ 615 40.00000000
626
+ 616 40.09740260
627
+ 617 40.03241491
628
+ 618 40.12944984
629
+ 619 40.06462036
630
+ 620 40.00000000
631
+ 621 39.93558776
632
+ 622 40.03215434
633
+ 623 39.96789727
634
+ 624 39.90384615
635
+ 625 39.84000000
636
+ 626 39.93610224
637
+ 627 39.87240829
638
+ 628 39.80891720
639
+ 629 39.90461049
640
+ 630 39.84126984
641
+ 631 39.93660856
642
+ 632 40.03164557
643
+ 633 39.96840442
644
+ 634 39.90536278
645
+ 635 39.84251969
646
+ 636 39.77987421
647
+ 637 39.71742543
648
+ 638 39.65517241
649
+ 639 39.74960876
650
+ 640 39.84375000
651
+ 641 39.78159126
652
+ 642 39.87538941
653
+ 643 39.81337481
654
+ 644 39.90683230
655
+ 645 39.84496124
656
+ 646 39.78328173
657
+ 647 39.72179289
658
+ 648 39.66049383
659
+ 649 39.59938367
660
+ 650 39.53846154
661
+ 651 39.47772657
662
+ 652 39.57055215
663
+ 653 39.50995406
664
+ 654 39.60244648
665
+ 655 39.54198473
666
+ 656 39.63414634
667
+ 657 39.57382040
668
+ 658 39.66565350
669
+ 659 39.60546282
670
+ 660 39.54545455
671
+ 661 39.48562784
672
+ 662 39.42598187
673
+ 663 39.36651584
674
+ 664 39.45783133
675
+ 665 39.54887218
676
+ 666 39.48948949
677
+ 667 39.58020990
678
+ 668 39.67065868
679
+ 669 39.76083707
680
+ 670 39.70149254
681
+ 671 39.64232489
682
+ 672 39.58333333
683
+ 673 39.52451709
684
+ 674 39.46587537
685
+ 675 39.40740741
686
+ 676 39.34911243
687
+ 677 39.29098966
688
+ 678 39.38053097
689
+ 679 39.32253314
690
+ 680 39.41176471
691
+ 681 39.50073421
692
+ 682 39.44281525
693
+ 683 39.38506589
694
+ 684 39.32748538
695
+ 685 39.27007299
696
+ 686 39.21282799
697
+ 687 39.15574964
698
+ 688 39.09883721
699
+ 689 39.04208999
700
+ 690 38.98550725
701
+ 691 39.07380608
702
+ 692 39.01734104
703
+ 693 38.96103896
704
+ 694 38.90489914
705
+ 695 38.84892086
706
+ 696 38.79310345
707
+ 697 38.73744620
708
+ 698 38.68194842
709
+ 699 38.76967096
710
+ 700 38.71428571
711
+ 701 38.80171184
712
+ 702 38.88888889
713
+ 703 38.97581792
714
+ 704 38.92045455
715
+ 705 38.86524823
716
+ 706 38.81019830
717
+ 707 38.75530410
718
+ 708 38.70056497
719
+ 709 38.78702398
720
+ 710 38.87323944
721
+ 711 38.95921238
722
+ 712 38.90449438
723
+ 713 38.99018233
724
+ 714 39.07563025
725
+ 715 39.02097902
726
+ 716 38.96648045
727
+ 717 38.91213389
728
+ 718 38.85793872
729
+ 719 38.94297636
730
+ 720 38.88888889
731
+ 721 38.83495146
732
+ 722 38.91966759
733
+ 723 39.00414938
734
+ 724 38.95027624
735
+ 725 39.03448276
736
+ 726 38.98071625
737
+ 727 38.92709766
738
+ 728 38.87362637
739
+ 729 38.82030178
740
+ 730 38.76712329
741
+ 731 38.71409029
742
+ 732 38.79781421
743
+ 733 38.88130969
744
+ 734 38.96457766
745
+ 735 38.91156463
746
+ 736 38.85869565
747
+ 737 38.94165536
748
+ 738 38.88888889
749
+ 739 38.97158322
750
+ 740 38.91891892
751
+ 741 39.00134953
752
+ 742 39.08355795
753
+ 743 39.03095559
754
+ 744 39.11290323
755
+ 745 39.06040268
756
+ 746 39.00804290
757
+ 747 38.95582329
758
+ 748 38.90374332
759
+ 749 38.85180240
760
+ 750 38.93333333
761
+
762
+ Final result: 38.9333 +/- 1.7816
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-iq4_nl.ppx ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ====== Perplexity statistics ======
2
+ Mean PPL(Q) : 9.689751 ± 0.069052
3
+ Mean PPL(base) : 9.366577 ± 0.066397
4
+ Cor(ln(PPL(Q)), ln(PPL(base))): 99.33%
5
+ Mean ln(PPL(Q)/PPL(base)) : 0.033921 ± 0.000825
6
+ Mean PPL(Q)/PPL(base) : 1.034503 ± 0.000854
7
+ Mean PPL(Q)-PPL(base) : 0.323173 ± 0.008290
8
+
9
+ ====== KL divergence statistics ======
10
+ Mean KLD: 0.041211 ± 0.000134
11
+ Maximum KLD: 2.917926
12
+ 99.9% KLD: 0.491688
13
+ 99.0% KLD: 0.222239
14
+ 99.0% KLD: 0.222239
15
+ Median KLD: 0.029107
16
+ 10.0% KLD: 0.000610
17
+ 5.0% KLD: 0.000122
18
+ 1.0% KLD: 0.000008
19
+ Minimum KLD: -0.000100
20
+
21
+ ====== Token probability statistics ======
22
+ Mean Δp: -0.582 ± 0.014 %
23
+ Maximum Δp: 64.365%
24
+ 99.9% Δp: 25.232%
25
+ 99.0% Δp: 14.187%
26
+ 95.0% Δp: 6.937%
27
+ 90.0% Δp: 3.840%
28
+ 75.0% Δp: 0.544%
29
+ Median Δp: -0.015%
30
+ 25.0% Δp: -1.487%
31
+ 10.0% Δp: -5.778%
32
+ 5.0% Δp: -9.357%
33
+ 1.0% Δp: -18.173%
34
+ 0.1% Δp: -34.147%
35
+ Minimum Δp: -77.512%
36
+ RMS Δp : 5.262 ± 0.025 %
37
+ Same top p: 88.641 ± 0.082 %
scores/Hammer2.1-7b-iq4_nl.tqa ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 817 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 817 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 14.28571429
18
+ 8 12.50000000
19
+ 9 11.11111111
20
+ 10 10.00000000
21
+ 11 9.09090909
22
+ 12 16.66666667
23
+ 13 23.07692308
24
+ 14 21.42857143
25
+ 15 20.00000000
26
+ 16 18.75000000
27
+ 17 17.64705882
28
+ 18 16.66666667
29
+ 19 21.05263158
30
+ 20 20.00000000
31
+ 21 19.04761905
32
+ 22 18.18181818
33
+ 23 17.39130435
34
+ 24 20.83333333
35
+ 25 20.00000000
36
+ 26 19.23076923
37
+ 27 18.51851852
38
+ 28 17.85714286
39
+ 29 20.68965517
40
+ 30 20.00000000
41
+ 31 19.35483871
42
+ 32 21.87500000
43
+ 33 24.24242424
44
+ 34 23.52941176
45
+ 35 22.85714286
46
+ 36 22.22222222
47
+ 37 24.32432432
48
+ 38 23.68421053
49
+ 39 25.64102564
50
+ 40 25.00000000
51
+ 41 24.39024390
52
+ 42 23.80952381
53
+ 43 25.58139535
54
+ 44 27.27272727
55
+ 45 28.88888889
56
+ 46 30.43478261
57
+ 47 29.78723404
58
+ 48 29.16666667
59
+ 49 28.57142857
60
+ 50 30.00000000
61
+ 51 31.37254902
62
+ 52 30.76923077
63
+ 53 30.18867925
64
+ 54 31.48148148
65
+ 55 32.72727273
66
+ 56 32.14285714
67
+ 57 31.57894737
68
+ 58 31.03448276
69
+ 59 30.50847458
70
+ 60 30.00000000
71
+ 61 29.50819672
72
+ 62 30.64516129
73
+ 63 30.15873016
74
+ 64 29.68750000
75
+ 65 29.23076923
76
+ 66 28.78787879
77
+ 67 28.35820896
78
+ 68 27.94117647
79
+ 69 28.98550725
80
+ 70 30.00000000
81
+ 71 29.57746479
82
+ 72 30.55555556
83
+ 73 30.13698630
84
+ 74 29.72972973
85
+ 75 29.33333333
86
+ 76 28.94736842
87
+ 77 28.57142857
88
+ 78 28.20512821
89
+ 79 27.84810127
90
+ 80 28.75000000
91
+ 81 28.39506173
92
+ 82 29.26829268
93
+ 83 28.91566265
94
+ 84 28.57142857
95
+ 85 28.23529412
96
+ 86 27.90697674
97
+ 87 27.58620690
98
+ 88 27.27272727
99
+ 89 28.08988764
100
+ 90 27.77777778
101
+ 91 27.47252747
102
+ 92 27.17391304
103
+ 93 26.88172043
104
+ 94 26.59574468
105
+ 95 27.36842105
106
+ 96 28.12500000
107
+ 97 27.83505155
108
+ 98 27.55102041
109
+ 99 28.28282828
110
+ 100 29.00000000
111
+ 101 29.70297030
112
+ 102 29.41176471
113
+ 103 29.12621359
114
+ 104 28.84615385
115
+ 105 28.57142857
116
+ 106 28.30188679
117
+ 107 28.97196262
118
+ 108 29.62962963
119
+ 109 29.35779817
120
+ 110 30.00000000
121
+ 111 29.72972973
122
+ 112 29.46428571
123
+ 113 29.20353982
124
+ 114 29.82456140
125
+ 115 29.56521739
126
+ 116 29.31034483
127
+ 117 29.05982906
128
+ 118 28.81355932
129
+ 119 28.57142857
130
+ 120 29.16666667
131
+ 121 29.75206612
132
+ 122 29.50819672
133
+ 123 30.08130081
134
+ 124 29.83870968
135
+ 125 30.40000000
136
+ 126 30.15873016
137
+ 127 29.92125984
138
+ 128 29.68750000
139
+ 129 29.45736434
140
+ 130 29.23076923
141
+ 131 29.00763359
142
+ 132 28.78787879
143
+ 133 28.57142857
144
+ 134 29.10447761
145
+ 135 29.62962963
146
+ 136 29.41176471
147
+ 137 29.92700730
148
+ 138 30.43478261
149
+ 139 30.93525180
150
+ 140 30.71428571
151
+ 141 30.49645390
152
+ 142 30.28169014
153
+ 143 30.06993007
154
+ 144 30.55555556
155
+ 145 30.34482759
156
+ 146 30.13698630
157
+ 147 29.93197279
158
+ 148 29.72972973
159
+ 149 29.53020134
160
+ 150 29.33333333
161
+ 151 29.80132450
162
+ 152 29.60526316
163
+ 153 29.41176471
164
+ 154 29.22077922
165
+ 155 29.67741935
166
+ 156 30.12820513
167
+ 157 30.57324841
168
+ 158 31.01265823
169
+ 159 30.81761006
170
+ 160 30.62500000
171
+ 161 30.43478261
172
+ 162 30.24691358
173
+ 163 30.06134969
174
+ 164 29.87804878
175
+ 165 30.30303030
176
+ 166 30.72289157
177
+ 167 30.53892216
178
+ 168 30.35714286
179
+ 169 30.17751479
180
+ 170 30.00000000
181
+ 171 29.82456140
182
+ 172 29.65116279
183
+ 173 29.47976879
184
+ 174 29.88505747
185
+ 175 30.28571429
186
+ 176 30.11363636
187
+ 177 29.94350282
188
+ 178 29.77528090
189
+ 179 30.16759777
190
+ 180 30.00000000
191
+ 181 30.38674033
192
+ 182 30.21978022
193
+ 183 30.05464481
194
+ 184 29.89130435
195
+ 185 29.72972973
196
+ 186 30.10752688
197
+ 187 29.94652406
198
+ 188 30.31914894
199
+ 189 30.68783069
200
+ 190 30.52631579
201
+ 191 30.36649215
202
+ 192 30.72916667
203
+ 193 30.56994819
204
+ 194 30.92783505
205
+ 195 30.76923077
206
+ 196 30.61224490
207
+ 197 30.96446701
208
+ 198 31.31313131
209
+ 199 31.15577889
210
+ 200 31.00000000
211
+ 201 30.84577114
212
+ 202 31.18811881
213
+ 203 31.03448276
214
+ 204 30.88235294
215
+ 205 30.73170732
216
+ 206 30.58252427
217
+ 207 30.91787440
218
+ 208 30.76923077
219
+ 209 30.62200957
220
+ 210 30.47619048
221
+ 211 30.33175355
222
+ 212 30.18867925
223
+ 213 30.04694836
224
+ 214 29.90654206
225
+ 215 29.76744186
226
+ 216 30.09259259
227
+ 217 29.95391705
228
+ 218 30.27522936
229
+ 219 30.13698630
230
+ 220 30.00000000
231
+ 221 29.86425339
232
+ 222 29.72972973
233
+ 223 29.59641256
234
+ 224 29.91071429
235
+ 225 29.77777778
236
+ 226 29.64601770
237
+ 227 29.95594714
238
+ 228 29.82456140
239
+ 229 29.69432314
240
+ 230 29.56521739
241
+ 231 29.43722944
242
+ 232 29.74137931
243
+ 233 30.04291845
244
+ 234 30.34188034
245
+ 235 30.21276596
246
+ 236 30.50847458
247
+ 237 30.80168776
248
+ 238 30.67226891
249
+ 239 30.54393305
250
+ 240 30.41666667
251
+ 241 30.29045643
252
+ 242 30.16528926
253
+ 243 30.04115226
254
+ 244 29.91803279
255
+ 245 29.79591837
256
+ 246 29.67479675
257
+ 247 29.55465587
258
+ 248 29.43548387
259
+ 249 29.31726908
260
+ 250 29.20000000
261
+ 251 29.48207171
262
+ 252 29.36507937
263
+ 253 29.24901186
264
+ 254 29.13385827
265
+ 255 29.41176471
266
+ 256 29.29687500
267
+ 257 29.18287938
268
+ 258 29.06976744
269
+ 259 28.95752896
270
+ 260 28.84615385
271
+ 261 29.11877395
272
+ 262 29.00763359
273
+ 263 28.89733840
274
+ 264 29.16666667
275
+ 265 29.43396226
276
+ 266 29.32330827
277
+ 267 29.58801498
278
+ 268 29.47761194
279
+ 269 29.36802974
280
+ 270 29.25925926
281
+ 271 29.15129151
282
+ 272 29.04411765
283
+ 273 28.93772894
284
+ 274 29.19708029
285
+ 275 29.09090909
286
+ 276 28.98550725
287
+ 277 28.88086643
288
+ 278 28.77697842
289
+ 279 28.67383513
290
+ 280 28.92857143
291
+ 281 29.18149466
292
+ 282 29.07801418
293
+ 283 28.97526502
294
+ 284 29.22535211
295
+ 285 29.12280702
296
+ 286 29.37062937
297
+ 287 29.61672474
298
+ 288 29.51388889
299
+ 289 29.41176471
300
+ 290 29.31034483
301
+ 291 29.55326460
302
+ 292 29.45205479
303
+ 293 29.69283276
304
+ 294 29.59183673
305
+ 295 29.83050847
306
+ 296 30.06756757
307
+ 297 30.30303030
308
+ 298 30.20134228
309
+ 299 30.10033445
310
+ 300 30.00000000
311
+ 301 29.90033223
312
+ 302 29.80132450
313
+ 303 29.70297030
314
+ 304 29.60526316
315
+ 305 29.50819672
316
+ 306 29.41176471
317
+ 307 29.31596091
318
+ 308 29.54545455
319
+ 309 29.44983819
320
+ 310 29.35483871
321
+ 311 29.26045016
322
+ 312 29.16666667
323
+ 313 29.07348243
324
+ 314 29.29936306
325
+ 315 29.20634921
326
+ 316 29.11392405
327
+ 317 29.02208202
328
+ 318 28.93081761
329
+ 319 28.84012539
330
+ 320 28.75000000
331
+ 321 28.66043614
332
+ 322 28.88198758
333
+ 323 28.79256966
334
+ 324 28.70370370
335
+ 325 28.61538462
336
+ 326 28.52760736
337
+ 327 28.74617737
338
+ 328 28.96341463
339
+ 329 28.87537994
340
+ 330 29.09090909
341
+ 331 29.00302115
342
+ 332 29.21686747
343
+ 333 29.12912913
344
+ 334 29.04191617
345
+ 335 29.25373134
346
+ 336 29.46428571
347
+ 337 29.37685460
348
+ 338 29.28994083
349
+ 339 29.49852507
350
+ 340 29.41176471
351
+ 341 29.61876833
352
+ 342 29.82456140
353
+ 343 29.73760933
354
+ 344 29.94186047
355
+ 345 30.14492754
356
+ 346 30.05780347
357
+ 347 30.25936599
358
+ 348 30.17241379
359
+ 349 30.08595989
360
+ 350 30.00000000
361
+ 351 29.91452991
362
+ 352 29.82954545
363
+ 353 29.74504249
364
+ 354 29.66101695
365
+ 355 29.85915493
366
+ 356 29.77528090
367
+ 357 29.97198880
368
+ 358 30.16759777
369
+ 359 30.08356546
370
+ 360 30.00000000
371
+ 361 30.19390582
372
+ 362 30.11049724
373
+ 363 30.30303030
374
+ 364 30.21978022
375
+ 365 30.41095890
376
+ 366 30.32786885
377
+ 367 30.51771117
378
+ 368 30.43478261
379
+ 369 30.35230352
380
+ 370 30.27027027
381
+ 371 30.18867925
382
+ 372 30.10752688
383
+ 373 30.02680965
384
+ 374 30.21390374
385
+ 375 30.13333333
386
+ 376 30.05319149
387
+ 377 29.97347480
388
+ 378 29.89417989
389
+ 379 29.81530343
390
+ 380 30.00000000
391
+ 381 29.92125984
392
+ 382 30.10471204
393
+ 383 30.02610966
394
+ 384 29.94791667
395
+ 385 29.87012987
396
+ 386 29.79274611
397
+ 387 29.71576227
398
+ 388 29.89690722
399
+ 389 29.82005141
400
+ 390 29.74358974
401
+ 391 29.92327366
402
+ 392 29.84693878
403
+ 393 30.02544529
404
+ 394 30.20304569
405
+ 395 30.12658228
406
+ 396 30.05050505
407
+ 397 30.22670025
408
+ 398 30.15075377
409
+ 399 30.07518797
410
+ 400 30.00000000
411
+ 401 29.92518703
412
+ 402 30.09950249
413
+ 403 30.27295285
414
+ 404 30.19801980
415
+ 405 30.12345679
416
+ 406 30.29556650
417
+ 407 30.22113022
418
+ 408 30.14705882
419
+ 409 30.07334963
420
+ 410 30.00000000
421
+ 411 29.92700730
422
+ 412 30.09708738
423
+ 413 30.02421308
424
+ 414 30.19323671
425
+ 415 30.36144578
426
+ 416 30.28846154
427
+ 417 30.21582734
428
+ 418 30.14354067
429
+ 419 30.07159905
430
+ 420 30.23809524
431
+ 421 30.40380048
432
+ 422 30.33175355
433
+ 423 30.26004728
434
+ 424 30.18867925
435
+ 425 30.11764706
436
+ 426 30.04694836
437
+ 427 29.97658080
438
+ 428 29.90654206
439
+ 429 30.06993007
440
+ 430 30.00000000
441
+ 431 29.93039443
442
+ 432 30.09259259
443
+ 433 30.25404157
444
+ 434 30.41474654
445
+ 435 30.34482759
446
+ 436 30.27522936
447
+ 437 30.20594966
448
+ 438 30.13698630
449
+ 439 30.06833713
450
+ 440 30.00000000
451
+ 441 29.93197279
452
+ 442 29.86425339
453
+ 443 29.79683973
454
+ 444 29.95495495
455
+ 445 29.88764045
456
+ 446 29.82062780
457
+ 447 29.75391499
458
+ 448 29.68750000
459
+ 449 29.84409800
460
+ 450 30.00000000
461
+ 451 29.93348115
462
+ 452 29.86725664
463
+ 453 29.80132450
464
+ 454 29.95594714
465
+ 455 29.89010989
466
+ 456 29.82456140
467
+ 457 29.75929978
468
+ 458 29.69432314
469
+ 459 29.62962963
470
+ 460 29.56521739
471
+ 461 29.50108460
472
+ 462 29.43722944
473
+ 463 29.58963283
474
+ 464 29.74137931
475
+ 465 29.89247312
476
+ 466 29.82832618
477
+ 467 29.76445396
478
+ 468 29.91452991
479
+ 469 29.85074627
480
+ 470 29.78723404
481
+ 471 29.93630573
482
+ 472 29.87288136
483
+ 473 29.80972516
484
+ 474 29.74683544
485
+ 475 29.89473684
486
+ 476 29.83193277
487
+ 477 29.97903564
488
+ 478 29.91631799
489
+ 479 29.85386221
490
+ 480 29.79166667
491
+ 481 29.72972973
492
+ 482 29.87551867
493
+ 483 29.81366460
494
+ 484 29.95867769
495
+ 485 30.10309278
496
+ 486 30.04115226
497
+ 487 30.18480493
498
+ 488 30.12295082
499
+ 489 30.26584867
500
+ 490 30.40816327
501
+ 491 30.54989817
502
+ 492 30.69105691
503
+ 493 30.62880325
504
+ 494 30.56680162
505
+ 495 30.70707071
506
+ 496 30.64516129
507
+ 497 30.58350101
508
+ 498 30.52208835
509
+ 499 30.66132265
510
+ 500 30.60000000
511
+ 501 30.73852295
512
+ 502 30.67729084
513
+ 503 30.61630219
514
+ 504 30.75396825
515
+ 505 30.69306931
516
+ 506 30.63241107
517
+ 507 30.76923077
518
+ 508 30.70866142
519
+ 509 30.84479371
520
+ 510 30.98039216
521
+ 511 30.91976517
522
+ 512 30.85937500
523
+ 513 30.79922027
524
+ 514 30.73929961
525
+ 515 30.87378641
526
+ 516 30.81395349
527
+ 517 30.94777563
528
+ 518 30.88803089
529
+ 519 30.82851638
530
+ 520 30.76923077
531
+ 521 30.71017274
532
+ 522 30.65134100
533
+ 523 30.59273423
534
+ 524 30.53435115
535
+ 525 30.47619048
536
+ 526 30.41825095
537
+ 527 30.55028463
538
+ 528 30.49242424
539
+ 529 30.43478261
540
+ 530 30.37735849
541
+ 531 30.32015066
542
+ 532 30.26315789
543
+ 533 30.39399625
544
+ 534 30.33707865
545
+ 535 30.28037383
546
+ 536 30.22388060
547
+ 537 30.35381750
548
+ 538 30.29739777
549
+ 539 30.24118738
550
+ 540 30.37037037
551
+ 541 30.31423290
552
+ 542 30.25830258
553
+ 543 30.20257827
554
+ 544 30.14705882
555
+ 545 30.09174312
556
+ 546 30.21978022
557
+ 547 30.34734918
558
+ 548 30.47445255
559
+ 549 30.41894353
560
+ 550 30.36363636
561
+ 551 30.30852995
562
+ 552 30.43478261
563
+ 553 30.56057866
564
+ 554 30.50541516
565
+ 555 30.45045045
566
+ 556 30.39568345
567
+ 557 30.34111311
568
+ 558 30.28673835
569
+ 559 30.23255814
570
+ 560 30.35714286
571
+ 561 30.48128342
572
+ 562 30.42704626
573
+ 563 30.37300178
574
+ 564 30.49645390
575
+ 565 30.61946903
576
+ 566 30.74204947
577
+ 567 30.68783069
578
+ 568 30.80985915
579
+ 569 30.93145870
580
+ 570 30.87719298
581
+ 571 30.82311734
582
+ 572 30.76923077
583
+ 573 30.71553229
584
+ 574 30.83623693
585
+ 575 30.95652174
586
+ 576 30.90277778
587
+ 577 31.02253033
588
+ 578 31.14186851
589
+ 579 31.26079447
590
+ 580 31.37931034
591
+ 581 31.32530120
592
+ 582 31.27147766
593
+ 583 31.21783877
594
+ 584 31.33561644
595
+ 585 31.28205128
596
+ 586 31.39931741
597
+ 587 31.34582624
598
+ 588 31.29251701
599
+ 589 31.23938879
600
+ 590 31.35593220
601
+ 591 31.30287648
602
+ 592 31.25000000
603
+ 593 31.19730185
604
+ 594 31.14478114
605
+ 595 31.09243697
606
+ 596 31.04026846
607
+ 597 30.98827471
608
+ 598 30.93645485
609
+ 599 30.88480801
610
+ 600 30.83333333
611
+ 601 30.94841930
612
+ 602 30.89700997
613
+ 603 31.01160862
614
+ 604 30.96026490
615
+ 605 30.90909091
616
+ 606 31.02310231
617
+ 607 30.97199341
618
+ 608 30.92105263
619
+ 609 30.87027915
620
+ 610 30.81967213
621
+ 611 30.76923077
622
+ 612 30.88235294
623
+ 613 30.83197390
624
+ 614 30.78175896
625
+ 615 30.73170732
626
+ 616 30.84415584
627
+ 617 30.95623987
628
+ 618 30.90614887
629
+ 619 31.01777060
630
+ 620 30.96774194
631
+ 621 30.91787440
632
+ 622 31.02893891
633
+ 623 30.97913323
634
+ 624 30.92948718
635
+ 625 30.88000000
636
+ 626 30.83067093
637
+ 627 30.78149920
638
+ 628 30.73248408
639
+ 629 30.68362480
640
+ 630 30.63492063
641
+ 631 30.74484945
642
+ 632 30.69620253
643
+ 633 30.64770932
644
+ 634 30.75709779
645
+ 635 30.70866142
646
+ 636 30.66037736
647
+ 637 30.61224490
648
+ 638 30.56426332
649
+ 639 30.51643192
650
+ 640 30.46875000
651
+ 641 30.42121685
652
+ 642 30.52959502
653
+ 643 30.48211509
654
+ 644 30.43478261
655
+ 645 30.54263566
656
+ 646 30.49535604
657
+ 647 30.44822257
658
+ 648 30.55555556
659
+ 649 30.50847458
660
+ 650 30.61538462
661
+ 651 30.72196621
662
+ 652 30.82822086
663
+ 653 30.93415008
664
+ 654 30.88685015
665
+ 655 30.83969466
666
+ 656 30.94512195
667
+ 657 31.05022831
668
+ 658 31.00303951
669
+ 659 31.10773900
670
+ 660 31.06060606
671
+ 661 31.01361573
672
+ 662 30.96676737
673
+ 663 30.92006033
674
+ 664 30.87349398
675
+ 665 30.82706767
676
+ 666 30.93093093
677
+ 667 30.88455772
678
+ 668 30.83832335
679
+ 669 30.79222720
680
+ 670 30.89552239
681
+ 671 30.84947839
682
+ 672 30.80357143
683
+ 673 30.75780089
684
+ 674 30.71216617
685
+ 675 30.81481481
686
+ 676 30.91715976
687
+ 677 30.87149188
688
+ 678 30.82595870
689
+ 679 30.78055965
690
+ 680 30.73529412
691
+ 681 30.83700441
692
+ 682 30.79178886
693
+ 683 30.74670571
694
+ 684 30.84795322
695
+ 685 30.80291971
696
+ 686 30.75801749
697
+ 687 30.71324600
698
+ 688 30.66860465
699
+ 689 30.76923077
700
+ 690 30.72463768
701
+ 691 30.82489146
702
+ 692 30.92485549
703
+ 693 31.02453102
704
+ 694 30.97982709
705
+ 695 30.93525180
706
+ 696 31.03448276
707
+ 697 30.98995696
708
+ 698 30.94555874
709
+ 699 30.90128755
710
+ 700 30.85714286
711
+ 701 30.81312411
712
+ 702 30.76923077
713
+ 703 30.72546230
714
+ 704 30.68181818
715
+ 705 30.63829787
716
+ 706 30.59490085
717
+ 707 30.69306931
718
+ 708 30.64971751
719
+ 709 30.60648801
720
+ 710 30.56338028
721
+ 711 30.52039381
722
+ 712 30.47752809
723
+ 713 30.43478261
724
+ 714 30.53221289
725
+ 715 30.62937063
726
+ 716 30.58659218
727
+ 717 30.54393305
728
+ 718 30.50139276
729
+ 719 30.45897079
730
+ 720 30.41666667
731
+ 721 30.37447989
732
+ 722 30.33240997
733
+ 723 30.29045643
734
+ 724 30.38674033
735
+ 725 30.34482759
736
+ 726 30.44077135
737
+ 727 30.39889959
738
+ 728 30.35714286
739
+ 729 30.45267490
740
+ 730 30.41095890
741
+ 731 30.36935705
742
+ 732 30.46448087
743
+ 733 30.42291951
744
+ 734 30.38147139
745
+ 735 30.34013605
746
+ 736 30.29891304
747
+ 737 30.39348711
748
+ 738 30.35230352
749
+ 739 30.44654939
750
+ 740 30.40540541
751
+ 741 30.49932524
752
+ 742 30.45822102
753
+ 743 30.41722746
754
+ 744 30.51075269
755
+ 745 30.46979866
756
+ 746 30.42895442
757
+ 747 30.52208835
758
+ 748 30.48128342
759
+ 749 30.57409880
760
+ 750 30.66666667
761
+
762
+ Final result: 30.6667 +/- 1.6849
763
+ Random chance: 19.8992 +/- 1.4588
scores/Hammer2.1-7b-iq4_nl.wng ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ winogrande_score : loaded 1267 tasks from prompt.
6
+ winogrande_score : selecting 750 random tasks
7
+ winogrande_score : tokenizing selected tasks
8
+ winogrande_score : calculating winogrande score over selected tasks.
9
+ 1 100.0000 -4.418299 -4.619166 1 1
10
+ 2 100.0000 -2.070422 -2.495530 1 1
11
+ 3 100.0000 -3.031530 -2.604157 2 2
12
+ 4 75.0000 -5.861772 -5.175596 2 1
13
+ 5 60.0000 -2.577596 -3.288728 1 2
14
+ 6 66.6667 -5.009685 -5.113752 1 1
15
+ 7 71.4286 -4.554048 -3.481607 2 2
16
+ 8 75.0000 -2.099048 -1.947240 2 2
17
+ 9 77.7778 -3.612383 -5.869130 1 1
18
+ 10 80.0000 -1.929910 -1.225112 2 2
19
+ 11 81.8182 -11.629316 -5.677623 2 2
20
+ 12 83.3333 -3.517350 -2.652758 2 2
21
+ 13 84.6154 -2.627002 -5.232495 1 1
22
+ 14 85.7143 -4.130492 -4.164995 1 1
23
+ 15 80.0000 -4.491251 -5.749799 1 2
24
+ 16 81.2500 -4.035238 -3.600291 2 2
25
+ 17 76.4706 -5.810210 -6.111511 1 2
26
+ 18 77.7778 -1.352429 -2.819095 1 1
27
+ 19 78.9474 -4.289385 -5.280352 1 1
28
+ 20 80.0000 -2.766293 -3.717720 1 1
29
+ 21 80.9524 -2.644405 -3.171520 1 1
30
+ 22 81.8182 -2.592984 -2.306226 2 2
31
+ 23 78.2609 -5.116902 -5.074554 2 1
32
+ 24 75.0000 -2.620137 -2.606624 2 1
33
+ 25 76.0000 -2.882200 -2.716182 2 2
34
+ 26 76.9231 -3.394986 -4.060211 1 1
35
+ 27 77.7778 -5.575120 -5.236444 2 2
36
+ 28 78.5714 -4.740743 -5.756232 1 1
37
+ 29 79.3103 -3.859046 -4.100281 1 1
38
+ 30 76.6667 -1.564112 -1.804942 1 2
39
+ 31 77.4194 -5.820964 -5.844945 1 1
40
+ 32 78.1250 -4.027864 -6.057354 1 1
41
+ 33 78.7879 -3.233722 -6.793154 1 1
42
+ 34 76.4706 -2.542052 -2.408365 2 1
43
+ 35 77.1429 -4.859407 -4.703992 2 2
44
+ 36 77.7778 -4.620427 -3.498757 2 2
45
+ 37 75.6757 -3.543613 -3.052344 2 1
46
+ 38 76.3158 -3.264313 -3.612480 1 1
47
+ 39 76.9231 -4.763495 -4.392353 2 2
48
+ 40 75.0000 -4.522845 -5.476006 1 2
49
+ 41 75.6098 -2.536103 -2.787876 1 1
50
+ 42 76.1905 -1.734004 -2.313555 1 1
51
+ 43 74.4186 -4.180313 -4.309362 1 2
52
+ 44 72.7273 -1.711477 -1.735237 1 2
53
+ 45 73.3333 -0.095666 -15.778098 1 1
54
+ 46 73.9130 -2.709033 -3.003858 1 1
55
+ 47 72.3404 -4.302525 -4.724817 1 2
56
+ 48 72.9167 -4.944968 -4.652920 2 2
57
+ 49 71.4286 -3.318548 -1.293269 2 1
58
+ 50 72.0000 -2.621959 -2.572858 2 2
59
+ 51 72.5490 -2.319582 -2.750175 1 1
60
+ 52 73.0769 -2.876687 -2.204776 2 2
61
+ 53 73.5849 -4.340459 -5.226057 1 1
62
+ 54 74.0741 -3.209441 -2.900822 2 2
63
+ 55 74.5455 -3.679151 -3.320808 2 2
64
+ 56 73.2143 -4.913237 -5.263210 1 2
65
+ 57 71.9298 -1.664788 -2.694240 1 2
66
+ 58 72.4138 -3.258584 -4.151484 1 1
67
+ 59 72.8814 -3.383315 -2.968983 2 2
68
+ 60 73.3333 -4.760187 -4.416277 2 2
69
+ 61 73.7705 -3.718547 -4.222094 1 1
70
+ 62 72.5806 -6.696435 -6.690025 2 1
71
+ 63 71.4286 -4.295175 -4.940872 1 2
72
+ 64 71.8750 -4.004683 -4.160928 1 1
73
+ 65 72.3077 -4.877863 -4.296346 2 2
74
+ 66 72.7273 -2.703487 -3.678301 1 1
75
+ 67 73.1343 -2.532809 -2.142450 2 2
76
+ 68 72.0588 -5.119693 -5.335374 1 2
77
+ 69 72.4638 -4.941449 -7.956357 1 1
78
+ 70 72.8571 -3.592359 -2.986384 2 2
79
+ 71 71.8310 -3.186395 -2.933035 2 1
80
+ 72 70.8333 -3.139573 -3.427915 1 2
81
+ 73 71.2329 -2.738861 -3.096912 1 1
82
+ 74 70.2703 -2.371827 -2.365895 2 1
83
+ 75 70.6667 -1.542754 -2.241779 1 1
84
+ 76 71.0526 -1.994306 -4.184102 1 1
85
+ 77 70.1299 -2.993095 -3.362268 1 2
86
+ 78 70.5128 -1.008514 -1.486710 1 1
87
+ 79 69.6203 -2.359926 -2.522784 1 2
88
+ 80 70.0000 -2.040689 -3.573364 1 1
89
+ 81 69.1358 -3.779114 -3.791726 1 2
90
+ 82 68.2927 -2.807167 -2.927162 1 2
91
+ 83 68.6747 -6.229354 -6.157567 2 2
92
+ 84 69.0476 -2.403419 -2.230640 2 2
93
+ 85 69.4118 -4.734597 -5.196246 1 1
94
+ 86 68.6047 -6.041379 -6.453060 1 2
95
+ 87 67.8161 -1.473862 -1.131824 2 1
96
+ 88 67.0455 -4.662306 -4.426327 2 1
97
+ 89 67.4157 -2.914879 -3.004446 1 1
98
+ 90 67.7778 -5.979321 -3.892459 2 2
99
+ 91 68.1319 -4.341753 -4.345001 1 1
100
+ 92 68.4783 -4.829423 -3.586698 2 2
101
+ 93 68.8172 -2.514850 -1.733101 2 2
102
+ 94 69.1489 -4.634693 -3.444912 2 2
103
+ 95 69.4737 -5.083945 -5.816718 1 1
104
+ 96 69.7917 -1.793170 -2.777949 1 1
105
+ 97 70.1031 -4.686289 -4.829115 1 1
106
+ 98 70.4082 -4.549412 -3.745421 2 2
107
+ 99 69.6970 -2.673515 -2.168936 2 1
108
+ 100 69.0000 -6.144491 -6.180767 1 2
109
+ 101 68.3168 -4.686498 -4.808729 1 2
110
+ 102 68.6275 -3.472150 -4.611152 1 1
111
+ 103 67.9612 -2.572364 -2.901377 1 2
112
+ 104 68.2692 -8.417217 -8.324403 2 2
113
+ 105 68.5714 -1.331519 -2.271922 1 1
114
+ 106 68.8679 -2.761971 -2.350054 2 2
115
+ 107 69.1589 -3.584884 -2.504413 2 2
116
+ 108 69.4444 -3.887989 -2.806545 2 2
117
+ 109 69.7248 -5.845431 -5.687120 2 2
118
+ 110 70.0000 -4.476697 -3.482686 2 2
119
+ 111 70.2703 -4.218455 -3.733480 2 2
120
+ 112 69.6429 -3.713498 -3.188667 2 1
121
+ 113 69.0265 -4.700823 -4.925471 1 2
122
+ 114 69.2982 -3.596948 -4.875608 1 1
123
+ 115 69.5652 -1.601822 -2.373304 1 1
124
+ 116 69.8276 -3.655377 -2.629475 2 2
125
+ 117 69.2308 -4.322877 -4.609472 1 2
126
+ 118 69.4915 -0.676153 -2.233510 1 1
127
+ 119 69.7479 -2.491239 -2.902760 1 1
128
+ 120 69.1667 -2.747201 -2.616075 2 1
129
+ 121 69.4215 -0.913721 -0.587534 2 2
130
+ 122 69.6721 -6.204949 -4.692659 2 2
131
+ 123 69.9187 -3.506568 -3.542624 1 1
132
+ 124 70.1613 -2.476432 -2.185685 2 2
133
+ 125 70.4000 -3.637763 -2.806173 2 2
134
+ 126 69.8413 -6.581263 -6.697656 1 2
135
+ 127 70.0787 -2.093587 -1.470722 2 2
136
+ 128 70.3125 -1.957196 -2.184368 1 1
137
+ 129 70.5426 -1.377751 -0.951320 2 2
138
+ 130 70.7692 -5.377387 -5.098845 2 2
139
+ 131 70.2290 -3.600817 -3.734136 1 2
140
+ 132 70.4545 -2.618450 -2.408478 2 2
141
+ 133 70.6767 -1.649903 -4.404119 1 1
142
+ 134 70.8955 -5.625316 -4.554232 2 2
143
+ 135 71.1111 -4.241134 -5.036801 1 1
144
+ 136 71.3235 -5.732466 -4.535905 2 2
145
+ 137 70.8029 -6.410373 -6.349894 2 1
146
+ 138 70.2899 -4.027330 -4.845378 1 2
147
+ 139 69.7842 -4.614197 -4.426840 2 1
148
+ 140 70.0000 -1.471656 -1.123267 2 2
149
+ 141 70.2128 -3.950045 -3.231669 2 2
150
+ 142 70.4225 -2.183441 -2.170391 2 2
151
+ 143 70.6294 -1.665077 -5.822059 1 1
152
+ 144 70.1389 -1.975867 -1.929646 2 1
153
+ 145 69.6552 -1.773778 -12.851291 1 2
154
+ 146 69.8630 -4.282153 -3.964455 2 2
155
+ 147 69.3878 -4.520148 -4.541721 1 2
156
+ 148 69.5946 -1.870669 -2.430368 1 1
157
+ 149 69.7987 -4.807074 -3.639402 2 2
158
+ 150 70.0000 -4.118602 -3.092689 2 2
159
+ 151 69.5364 -1.994709 -1.935907 2 1
160
+ 152 69.0789 -3.787884 -4.101937 1 2
161
+ 153 69.2810 -3.295340 -3.518375 1 1
162
+ 154 69.4805 -3.462675 -3.752500 1 1
163
+ 155 69.0323 -2.389501 -2.457137 1 2
164
+ 156 68.5897 -2.432020 -2.247219 2 1
165
+ 157 68.7898 -2.820298 -3.949622 1 1
166
+ 158 68.9873 -4.092351 -3.512078 2 2
167
+ 159 69.1824 -3.338179 -5.121031 1 1
168
+ 160 69.3750 -4.995810 -3.741373 2 2
169
+ 161 69.5652 -2.514312 -2.827583 1 1
170
+ 162 69.1358 -0.406587 -15.660112 1 2
171
+ 163 69.3252 -1.878774 -1.974573 1 1
172
+ 164 69.5122 -5.169071 -4.702465 2 2
173
+ 165 69.6970 -4.853898 -4.764339 2 2
174
+ 166 69.8795 -4.267756 -3.543939 2 2
175
+ 167 69.4611 -4.894445 -4.332355 2 1
176
+ 168 69.6429 -5.336025 -5.322507 2 2
177
+ 169 69.2308 -3.693990 -4.070814 1 2
178
+ 170 68.8235 -4.280351 -4.012453 2 1
179
+ 171 69.0058 -3.257315 -4.592891 1 1
180
+ 172 69.1860 -4.292871 -5.342720 1 1
181
+ 173 69.3642 -3.322020 -2.815814 2 2
182
+ 174 69.5402 -6.887690 -6.375874 2 2
183
+ 175 69.1429 -4.171055 -4.074673 2 1
184
+ 176 69.3182 -1.442145 -3.471032 1 1
185
+ 177 69.4915 -5.792184 -4.902866 2 2
186
+ 178 69.1011 -2.488362 -2.225061 2 1
187
+ 179 69.2737 -2.723380 -2.830927 1 1
188
+ 180 69.4444 -6.203972 -5.630623 2 2
189
+ 181 69.6133 -3.755229 -2.467504 2 2
190
+ 182 69.7802 -4.397985 -3.617083 2 2
191
+ 183 69.9454 -6.126502 -5.784617 2 2
192
+ 184 70.1087 -1.732876 -2.908546 1 1
193
+ 185 69.7297 -6.106133 -6.136929 1 2
194
+ 186 69.8925 -2.663506 -3.258497 1 1
195
+ 187 70.0535 -4.838440 -4.487029 2 2
196
+ 188 69.6809 -3.238554 -2.890547 2 1
197
+ 189 69.3122 -5.731422 -6.261979 1 2
198
+ 190 69.4737 -5.288620 -5.183308 2 2
199
+ 191 69.6335 -4.015120 -4.237835 1 1
200
+ 192 69.7917 -6.415080 -4.787334 2 2
201
+ 193 69.9482 -4.086745 -3.862152 2 2
202
+ 194 70.1031 -1.963740 -1.266086 2 2
203
+ 195 69.7436 -4.151508 -3.984299 2 1
204
+ 196 69.8980 -3.550102 -2.482609 2 2
205
+ 197 70.0508 -0.449082 -2.519737 1 1
206
+ 198 70.2020 -5.491905 -6.232700 1 1
207
+ 199 70.3518 -4.955074 -4.112594 2 2
208
+ 200 70.5000 -3.419264 -3.035205 2 2
209
+ 201 70.6468 -2.384644 -3.062852 1 1
210
+ 202 70.7921 -7.921449 -7.621966 2 2
211
+ 203 70.9360 -3.864159 -3.632021 2 2
212
+ 204 71.0784 -2.830582 -3.293395 1 1
213
+ 205 71.2195 -3.693132 -2.274226 2 2
214
+ 206 70.8738 -5.591271 -5.917291 1 2
215
+ 207 71.0145 -2.776459 -3.003778 1 1
216
+ 208 71.1538 -4.400286 -2.887972 2 2
217
+ 209 71.2919 -4.700597 -4.241984 2 2
218
+ 210 70.9524 -7.932116 -8.602603 1 2
219
+ 211 70.6161 -5.817986 -5.431561 2 1
220
+ 212 70.7547 -4.890416 -3.565583 2 2
221
+ 213 70.4225 -5.303797 -5.933870 1 2
222
+ 214 70.5607 -2.917966 -3.133182 1 1
223
+ 215 70.6977 -5.125586 -4.583827 2 2
224
+ 216 70.3704 -3.489779 -3.316344 2 1
225
+ 217 70.5069 -1.060189 -1.796556 1 1
226
+ 218 70.6422 -2.227160 -2.175934 2 2
227
+ 219 70.7763 -2.637995 -3.030991 1 1
228
+ 220 70.9091 -3.157192 -3.015216 2 2
229
+ 221 71.0407 -2.265893 -8.129313 1 1
230
+ 222 70.7207 -0.216800 -7.303201 1 2
231
+ 223 70.8520 -4.914366 -4.796498 2 2
232
+ 224 70.5357 -5.635173 -6.192070 1 2
233
+ 225 70.6667 -4.006585 -3.013304 2 2
234
+ 226 70.7965 -2.967479 -3.702906 1 1
235
+ 227 70.4846 -3.804364 -3.563816 2 1
236
+ 228 70.6140 -1.717456 -3.473270 1 1
237
+ 229 70.7424 -4.215075 -3.954331 2 2
238
+ 230 70.8696 -3.030849 -3.328624 1 1
239
+ 231 70.9957 -1.989857 -2.409356 1 1
240
+ 232 71.1207 -2.459424 -0.957460 2 2
241
+ 233 71.2446 -1.805691 -2.221792 1 1
242
+ 234 71.3675 -4.117909 -3.219474 2 2
243
+ 235 71.4894 -3.542065 -2.838252 2 2
244
+ 236 71.1864 -2.052526 -1.748463 2 1
245
+ 237 71.3080 -1.099594 -1.484257 1 1
246
+ 238 71.4286 -3.269038 -2.908099 2 2
247
+ 239 71.5481 -2.693504 -2.300265 2 2
248
+ 240 71.2500 -7.305702 -7.746001 1 2
249
+ 241 71.3693 -4.128916 -3.215656 2 2
250
+ 242 71.4876 -6.109528 -5.845397 2 2
251
+ 243 71.1934 -4.839910 -4.702204 2 1
252
+ 244 71.3115 -1.942214 -3.654129 1 1
253
+ 245 71.4286 -4.101891 -5.395956 1 1
254
+ 246 71.5447 -2.477904 -3.129282 1 1
255
+ 247 71.2551 -4.098292 -4.908321 1 2
256
+ 248 71.3710 -3.448144 -3.102505 2 2
257
+ 249 71.4859 -3.300898 -2.019182 2 2
258
+ 250 71.6000 -3.247970 -3.984318 1 1
259
+ 251 71.7131 -0.586380 -1.011775 1 1
260
+ 252 71.8254 -6.672043 -6.367652 2 2
261
+ 253 71.9368 -5.503808 -5.618893 1 1
262
+ 254 72.0472 -4.189188 -2.982450 2 2
263
+ 255 72.1569 -2.848317 -3.732609 1 1
264
+ 256 72.2656 -3.003611 -4.189972 1 1
265
+ 257 71.9844 -2.652625 -2.009423 2 1
266
+ 258 71.7054 -3.514595 -3.811013 1 2
267
+ 259 71.8147 -4.094537 -4.334101 1 1
268
+ 260 71.9231 -4.580043 -5.161264 1 1
269
+ 261 72.0307 -3.681492 -3.534590 2 2
270
+ 262 72.1374 -5.584624 -5.178143 2 2
271
+ 263 72.2433 -4.982312 -6.695596 1 1
272
+ 264 72.3485 -4.154855 -3.980212 2 2
273
+ 265 72.4528 -1.676907 -2.385245 1 1
274
+ 266 72.5564 -6.088987 -7.541898 1 1
275
+ 267 72.6592 -3.312393 -3.337814 1 1
276
+ 268 72.3881 -2.763369 -2.540646 2 1
277
+ 269 72.4907 -3.074692 -2.969536 2 2
278
+ 270 72.5926 -1.947204 -3.449158 1 1
279
+ 271 72.6937 -3.195076 -2.932314 2 2
280
+ 272 72.7941 -4.544399 -3.933302 2 2
281
+ 273 72.8938 -3.688499 -3.490246 2 2
282
+ 274 72.9927 -2.841505 -2.177621 2 2
283
+ 275 73.0909 -1.562591 -2.449728 1 1
284
+ 276 72.8261 -2.055274 -1.973737 2 1
285
+ 277 72.5632 -2.972926 -2.934306 2 1
286
+ 278 72.6619 -2.402734 -2.448885 1 1
287
+ 279 72.7599 -4.589621 -3.078335 2 2
288
+ 280 72.8571 -3.958483 -3.793277 2 2
289
+ 281 72.9537 -2.372803 -1.922627 2 2
290
+ 282 73.0496 -4.156446 -3.714797 2 2
291
+ 283 72.7915 -3.373396 -3.374803 1 2
292
+ 284 72.5352 -2.836818 -2.458430 2 1
293
+ 285 72.6316 -5.197610 -5.477826 1 1
294
+ 286 72.7273 -3.458401 -3.618382 1 1
295
+ 287 72.8223 -2.034637 -4.395335 1 1
296
+ 288 72.9167 -3.502313 -2.375783 2 2
297
+ 289 73.0104 -2.414752 -2.515028 1 1
298
+ 290 73.1034 -5.823126 -6.390375 1 1
299
+ 291 73.1959 -1.618767 -3.122965 1 1
300
+ 292 72.9452 -1.626064 -1.693746 1 2
301
+ 293 73.0375 -3.549498 -2.902879 2 2
302
+ 294 72.7891 -4.216201 -3.453906 2 1
303
+ 295 72.8814 -1.389201 -2.011422 1 1
304
+ 296 72.9730 -4.640824 -4.548170 2 2
305
+ 297 72.7273 -1.068971 -1.212012 1 2
306
+ 298 72.4832 -5.787668 -6.630514 1 2
307
+ 299 72.5753 -2.064828 -1.677239 2 2
308
+ 300 72.6667 -2.916180 -3.738809 1 1
309
+ 301 72.7575 -4.419963 -5.175259 1 1
310
+ 302 72.5166 -4.127557 -4.138700 1 2
311
+ 303 72.2772 -8.702826 -7.960027 2 1
312
+ 304 72.3684 -3.946924 -3.461263 2 2
313
+ 305 72.1311 -2.723424 -2.894026 1 2
314
+ 306 72.2222 -2.648530 -2.295392 2 2
315
+ 307 72.3127 -3.141615 -3.334466 1 1
316
+ 308 72.0779 -4.671736 -4.919485 1 2
317
+ 309 72.1683 -3.303241 -3.931969 1 1
318
+ 310 71.9355 -4.540194 -4.622925 1 2
319
+ 311 72.0257 -1.319609 -4.634738 1 1
320
+ 312 71.7949 -4.907514 -4.104546 2 1
321
+ 313 71.5655 -4.566665 -4.923105 1 2
322
+ 314 71.6561 -6.254356 -5.507623 2 2
323
+ 315 71.7460 -2.181932 -3.553520 1 1
324
+ 316 71.8354 -6.192624 -5.245033 2 2
325
+ 317 71.6088 -5.131862 -4.695397 2 1
326
+ 318 71.6981 -5.287409 -4.305748 2 2
327
+ 319 71.7868 -1.409603 -2.153005 1 1
328
+ 320 71.8750 -4.604632 -5.227566 1 1
329
+ 321 71.9626 -2.976700 -2.890110 2 2
330
+ 322 72.0497 -2.114422 -2.102063 2 2
331
+ 323 72.1362 -2.507812 -2.964731 1 1
332
+ 324 71.9136 -4.372850 -4.176886 2 1
333
+ 325 72.0000 -5.110141 -4.480750 2 2
334
+ 326 72.0859 -2.597423 -1.834357 2 2
335
+ 327 72.1713 -2.308407 -3.256322 1 1
336
+ 328 72.2561 -3.954012 -4.264306 1 1
337
+ 329 72.3404 -3.532144 -9.531513 1 1
338
+ 330 72.4242 -0.968339 -2.054524 1 1
339
+ 331 72.2054 -5.681931 -3.751141 2 1
340
+ 332 72.2892 -1.988488 -2.836335 1 1
341
+ 333 72.3724 -3.188645 -1.946638 2 2
342
+ 334 72.4551 -2.274853 -5.371598 1 1
343
+ 335 72.2388 -4.460433 -4.242996 2 1
344
+ 336 72.0238 -3.281590 -3.955529 1 2
345
+ 337 71.8101 -1.702627 -1.761773 1 2
346
+ 338 71.5976 -4.507634 -3.394645 2 1
347
+ 339 71.6814 -5.533100 -4.886186 2 2
348
+ 340 71.7647 -5.286868 -5.539445 1 1
349
+ 341 71.8475 -2.314493 -4.457198 1 1
350
+ 342 71.9298 -6.391887 -6.231358 2 2
351
+ 343 72.0117 -3.987386 -4.427575 1 1
352
+ 344 72.0930 -2.558752 -2.806258 1 1
353
+ 345 72.1739 -0.898645 -3.310638 1 1
354
+ 346 71.9653 -4.873615 -7.255923 1 2
355
+ 347 72.0461 -4.208307 -3.634321 2 2
356
+ 348 72.1264 -0.910380 -2.810121 1 1
357
+ 349 71.9198 -3.525763 -5.539117 1 2
358
+ 350 71.7143 -1.980561 -2.860112 1 2
359
+ 351 71.7949 -2.367424 -3.372527 1 1
360
+ 352 71.8750 -3.413399 -2.604934 2 2
361
+ 353 71.6714 -1.863473 -1.657270 2 1
362
+ 354 71.7514 -5.207282 -3.532607 2 2
363
+ 355 71.8310 -2.659701 -2.182903 2 2
364
+ 356 71.6292 -3.145896 -3.300492 1 2
365
+ 357 71.7087 -4.047023 -3.565490 2 2
366
+ 358 71.7877 -2.314090 -2.050628 2 2
367
+ 359 71.8663 -6.313223 -3.562793 2 2
368
+ 360 71.6667 -2.684832 -3.077395 1 2
369
+ 361 71.7452 -2.882652 -1.387905 2 2
370
+ 362 71.8232 -5.102716 -4.754028 2 2
371
+ 363 71.6253 -2.597298 -2.529682 2 1
372
+ 364 71.7033 -5.678778 -5.839438 1 1
373
+ 365 71.7808 -5.486044 -5.993742 1 1
374
+ 366 71.5847 -3.196888 -8.989987 1 2
375
+ 367 71.6621 -5.384660 -6.039328 1 1
376
+ 368 71.7391 -3.120132 -1.399676 2 2
377
+ 369 71.8157 -3.655012 -3.587749 2 2
378
+ 370 71.6216 -5.269190 -5.252578 2 1
379
+ 371 71.6981 -4.324592 -4.415491 1 1
380
+ 372 71.5054 -2.596425 -2.213792 2 1
381
+ 373 71.5818 -5.009544 -5.144650 1 1
382
+ 374 71.6578 -1.725096 -2.841675 1 1
383
+ 375 71.7333 -3.292391 -2.706331 2 2
384
+ 376 71.8085 -5.986959 -4.581353 2 2
385
+ 377 71.6180 -3.941825 -4.249111 1 2
386
+ 378 71.4286 -3.385898 -3.444179 1 2
387
+ 379 71.2401 -0.769365 -1.565052 1 2
388
+ 380 71.0526 -8.144333 -7.526324 2 1
389
+ 381 71.1286 -11.412066 -8.430237 2 2
390
+ 382 71.2042 -2.459449 -2.441188 2 2
391
+ 383 71.2794 -2.019629 -0.459322 2 2
392
+ 384 71.0938 -3.603493 -2.579292 2 1
393
+ 385 70.9091 -1.032505 -1.028839 2 1
394
+ 386 70.9845 -1.444488 -9.759501 1 1
395
+ 387 71.0594 -1.755537 -1.887391 1 1
396
+ 388 70.8763 -5.173263 -5.990792 1 2
397
+ 389 70.9512 -2.846662 -1.916014 2 2
398
+ 390 71.0256 -2.575653 -3.816717 1 1
399
+ 391 70.8440 -3.854002 -5.437283 1 2
400
+ 392 70.6633 -5.484492 -3.867898 2 1
401
+ 393 70.7379 -7.155829 -6.826459 2 2
402
+ 394 70.8122 -1.507431 -4.482626 1 1
403
+ 395 70.6329 -4.041436 -4.057373 1 2
404
+ 396 70.4545 -5.112051 -4.816680 2 1
405
+ 397 70.2771 -7.518550 -6.960499 2 1
406
+ 398 70.3518 -3.692036 -6.882343 1 1
407
+ 399 70.1754 -4.446433 -4.304679 2 1
408
+ 400 70.2500 -2.456800 -5.457766 1 1
409
+ 401 70.0748 -2.898443 -2.893278 2 1
410
+ 402 69.9005 -4.351166 -4.692084 1 2
411
+ 403 69.9752 -7.237162 -7.208704 2 2
412
+ 404 70.0495 -4.151793 -3.149372 2 2
413
+ 405 70.1235 -2.947739 -4.088308 1 1
414
+ 406 70.1970 -8.594822 -9.804606 1 1
415
+ 407 70.2703 -5.339020 -5.258686 2 2
416
+ 408 70.3431 -3.314886 -3.166919 2 2
417
+ 409 70.4156 -3.829010 -4.140568 1 1
418
+ 410 70.4878 -4.792028 -5.414678 1 1
419
+ 411 70.5596 -5.149868 -5.448017 1 1
420
+ 412 70.6311 -4.080167 -2.854685 2 2
421
+ 413 70.7022 -3.593982 -2.697487 2 2
422
+ 414 70.7729 -4.669857 -4.667049 2 2
423
+ 415 70.8434 -2.049613 -1.768316 2 2
424
+ 416 70.9135 -4.075779 -2.075249 2 2
425
+ 417 70.9832 -4.484694 -5.531219 1 1
426
+ 418 71.0526 -5.608287 -6.896503 1 1
427
+ 419 70.8831 -4.645006 -5.039724 1 2
428
+ 420 70.7143 -3.243820 -3.391709 1 2
429
+ 421 70.7838 -4.400527 -3.704342 2 2
430
+ 422 70.6161 -2.588113 -2.192805 2 1
431
+ 423 70.4492 -3.308339 -3.149084 2 1
432
+ 424 70.5189 -5.059673 -5.563927 1 1
433
+ 425 70.5882 -4.510106 -3.302029 2 2
434
+ 426 70.4225 -1.521123 -2.004500 1 2
435
+ 427 70.4918 -4.045552 -5.293495 1 1
436
+ 428 70.3271 -4.645856 -3.584453 2 1
437
+ 429 70.1632 -3.544729 -3.601600 1 2
438
+ 430 70.0000 -5.778010 -4.963014 2 1
439
+ 431 70.0696 -5.014148 -6.112619 1 1
440
+ 432 69.9074 -4.043424 -4.613163 1 2
441
+ 433 69.9769 -1.690210 -1.629013 2 2
442
+ 434 70.0461 -6.806449 -7.302669 1 1
443
+ 435 70.1149 -3.411008 -3.676265 1 1
444
+ 436 70.1835 -0.730498 -2.370441 1 1
445
+ 437 70.2517 -3.332868 -3.244327 2 2
446
+ 438 70.0913 -3.444915 -3.459481 1 2
447
+ 439 69.9317 -3.331588 -2.591869 2 1
448
+ 440 70.0000 -4.286671 -4.937441 1 1
449
+ 441 70.0680 -3.329640 -5.148719 1 1
450
+ 442 70.1357 -5.066804 -5.176049 1 1
451
+ 443 70.2032 -1.652006 -3.673936 1 1
452
+ 444 70.0450 -4.916903 -6.325884 1 2
453
+ 445 70.1124 -1.461592 -2.684252 1 1
454
+ 446 70.1794 -4.597349 -3.811211 2 2
455
+ 447 70.0224 -2.083045 -1.958917 2 1
456
+ 448 70.0893 -3.869832 -3.116522 2 2
457
+ 449 69.9332 -2.403824 -2.574322 1 2
458
+ 450 70.0000 -4.357641 -3.282457 2 2
459
+ 451 70.0665 -4.033966 -3.372191 2 2
460
+ 452 70.1327 -4.899635 -5.184809 1 1
461
+ 453 70.1987 -4.701832 -4.553805 2 2
462
+ 454 70.2643 -3.241464 -3.601138 1 1
463
+ 455 70.1099 -1.320954 -7.999351 1 2
464
+ 456 70.1754 -4.743350 -4.445738 2 2
465
+ 457 70.0219 -10.772182 -9.824441 2 1
466
+ 458 70.0873 -4.403267 -3.639236 2 2
467
+ 459 70.1525 -4.526957 -4.377284 2 2
468
+ 460 70.2174 -0.158614 -7.290477 1 1
469
+ 461 70.2820 -2.582708 -3.640149 1 1
470
+ 462 70.1299 -5.334623 -5.122603 2 1
471
+ 463 70.1944 -1.117184 -1.168725 1 1
472
+ 464 70.0431 -2.879795 -2.827806 2 1
473
+ 465 69.8925 -1.788113 -1.921502 1 2
474
+ 466 69.9571 -5.427373 -4.941589 2 2
475
+ 467 70.0214 -5.078450 -5.505743 1 1
476
+ 468 70.0855 -5.309601 -4.152256 2 2
477
+ 469 69.9360 -1.729138 -1.157794 2 1
478
+ 470 70.0000 -1.425021 -1.180133 2 2
479
+ 471 69.8514 -2.222095 -2.073794 2 1
480
+ 472 69.9153 -4.293944 -3.854010 2 2
481
+ 473 69.9789 -4.034888 -2.564639 2 2
482
+ 474 70.0422 -4.271921 -4.731838 1 1
483
+ 475 70.1053 -1.120350 -1.006868 2 2
484
+ 476 70.1681 -4.730613 -4.592992 2 2
485
+ 477 70.0210 -4.341600 -5.001170 1 2
486
+ 478 69.8745 -4.296506 -3.837858 2 1
487
+ 479 69.9374 -3.181595 -3.291827 1 1
488
+ 480 69.7917 -5.204616 -5.491075 1 2
489
+ 481 69.8545 -2.048047 -2.541311 1 1
490
+ 482 69.7095 -4.729505 -5.467352 1 2
491
+ 483 69.5652 -4.998275 -4.188862 2 1
492
+ 484 69.4215 -2.838273 -1.720777 2 1
493
+ 485 69.4845 -1.672997 -2.679798 1 1
494
+ 486 69.5473 -2.378071 -3.017376 1 1
495
+ 487 69.6099 -2.794112 -2.059233 2 2
496
+ 488 69.6721 -3.555603 -2.469780 2 2
497
+ 489 69.7342 -1.795865 -2.211408 1 1
498
+ 490 69.7959 -3.330091 -3.820684 1 1
499
+ 491 69.8574 -5.224546 -4.252823 2 2
500
+ 492 69.9187 -5.673110 -5.095194 2 2
501
+ 493 69.9797 -1.029051 -2.548517 1 1
502
+ 494 70.0405 -4.236117 -4.053613 2 2
503
+ 495 70.1010 -4.104562 -4.257297 1 1
504
+ 496 69.9597 -3.357428 -3.376741 1 2
505
+ 497 69.8189 -5.511664 -5.065009 2 1
506
+ 498 69.6787 -1.336086 -1.817140 1 2
507
+ 499 69.7395 -2.898631 -3.443978 1 1
508
+ 500 69.8000 -5.518026 -4.144943 2 2
509
+ 501 69.8603 -0.685472 -1.496915 1 1
510
+ 502 69.9203 -1.316462 -1.669493 1 1
511
+ 503 69.9801 -2.735606 -2.722636 2 2
512
+ 504 70.0397 -2.552075 -2.923025 1 1
513
+ 505 69.9010 -8.530472 -8.888788 1 2
514
+ 506 69.9605 -5.393888 -5.419087 1 1
515
+ 507 70.0197 -4.374657 -4.424517 1 1
516
+ 508 70.0787 -6.499279 -3.319683 2 2
517
+ 509 70.1375 -2.962125 -8.340458 1 1
518
+ 510 70.0000 -3.562418 -3.388553 2 1
519
+ 511 70.0587 -5.023645 -3.667746 2 2
520
+ 512 70.1172 -3.436833 -3.855163 1 1
521
+ 513 70.1754 -3.564609 -3.476646 2 2
522
+ 514 70.0389 -5.024211 -4.617437 2 1
523
+ 515 70.0971 -6.677651 -7.066770 1 1
524
+ 516 70.1550 -4.327910 -4.373932 1 1
525
+ 517 70.2128 -1.593675 -1.601542 1 1
526
+ 518 70.2703 -2.421230 -1.848001 2 2
527
+ 519 70.3276 -9.689257 -9.447565 2 2
528
+ 520 70.3846 -2.687063 -2.019567 2 2
529
+ 521 70.4415 -0.893570 -1.114365 1 1
530
+ 522 70.4981 -3.828295 -3.031498 2 2
531
+ 523 70.5545 -4.124588 -4.113674 2 2
532
+ 524 70.4198 -6.959944 -6.863920 2 1
533
+ 525 70.4762 -4.749471 -2.038346 2 2
534
+ 526 70.5323 -0.482199 -16.879553 1 1
535
+ 527 70.5882 -6.124587 -4.497903 2 2
536
+ 528 70.6439 -6.636361 -6.169066 2 2
537
+ 529 70.6994 -4.941807 -6.107344 1 1
538
+ 530 70.7547 -5.618569 -4.484121 2 2
539
+ 531 70.6215 -3.772657 -3.499367 2 1
540
+ 532 70.6767 -2.350006 -3.187421 1 1
541
+ 533 70.7317 -3.385371 -2.756266 2 2
542
+ 534 70.7865 -2.633723 -3.295274 1 1
543
+ 535 70.6542 -3.635625 -3.757138 1 2
544
+ 536 70.5224 -2.437700 -2.216087 2 1
545
+ 537 70.5773 -5.695769 -6.063893 1 1
546
+ 538 70.6320 -1.922484 -2.635114 1 1
547
+ 539 70.6865 -4.158110 -2.517257 2 2
548
+ 540 70.5556 -2.085053 -2.635058 1 2
549
+ 541 70.6100 -2.411968 -2.506402 1 1
550
+ 542 70.4797 -3.504957 -4.343110 1 2
551
+ 543 70.5341 -5.248216 -4.665016 2 2
552
+ 544 70.5882 -2.562424 -3.917785 1 1
553
+ 545 70.4587 -1.452251 -4.796637 1 2
554
+ 546 70.5128 -2.870189 -3.730812 1 1
555
+ 547 70.5667 -6.584022 -5.312266 2 2
556
+ 548 70.6204 -4.104489 -4.307259 1 1
557
+ 549 70.4918 -5.180824 -5.081308 2 1
558
+ 550 70.5455 -3.414137 -3.356222 2 2
559
+ 551 70.4174 -1.802784 -7.645470 1 2
560
+ 552 70.4710 -2.091669 -2.828665 1 1
561
+ 553 70.5244 -6.374506 -5.374364 2 2
562
+ 554 70.3971 -4.640583 -5.005713 1 2
563
+ 555 70.4505 -2.705012 -1.903875 2 2
564
+ 556 70.5036 -5.782387 -4.453732 2 2
565
+ 557 70.5566 -4.354425 -5.494856 1 1
566
+ 558 70.6093 -1.425357 -3.371072 1 1
567
+ 559 70.6619 -2.400806 -2.364669 2 2
568
+ 560 70.5357 -2.107309 -1.968660 2 1
569
+ 561 70.5882 -2.074645 -2.463257 1 1
570
+ 562 70.6406 -0.400646 -1.400774 1 1
571
+ 563 70.5151 -1.299155 -2.517965 1 2
572
+ 564 70.5674 -3.036172 -3.676497 1 1
573
+ 565 70.6195 -3.326687 -3.136685 2 2
574
+ 566 70.6714 -4.433135 -5.564830 1 1
575
+ 567 70.7231 -2.599395 -2.361403 2 2
576
+ 568 70.7746 -6.797820 -4.427729 2 2
577
+ 569 70.6503 -5.926931 -5.248518 2 1
578
+ 570 70.7018 -5.398033 -6.221379 1 1
579
+ 571 70.7531 -4.766172 -3.240433 2 2
580
+ 572 70.8042 -2.552889 -2.466470 2 2
581
+ 573 70.6806 -3.755174 -4.260272 1 2
582
+ 574 70.7317 -3.416668 -2.621783 2 2
583
+ 575 70.7826 -4.934521 -4.451932 2 2
584
+ 576 70.8333 -2.912508 -3.600003 1 1
585
+ 577 70.8839 -4.334430 -4.221344 2 2
586
+ 578 70.9343 -3.309745 -2.996025 2 2
587
+ 579 70.9845 -4.849188 -4.823846 2 2
588
+ 580 71.0345 -1.827028 -3.384270 1 1
589
+ 581 71.0843 -6.107439 -8.457978 1 1
590
+ 582 70.9622 -5.238088 -5.610653 1 2
591
+ 583 71.0120 -4.062747 -4.020084 2 2
592
+ 584 70.8904 -4.890682 -4.585644 2 1
593
+ 585 70.9402 -0.498656 -1.003168 1 1
594
+ 586 70.9898 -3.837120 -4.092930 1 1
595
+ 587 70.8688 -4.488721 -4.127244 2 1
596
+ 588 70.9184 -4.668731 -2.271761 2 2
597
+ 589 70.7980 -3.216327 -3.769351 1 2
598
+ 590 70.8475 -4.304127 -3.792610 2 2
599
+ 591 70.7276 -3.528735 -4.056598 1 2
600
+ 592 70.7770 -3.436164 -4.257457 1 1
601
+ 593 70.8263 -4.470758 -4.303078 2 2
602
+ 594 70.7071 -4.789356 -4.083279 2 1
603
+ 595 70.5882 -1.825129 -1.855348 1 2
604
+ 596 70.6376 -5.007508 -4.883888 2 2
605
+ 597 70.6868 -2.333527 -2.362035 1 1
606
+ 598 70.7358 -3.084201 -3.380183 1 1
607
+ 599 70.7846 -3.115813 -4.551075 1 1
608
+ 600 70.8333 -3.704764 -3.660796 2 2
609
+ 601 70.8819 -3.214335 -2.542371 2 2
610
+ 602 70.9302 -2.473015 -4.509099 1 1
611
+ 603 70.9784 -6.508646 -4.955961 2 2
612
+ 604 70.8609 -3.033313 -3.035506 1 2
613
+ 605 70.7438 -2.527936 -1.699420 2 1
614
+ 606 70.7921 -3.178909 -3.002276 2 2
615
+ 607 70.6755 -2.776672 -2.583925 2 1
616
+ 608 70.7237 -2.301309 -3.389185 1 1
617
+ 609 70.6076 -7.784964 -7.412727 2 1
618
+ 610 70.6557 -2.799413 -3.061914 1 1
619
+ 611 70.5401 -5.124508 -5.026257 2 1
620
+ 612 70.5882 -5.157837 -6.634867 1 1
621
+ 613 70.6362 -2.864246 -2.096493 2 2
622
+ 614 70.6840 -5.569239 -5.123873 2 2
623
+ 615 70.7317 -2.819040 -2.362040 2 2
624
+ 616 70.6169 -5.763797 -5.796136 1 2
625
+ 617 70.6645 -10.502280 -5.536719 2 2
626
+ 618 70.5502 -3.685071 -3.950779 1 2
627
+ 619 70.5977 -3.948876 -3.421469 2 2
628
+ 620 70.4839 -5.243181 -4.260417 2 1
629
+ 621 70.5314 -2.991943 -1.938521 2 2
630
+ 622 70.5788 -3.509234 -3.961004 1 1
631
+ 623 70.6260 -2.729832 -2.738378 1 1
632
+ 624 70.6731 -4.754392 -4.009025 2 2
633
+ 625 70.7200 -3.556132 -5.790721 1 1
634
+ 626 70.6070 -1.370895 -0.737743 2 1
635
+ 627 70.6539 -3.357903 -4.231627 1 1
636
+ 628 70.7006 -2.717593 -2.268391 2 2
637
+ 629 70.7472 -1.955891 -2.474249 1 1
638
+ 630 70.7937 -0.884401 -1.536693 1 1
639
+ 631 70.8399 -2.155596 -3.670567 1 1
640
+ 632 70.8861 -1.673439 -1.231469 2 2
641
+ 633 70.9321 -1.408344 -1.359027 2 2
642
+ 634 70.9779 -2.343346 -3.366927 1 1
643
+ 635 70.8661 -5.119327 -5.428386 1 2
644
+ 636 70.9119 -1.030618 -2.999222 1 1
645
+ 637 70.9576 -3.234731 -5.631665 1 1
646
+ 638 71.0031 -4.861992 -5.820968 1 1
647
+ 639 71.0485 -1.178170 -2.141917 1 1
648
+ 640 71.0938 -1.530946 -1.347403 2 2
649
+ 641 70.9828 -7.202188 -6.012979 2 1
650
+ 642 71.0280 -4.663561 -3.519462 2 2
651
+ 643 71.0731 -1.227913 -0.728365 2 2
652
+ 644 71.1180 -3.109253 -4.067436 1 1
653
+ 645 71.1628 -1.237167 -3.526211 1 1
654
+ 646 71.0526 -4.242104 -3.979429 2 1
655
+ 647 70.9428 -3.523831 -2.392741 2 1
656
+ 648 70.9877 -3.114791 -2.840316 2 2
657
+ 649 71.0324 -3.467635 -3.894268 1 1
658
+ 650 71.0769 -3.481384 -4.193570 1 1
659
+ 651 70.9677 -6.170937 -6.460481 1 2
660
+ 652 71.0123 -5.295757 -4.655994 2 2
661
+ 653 71.0567 -4.934487 -3.607936 2 2
662
+ 654 70.9480 -3.353817 -4.112107 1 2
663
+ 655 70.8397 -2.296469 -2.412515 1 2
664
+ 656 70.8841 -4.045520 -2.710417 2 2
665
+ 657 70.7763 -5.543006 -5.768118 1 2
666
+ 658 70.8207 -3.622684 -1.442844 2 2
667
+ 659 70.8649 -2.257070 -1.552161 2 2
668
+ 660 70.7576 -2.889176 -3.431077 1 2
669
+ 661 70.8018 -3.002466 -3.228163 1 1
670
+ 662 70.8459 -2.959324 -4.317432 1 1
671
+ 663 70.8899 -6.729429 -6.693894 2 2
672
+ 664 70.7831 -3.543148 -3.435778 2 1
673
+ 665 70.6767 -3.134355 -3.362544 1 2
674
+ 666 70.7207 -3.796916 -4.875072 1 1
675
+ 667 70.7646 -9.127570 -6.923685 2 2
676
+ 668 70.8084 -1.628428 -1.681859 1 1
677
+ 669 70.8520 -2.951355 -3.211749 1 1
678
+ 670 70.7463 -3.476868 -3.103801 2 1
679
+ 671 70.7899 -3.282904 -4.700931 1 1
680
+ 672 70.8333 -2.591162 -2.414384 2 2
681
+ 673 70.8767 -1.873711 -1.692885 2 2
682
+ 674 70.7715 -5.498070 -4.736485 2 1
683
+ 675 70.8148 -3.497882 -3.596659 1 1
684
+ 676 70.7101 -2.116014 -2.642191 1 2
685
+ 677 70.7533 -3.921953 -4.427319 1 1
686
+ 678 70.6490 -5.531615 -5.914170 1 2
687
+ 679 70.6922 -5.831302 -5.034846 2 2
688
+ 680 70.7353 -3.751388 -4.616244 1 1
689
+ 681 70.7783 -2.983458 -3.122614 1 1
690
+ 682 70.8211 -1.976952 -1.936428 2 2
691
+ 683 70.8638 -1.590330 -2.753879 1 1
692
+ 684 70.9064 -4.847046 -7.971983 1 1
693
+ 685 70.9489 -4.539532 -2.442572 2 2
694
+ 686 70.9913 -5.853121 -5.022080 2 2
695
+ 687 71.0335 -4.005438 -3.018363 2 2
696
+ 688 71.0756 -8.069085 -8.214204 1 1
697
+ 689 71.1176 -2.915359 -2.297020 2 2
698
+ 690 71.0145 -2.431536 -2.604404 1 2
699
+ 691 70.9117 -2.651201 -2.193406 2 1
700
+ 692 70.9538 -1.727211 -2.635116 1 1
701
+ 693 70.9957 -4.756930 -5.300750 1 1
702
+ 694 71.0375 -3.139235 -3.077096 2 2
703
+ 695 71.0791 -3.996683 -4.618560 1 1
704
+ 696 71.1207 -4.837161 -5.319623 1 1
705
+ 697 71.1621 -2.173181 -2.636428 1 1
706
+ 698 71.2034 -4.394722 -3.913301 2 2
707
+ 699 71.2446 -3.432199 -4.773718 1 1
708
+ 700 71.2857 -3.952192 -3.402476 2 2
709
+ 701 71.1840 -2.694081 -2.270214 2 1
710
+ 702 71.0826 -4.090547 -4.973228 1 2
711
+ 703 70.9815 -3.529938 -3.506216 2 1
712
+ 704 71.0227 -1.123170 -1.422539 1 1
713
+ 705 70.9220 -3.194118 -3.466463 1 2
714
+ 706 70.9632 -3.385561 -3.373310 2 2
715
+ 707 70.8628 -4.385662 -4.307924 2 1
716
+ 708 70.9040 -3.950245 -2.815328 2 2
717
+ 709 70.9450 -1.970840 -1.622243 2 2
718
+ 710 70.9859 -3.610569 -4.461593 1 1
719
+ 711 70.8861 -1.566838 -2.087878 1 2
720
+ 712 70.7865 -2.460367 -4.186742 1 2
721
+ 713 70.8275 -3.379189 -3.351597 2 2
722
+ 714 70.8683 -3.081346 -3.049501 2 2
723
+ 715 70.9091 -4.689781 -3.680485 2 2
724
+ 716 70.9497 -4.662166 -4.648278 2 2
725
+ 717 70.9902 -1.732030 -0.205748 2 2
726
+ 718 71.0306 -5.076797 -5.343299 1 1
727
+ 719 70.9318 -4.894451 -4.825175 2 1
728
+ 720 70.8333 -2.231647 -4.799362 1 2
729
+ 721 70.7351 -4.750460 -5.635036 1 2
730
+ 722 70.6371 -2.396529 -2.107147 2 1
731
+ 723 70.6777 -2.277340 -2.683260 1 1
732
+ 724 70.7182 -4.161343 -4.127849 2 2
733
+ 725 70.6207 -2.100264 -2.795871 1 2
734
+ 726 70.6612 -4.442380 -4.158481 2 2
735
+ 727 70.7015 -3.131406 -2.525559 2 2
736
+ 728 70.6044 -3.538441 -3.387502 2 1
737
+ 729 70.6447 -1.154430 -2.440291 1 1
738
+ 730 70.6849 -3.915323 -4.521620 1 1
739
+ 731 70.5882 -4.902089 -4.532334 2 1
740
+ 732 70.6284 -3.120808 -2.360277 2 2
741
+ 733 70.6685 -6.726391 -6.480646 2 2
742
+ 734 70.7084 -1.284283 -11.671963 1 1
743
+ 735 70.6122 -3.674076 -2.976214 2 1
744
+ 736 70.6522 -1.425272 -2.531844 1 1
745
+ 737 70.6920 -4.644798 -5.121528 1 1
746
+ 738 70.5962 -4.103679 -4.661181 1 2
747
+ 739 70.6360 -0.783295 -7.490164 1 1
748
+ 740 70.6757 -4.476132 -3.235666 2 2
749
+ 741 70.5803 -4.948819 -4.177369 2 1
750
+ 742 70.6199 -2.484437 -2.171486 2 2
751
+ 743 70.5249 -3.914308 -3.931680 1 2
752
+ 744 70.5645 -4.853261 -4.080504 2 2
753
+ 745 70.6040 -2.709201 -3.656614 1 1
754
+ 746 70.5094 -7.741308 -7.313328 2 1
755
+ 747 70.5489 -5.069568 -6.502689 1 1
756
+ 748 70.5882 -4.395025 -3.378073 2 2
757
+ 749 70.6275 -3.721605 -4.552577 1 1
758
+ 750 70.6667 -3.502844 -9.052788 1 1
759
+
760
+ Final Winogrande score(750 tasks): 70.6667 +/- 1.6636
scores/Hammer2.1-7b-q3_k_l.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 66.66666667
20
+ 10 70.00000000
21
+ 11 63.63636364
22
+ 12 66.66666667
23
+ 13 69.23076923
24
+ 14 71.42857143
25
+ 15 73.33333333
26
+ 16 68.75000000
27
+ 17 70.58823529
28
+ 18 66.66666667
29
+ 19 63.15789474
30
+ 20 60.00000000
31
+ 21 57.14285714
32
+ 22 59.09090909
33
+ 23 60.86956522
34
+ 24 62.50000000
35
+ 25 60.00000000
36
+ 26 57.69230769
37
+ 27 59.25925926
38
+ 28 57.14285714
39
+ 29 58.62068966
40
+ 30 60.00000000
41
+ 31 58.06451613
42
+ 32 56.25000000
43
+ 33 57.57575758
44
+ 34 55.88235294
45
+ 35 57.14285714
46
+ 36 55.55555556
47
+ 37 56.75675676
48
+ 38 55.26315789
49
+ 39 56.41025641
50
+ 40 55.00000000
51
+ 41 56.09756098
52
+ 42 57.14285714
53
+ 43 55.81395349
54
+ 44 54.54545455
55
+ 45 53.33333333
56
+ 46 54.34782609
57
+ 47 55.31914894
58
+ 48 56.25000000
59
+ 49 55.10204082
60
+ 50 54.00000000
61
+ 51 52.94117647
62
+ 52 53.84615385
63
+ 53 54.71698113
64
+ 54 53.70370370
65
+ 55 52.72727273
66
+ 56 53.57142857
67
+ 57 54.38596491
68
+ 58 55.17241379
69
+ 59 55.93220339
70
+ 60 56.66666667
71
+ 61 55.73770492
72
+ 62 56.45161290
73
+ 63 57.14285714
74
+ 64 57.81250000
75
+ 65 56.92307692
76
+ 66 57.57575758
77
+ 67 58.20895522
78
+ 68 58.82352941
79
+ 69 57.97101449
80
+ 70 57.14285714
81
+ 71 56.33802817
82
+ 72 55.55555556
83
+ 73 56.16438356
84
+ 74 56.75675676
85
+ 75 57.33333333
86
+ 76 56.57894737
87
+ 77 57.14285714
88
+ 78 57.69230769
89
+ 79 56.96202532
90
+ 80 57.50000000
91
+ 81 56.79012346
92
+ 82 57.31707317
93
+ 83 57.83132530
94
+ 84 57.14285714
95
+ 85 57.64705882
96
+ 86 58.13953488
97
+ 87 58.62068966
98
+ 88 59.09090909
99
+ 89 59.55056180
100
+ 90 58.88888889
101
+ 91 59.34065934
102
+ 92 59.78260870
103
+ 93 59.13978495
104
+ 94 58.51063830
105
+ 95 58.94736842
106
+ 96 59.37500000
107
+ 97 58.76288660
108
+ 98 58.16326531
109
+ 99 58.58585859
110
+ 100 59.00000000
111
+ 101 58.41584158
112
+ 102 58.82352941
113
+ 103 58.25242718
114
+ 104 57.69230769
115
+ 105 57.14285714
116
+ 106 56.60377358
117
+ 107 57.00934579
118
+ 108 56.48148148
119
+ 109 55.96330275
120
+ 110 56.36363636
121
+ 111 56.75675676
122
+ 112 57.14285714
123
+ 113 56.63716814
124
+ 114 56.14035088
125
+ 115 56.52173913
126
+ 116 56.03448276
127
+ 117 55.55555556
128
+ 118 55.93220339
129
+ 119 55.46218487
130
+ 120 55.83333333
131
+ 121 55.37190083
132
+ 122 55.73770492
133
+ 123 56.09756098
134
+ 124 55.64516129
135
+ 125 56.00000000
136
+ 126 56.34920635
137
+ 127 55.90551181
138
+ 128 56.25000000
139
+ 129 55.81395349
140
+ 130 55.38461538
141
+ 131 55.72519084
142
+ 132 56.06060606
143
+ 133 56.39097744
144
+ 134 56.71641791
145
+ 135 57.03703704
146
+ 136 56.61764706
147
+ 137 56.93430657
148
+ 138 57.24637681
149
+ 139 57.55395683
150
+ 140 57.85714286
151
+ 141 58.15602837
152
+ 142 57.74647887
153
+ 143 58.04195804
154
+ 144 57.63888889
155
+ 145 57.93103448
156
+ 146 57.53424658
157
+ 147 57.82312925
158
+ 148 57.43243243
159
+ 149 57.71812081
160
+ 150 58.00000000
161
+ 151 58.27814570
162
+ 152 58.55263158
163
+ 153 58.82352941
164
+ 154 58.44155844
165
+ 155 58.06451613
166
+ 156 58.33333333
167
+ 157 58.59872611
168
+ 158 58.86075949
169
+ 159 59.11949686
170
+ 160 58.75000000
171
+ 161 58.38509317
172
+ 162 58.64197531
173
+ 163 58.28220859
174
+ 164 57.92682927
175
+ 165 58.18181818
176
+ 166 58.43373494
177
+ 167 58.08383234
178
+ 168 58.33333333
179
+ 169 58.57988166
180
+ 170 58.82352941
181
+ 171 58.47953216
182
+ 172 58.72093023
183
+ 173 58.95953757
184
+ 174 59.19540230
185
+ 175 58.85714286
186
+ 176 58.52272727
187
+ 177 58.75706215
188
+ 178 58.42696629
189
+ 179 58.10055866
190
+ 180 57.77777778
191
+ 181 58.01104972
192
+ 182 58.24175824
193
+ 183 57.92349727
194
+ 184 57.60869565
195
+ 185 57.83783784
196
+ 186 58.06451613
197
+ 187 58.28877005
198
+ 188 57.97872340
199
+ 189 57.67195767
200
+ 190 57.36842105
201
+ 191 57.59162304
202
+ 192 57.29166667
203
+ 193 56.99481865
204
+ 194 57.21649485
205
+ 195 57.43589744
206
+ 196 57.65306122
207
+ 197 57.86802030
208
+ 198 57.57575758
209
+ 199 57.28643216
210
+ 200 57.50000000
211
+ 201 57.71144279
212
+ 202 57.92079208
213
+ 203 58.12807882
214
+ 204 58.33333333
215
+ 205 58.04878049
216
+ 206 57.76699029
217
+ 207 57.97101449
218
+ 208 58.17307692
219
+ 209 58.37320574
220
+ 210 58.57142857
221
+ 211 58.76777251
222
+ 212 58.49056604
223
+ 213 58.21596244
224
+ 214 58.41121495
225
+ 215 58.60465116
226
+ 216 58.33333333
227
+ 217 58.06451613
228
+ 218 57.79816514
229
+ 219 57.99086758
230
+ 220 58.18181818
231
+ 221 57.91855204
232
+ 222 58.10810811
233
+ 223 57.84753363
234
+ 224 58.03571429
235
+ 225 57.77777778
236
+ 226 57.52212389
237
+ 227 57.70925110
238
+ 228 57.89473684
239
+ 229 58.07860262
240
+ 230 58.26086957
241
+ 231 58.00865801
242
+ 232 57.75862069
243
+ 233 57.93991416
244
+ 234 58.11965812
245
+ 235 57.87234043
246
+ 236 58.05084746
247
+ 237 57.80590717
248
+ 238 57.56302521
249
+ 239 57.74058577
250
+ 240 57.91666667
251
+ 241 58.09128631
252
+ 242 57.85123967
253
+ 243 57.61316872
254
+ 244 57.37704918
255
+ 245 57.55102041
256
+ 246 57.72357724
257
+ 247 57.89473684
258
+ 248 58.06451613
259
+ 249 58.23293173
260
+ 250 58.40000000
261
+ 251 58.56573705
262
+ 252 58.73015873
263
+ 253 58.89328063
264
+ 254 58.66141732
265
+ 255 58.82352941
266
+ 256 58.98437500
267
+ 257 59.14396887
268
+ 258 59.30232558
269
+ 259 59.07335907
270
+ 260 59.23076923
271
+ 261 59.38697318
272
+ 262 59.54198473
273
+ 263 59.69581749
274
+ 264 59.84848485
275
+ 265 59.62264151
276
+ 266 59.39849624
277
+ 267 59.55056180
278
+ 268 59.32835821
279
+ 269 59.47955390
280
+ 270 59.62962963
281
+ 271 59.77859779
282
+ 272 59.55882353
283
+ 273 59.34065934
284
+ 274 59.12408759
285
+ 275 58.90909091
286
+ 276 59.05797101
287
+ 277 59.20577617
288
+ 278 58.99280576
289
+ 279 59.13978495
290
+ 280 59.28571429
291
+ 281 59.43060498
292
+ 282 59.57446809
293
+ 283 59.36395760
294
+ 284 59.50704225
295
+ 285 59.64912281
296
+ 286 59.79020979
297
+ 287 59.58188153
298
+ 288 59.72222222
299
+ 289 59.51557093
300
+ 290 59.65517241
301
+ 291 59.79381443
302
+ 292 59.58904110
303
+ 293 59.72696246
304
+ 294 59.52380952
305
+ 295 59.66101695
306
+ 296 59.79729730
307
+ 297 59.93265993
308
+ 298 60.06711409
309
+ 299 60.20066890
310
+ 300 60.33333333
311
+ 301 60.13289037
312
+ 302 60.26490066
313
+ 303 60.39603960
314
+ 304 60.52631579
315
+ 305 60.65573770
316
+ 306 60.45751634
317
+ 307 60.58631922
318
+ 308 60.71428571
319
+ 309 60.84142395
320
+ 310 60.64516129
321
+ 311 60.77170418
322
+ 312 60.57692308
323
+ 313 60.70287540
324
+ 314 60.50955414
325
+ 315 60.63492063
326
+ 316 60.75949367
327
+ 317 60.88328076
328
+ 318 60.69182390
329
+ 319 60.50156740
330
+ 320 60.31250000
331
+ 321 60.12461059
332
+ 322 60.24844720
333
+ 323 60.06191950
334
+ 324 59.87654321
335
+ 325 59.69230769
336
+ 326 59.81595092
337
+ 327 59.63302752
338
+ 328 59.75609756
339
+ 329 59.87841945
340
+ 330 60.00000000
341
+ 331 60.12084592
342
+ 332 60.24096386
343
+ 333 60.36036036
344
+ 334 60.47904192
345
+ 335 60.29850746
346
+ 336 60.41666667
347
+ 337 60.23738872
348
+ 338 60.05917160
349
+ 339 59.88200590
350
+ 340 60.00000000
351
+ 341 60.11730205
352
+ 342 60.23391813
353
+ 343 60.34985423
354
+ 344 60.46511628
355
+ 345 60.28985507
356
+ 346 60.40462428
357
+ 347 60.23054755
358
+ 348 60.05747126
359
+ 349 60.17191977
360
+ 350 60.00000000
361
+ 351 60.11396011
362
+ 352 60.22727273
363
+ 353 60.33994334
364
+ 354 60.16949153
365
+ 355 60.00000000
366
+ 356 59.83146067
367
+ 357 59.66386555
368
+ 358 59.49720670
369
+ 359 59.33147632
370
+ 360 59.44444444
371
+ 361 59.55678670
372
+ 362 59.66850829
373
+ 363 59.77961433
374
+ 364 59.61538462
375
+ 365 59.45205479
376
+ 366 59.28961749
377
+ 367 59.40054496
378
+ 368 59.51086957
379
+ 369 59.62059621
380
+ 370 59.72972973
381
+ 371 59.83827493
382
+ 372 59.67741935
383
+ 373 59.51742627
384
+ 374 59.62566845
385
+ 375 59.73333333
386
+ 376 59.84042553
387
+ 377 59.68169761
388
+ 378 59.78835979
389
+ 379 59.89445910
390
+ 380 60.00000000
391
+ 381 59.84251969
392
+ 382 59.68586387
393
+ 383 59.53002611
394
+ 384 59.63541667
395
+ 385 59.74025974
396
+ 386 59.84455959
397
+ 387 59.68992248
398
+ 388 59.53608247
399
+ 389 59.38303342
400
+ 390 59.23076923
401
+ 391 59.07928389
402
+ 392 59.18367347
403
+ 393 59.03307888
404
+ 394 59.13705584
405
+ 395 59.24050633
406
+ 396 59.09090909
407
+ 397 59.19395466
408
+ 398 59.04522613
409
+ 399 59.14786967
410
+ 400 59.25000000
411
+ 401 59.35162095
412
+ 402 59.45273632
413
+ 403 59.55334988
414
+ 404 59.65346535
415
+ 405 59.75308642
416
+ 406 59.85221675
417
+ 407 59.95085995
418
+ 408 60.04901961
419
+ 409 59.90220049
420
+ 410 60.00000000
421
+ 411 60.09732360
422
+ 412 60.19417476
423
+ 413 60.04842615
424
+ 414 60.14492754
425
+ 415 60.00000000
426
+ 416 59.85576923
427
+ 417 59.71223022
428
+ 418 59.80861244
429
+ 419 59.66587112
430
+ 420 59.52380952
431
+ 421 59.38242280
432
+ 422 59.47867299
433
+ 423 59.57446809
434
+ 424 59.43396226
435
+ 425 59.52941176
436
+ 426 59.38967136
437
+ 427 59.48477752
438
+ 428 59.34579439
439
+ 429 59.44055944
440
+ 430 59.30232558
441
+ 431 59.39675174
442
+ 432 59.49074074
443
+ 433 59.35334873
444
+ 434 59.44700461
445
+ 435 59.54022989
446
+ 436 59.63302752
447
+ 437 59.49656751
448
+ 438 59.36073059
449
+ 439 59.45330296
450
+ 440 59.31818182
451
+ 441 59.41043084
452
+ 442 59.27601810
453
+ 443 59.14221219
454
+ 444 59.23423423
455
+ 445 59.32584270
456
+ 446 59.41704036
457
+ 447 59.50782998
458
+ 448 59.37500000
459
+ 449 59.46547884
460
+ 450 59.33333333
461
+ 451 59.42350333
462
+ 452 59.51327434
463
+ 453 59.38189845
464
+ 454 59.25110132
465
+ 455 59.34065934
466
+ 456 59.21052632
467
+ 457 59.29978118
468
+ 458 59.38864629
469
+ 459 59.47712418
470
+ 460 59.34782609
471
+ 461 59.21908894
472
+ 462 59.30735931
473
+ 463 59.17926566
474
+ 464 59.05172414
475
+ 465 58.92473118
476
+ 466 59.01287554
477
+ 467 59.10064240
478
+ 468 59.18803419
479
+ 469 59.06183369
480
+ 470 59.14893617
481
+ 471 59.23566879
482
+ 472 59.11016949
483
+ 473 58.98520085
484
+ 474 59.07172996
485
+ 475 58.94736842
486
+ 476 58.82352941
487
+ 477 58.90985325
488
+ 478 58.99581590
489
+ 479 59.08141962
490
+ 480 59.16666667
491
+ 481 59.04365904
492
+ 482 59.12863071
493
+ 483 59.21325052
494
+ 484 59.09090909
495
+ 485 59.17525773
496
+ 486 59.25925926
497
+ 487 59.13757700
498
+ 488 59.22131148
499
+ 489 59.10020450
500
+ 490 58.97959184
501
+ 491 58.85947047
502
+ 492 58.94308943
503
+ 493 58.82352941
504
+ 494 58.90688259
505
+ 495 58.78787879
506
+ 496 58.87096774
507
+ 497 58.95372233
508
+ 498 59.03614458
509
+ 499 59.11823647
510
+ 500 59.20000000
511
+ 501 59.28143713
512
+ 502 59.36254980
513
+ 503 59.44333996
514
+ 504 59.32539683
515
+ 505 59.40594059
516
+ 506 59.28853755
517
+ 507 59.17159763
518
+ 508 59.05511811
519
+ 509 58.93909627
520
+ 510 59.01960784
521
+ 511 58.90410959
522
+ 512 58.98437500
523
+ 513 58.86939571
524
+ 514 58.94941634
525
+ 515 59.02912621
526
+ 516 58.91472868
527
+ 517 58.99419729
528
+ 518 59.07335907
529
+ 519 58.95953757
530
+ 520 58.84615385
531
+ 521 58.92514395
532
+ 522 59.00383142
533
+ 523 58.89101338
534
+ 524 58.77862595
535
+ 525 58.66666667
536
+ 526 58.55513308
537
+ 527 58.44402277
538
+ 528 58.33333333
539
+ 529 58.22306238
540
+ 530 58.30188679
541
+ 531 58.19209040
542
+ 532 58.27067669
543
+ 533 58.34896811
544
+ 534 58.23970037
545
+ 535 58.13084112
546
+ 536 58.20895522
547
+ 537 58.28677840
548
+ 538 58.17843866
549
+ 539 58.25602968
550
+ 540 58.14814815
551
+ 541 58.22550832
552
+ 542 58.11808118
553
+ 543 58.01104972
554
+ 544 57.90441176
555
+ 545 57.98165138
556
+ 546 57.87545788
557
+ 547 57.76965265
558
+ 548 57.84671533
559
+ 549 57.92349727
560
+ 550 58.00000000
561
+ 551 57.89473684
562
+ 552 57.78985507
563
+ 553 57.86618445
564
+ 554 57.94223827
565
+ 555 58.01801802
566
+ 556 57.91366906
567
+ 557 57.98922801
568
+ 558 57.88530466
569
+ 559 57.96064401
570
+ 560 58.03571429
571
+ 561 58.11051693
572
+ 562 58.18505338
573
+ 563 58.25932504
574
+ 564 58.15602837
575
+ 565 58.05309735
576
+ 566 58.12720848
577
+ 567 58.02469136
578
+ 568 58.09859155
579
+ 569 57.99648506
580
+ 570 57.89473684
581
+ 571 57.96847636
582
+ 572 57.86713287
583
+ 573 57.94066318
584
+ 574 58.01393728
585
+ 575 58.08695652
586
+ 576 58.15972222
587
+ 577 58.23223570
588
+ 578 58.30449827
589
+ 579 58.37651123
590
+ 580 58.27586207
591
+ 581 58.17555938
592
+ 582 58.07560137
593
+ 583 58.14751286
594
+ 584 58.21917808
595
+ 585 58.29059829
596
+ 586 58.19112628
597
+ 587 58.09199319
598
+ 588 57.99319728
599
+ 589 58.06451613
600
+ 590 58.13559322
601
+ 591 58.20642978
602
+ 592 58.10810811
603
+ 593 58.17875211
604
+ 594 58.24915825
605
+ 595 58.15126050
606
+ 596 58.05369128
607
+ 597 57.95644891
608
+ 598 57.85953177
609
+ 599 57.76293823
610
+ 600 57.83333333
611
+ 601 57.90349418
612
+ 602 57.80730897
613
+ 603 57.71144279
614
+ 604 57.61589404
615
+ 605 57.52066116
616
+ 606 57.59075908
617
+ 607 57.66062603
618
+ 608 57.73026316
619
+ 609 57.63546798
620
+ 610 57.70491803
621
+ 611 57.61047463
622
+ 612 57.51633987
623
+ 613 57.42251223
624
+ 614 57.32899023
625
+ 615 57.23577236
626
+ 616 57.14285714
627
+ 617 57.21231767
628
+ 618 57.11974110
629
+ 619 57.02746365
630
+ 620 57.09677419
631
+ 621 57.16586151
632
+ 622 57.07395498
633
+ 623 57.14285714
634
+ 624 57.05128205
635
+ 625 56.96000000
636
+ 626 57.02875399
637
+ 627 57.09728868
638
+ 628 57.00636943
639
+ 629 57.07472178
640
+ 630 57.14285714
641
+ 631 57.05229794
642
+ 632 56.96202532
643
+ 633 56.87203791
644
+ 634 56.94006309
645
+ 635 57.00787402
646
+ 636 57.07547170
647
+ 637 56.98587127
648
+ 638 57.05329154
649
+ 639 57.12050078
650
+ 640 57.03125000
651
+ 641 56.94227769
652
+ 642 56.85358255
653
+ 643 56.76516330
654
+ 644 56.83229814
655
+ 645 56.74418605
656
+ 646 56.81114551
657
+ 647 56.87789799
658
+ 648 56.79012346
659
+ 649 56.70261941
660
+ 650 56.61538462
661
+ 651 56.68202765
662
+ 652 56.59509202
663
+ 653 56.66156202
664
+ 654 56.72782875
665
+ 655 56.64122137
666
+ 656 56.70731707
667
+ 657 56.77321157
668
+ 658 56.68693009
669
+ 659 56.75265554
670
+ 660 56.81818182
671
+ 661 56.73222390
672
+ 662 56.79758308
673
+ 663 56.71191554
674
+ 664 56.77710843
675
+ 665 56.84210526
676
+ 666 56.90690691
677
+ 667 56.97151424
678
+ 668 56.88622754
679
+ 669 56.80119581
680
+ 670 56.86567164
681
+ 671 56.92995529
682
+ 672 56.84523810
683
+ 673 56.76077266
684
+ 674 56.67655786
685
+ 675 56.74074074
686
+ 676 56.80473373
687
+ 677 56.86853767
688
+ 678 56.93215339
689
+ 679 56.99558174
690
+ 680 57.05882353
691
+ 681 56.97503671
692
+ 682 56.89149560
693
+ 683 56.80819912
694
+ 684 56.87134503
695
+ 685 56.93430657
696
+ 686 56.85131195
697
+ 687 56.76855895
698
+ 688 56.83139535
699
+ 689 56.89404935
700
+ 690 56.95652174
701
+ 691 57.01881331
702
+ 692 57.08092486
703
+ 693 57.14285714
704
+ 694 57.20461095
705
+ 695 57.12230216
706
+ 696 57.04022989
707
+ 697 57.10186514
708
+ 698 57.02005731
709
+ 699 56.93848355
710
+ 700 57.00000000
711
+ 701 57.06134094
712
+ 702 57.12250712
713
+ 703 57.04125178
714
+ 704 57.10227273
715
+ 705 57.16312057
716
+ 706 57.22379603
717
+ 707 57.28429986
718
+ 708 57.34463277
719
+ 709 57.40479549
720
+ 710 57.32394366
721
+ 711 57.38396624
722
+ 712 57.44382022
723
+ 713 57.50350631
724
+ 714 57.42296919
725
+ 715 57.34265734
726
+ 716 57.26256983
727
+ 717 57.18270572
728
+ 718 57.10306407
729
+ 719 57.02364395
730
+ 720 57.08333333
731
+ 721 57.00416089
732
+ 722 57.06371191
733
+ 723 57.12309820
734
+ 724 57.18232044
735
+ 725 57.24137931
736
+ 726 57.16253444
737
+ 727 57.08390646
738
+ 728 57.14285714
739
+ 729 57.20164609
740
+ 730 57.26027397
741
+ 731 57.31874145
742
+ 732 57.37704918
743
+ 733 57.43519782
744
+ 734 57.49318801
745
+ 735 57.41496599
746
+ 736 57.47282609
747
+ 737 57.39484396
748
+ 738 57.31707317
749
+ 739 57.23951286
750
+ 740 57.16216216
751
+ 741 57.21997301
752
+ 742 57.27762803
753
+ 743 57.33512786
754
+ 744 57.25806452
755
+ 745 57.31543624
756
+ 746 57.23860590
757
+ 747 57.16198126
758
+ 748 57.08556150
759
+ 749 57.00934579
760
+ 750 57.06666667
761
+
762
+ Final result: 57.0667 +/- 1.8086
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-q3_k_l.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 75.00000000
15
+ 5 80.00000000
16
+ 6 83.33333333
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 77.77777778
20
+ 10 80.00000000
21
+ 11 81.81818182
22
+ 12 75.00000000
23
+ 13 76.92307692
24
+ 14 78.57142857
25
+ 15 80.00000000
26
+ 16 81.25000000
27
+ 17 76.47058824
28
+ 18 72.22222222
29
+ 19 68.42105263
30
+ 20 70.00000000
31
+ 21 71.42857143
32
+ 22 72.72727273
33
+ 23 73.91304348
34
+ 24 70.83333333
35
+ 25 68.00000000
36
+ 26 65.38461538
37
+ 27 62.96296296
38
+ 28 64.28571429
39
+ 29 65.51724138
40
+ 30 63.33333333
41
+ 31 64.51612903
42
+ 32 62.50000000
43
+ 33 63.63636364
44
+ 34 64.70588235
45
+ 35 65.71428571
46
+ 36 63.88888889
47
+ 37 62.16216216
48
+ 38 63.15789474
49
+ 39 64.10256410
50
+ 40 65.00000000
51
+ 41 65.85365854
52
+ 42 64.28571429
53
+ 43 62.79069767
54
+ 44 63.63636364
55
+ 45 62.22222222
56
+ 46 60.86956522
57
+ 47 59.57446809
58
+ 48 58.33333333
59
+ 49 59.18367347
60
+ 50 60.00000000
61
+ 51 60.78431373
62
+ 52 59.61538462
63
+ 53 58.49056604
64
+ 54 57.40740741
65
+ 55 56.36363636
66
+ 56 55.35714286
67
+ 57 56.14035088
68
+ 58 56.89655172
69
+ 59 57.62711864
70
+ 60 58.33333333
71
+ 61 57.37704918
72
+ 62 56.45161290
73
+ 63 57.14285714
74
+ 64 57.81250000
75
+ 65 56.92307692
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 57.97101449
80
+ 70 58.57142857
81
+ 71 57.74647887
82
+ 72 58.33333333
83
+ 73 58.90410959
84
+ 74 59.45945946
85
+ 75 58.66666667
86
+ 76 59.21052632
87
+ 77 59.74025974
88
+ 78 60.25641026
89
+ 79 60.75949367
90
+ 80 61.25000000
91
+ 81 61.72839506
92
+ 82 62.19512195
93
+ 83 62.65060241
94
+ 84 63.09523810
95
+ 85 62.35294118
96
+ 86 62.79069767
97
+ 87 63.21839080
98
+ 88 63.63636364
99
+ 89 62.92134831
100
+ 90 63.33333333
101
+ 91 63.73626374
102
+ 92 63.04347826
103
+ 93 63.44086022
104
+ 94 63.82978723
105
+ 95 64.21052632
106
+ 96 64.58333333
107
+ 97 64.94845361
108
+ 98 65.30612245
109
+ 99 65.65656566
110
+ 100 65.00000000
111
+ 101 65.34653465
112
+ 102 64.70588235
113
+ 103 64.07766990
114
+ 104 64.42307692
115
+ 105 64.76190476
116
+ 106 65.09433962
117
+ 107 65.42056075
118
+ 108 65.74074074
119
+ 109 65.13761468
120
+ 110 65.45454545
121
+ 111 65.76576577
122
+ 112 66.07142857
123
+ 113 66.37168142
124
+ 114 66.66666667
125
+ 115 66.95652174
126
+ 116 67.24137931
127
+ 117 66.66666667
128
+ 118 66.94915254
129
+ 119 67.22689076
130
+ 120 67.50000000
131
+ 121 66.94214876
132
+ 122 67.21311475
133
+ 123 67.47967480
134
+ 124 66.93548387
135
+ 125 67.20000000
136
+ 126 67.46031746
137
+ 127 67.71653543
138
+ 128 67.18750000
139
+ 129 67.44186047
140
+ 130 66.92307692
141
+ 131 67.17557252
142
+ 132 66.66666667
143
+ 133 66.91729323
144
+ 134 67.16417910
145
+ 135 67.40740741
146
+ 136 67.64705882
147
+ 137 67.15328467
148
+ 138 67.39130435
149
+ 139 67.62589928
150
+ 140 67.14285714
151
+ 141 67.37588652
152
+ 142 67.60563380
153
+ 143 67.83216783
154
+ 144 68.05555556
155
+ 145 68.27586207
156
+ 146 68.49315068
157
+ 147 68.70748299
158
+ 148 68.91891892
159
+ 149 69.12751678
160
+ 150 69.33333333
161
+ 151 68.87417219
162
+ 152 68.42105263
163
+ 153 68.62745098
164
+ 154 68.83116883
165
+ 155 69.03225806
166
+ 156 68.58974359
167
+ 157 68.15286624
168
+ 158 68.35443038
169
+ 159 67.92452830
170
+ 160 68.12500000
171
+ 161 68.32298137
172
+ 162 68.51851852
173
+ 163 68.71165644
174
+ 164 68.90243902
175
+ 165 69.09090909
176
+ 166 69.27710843
177
+ 167 69.46107784
178
+ 168 69.64285714
179
+ 169 69.23076923
180
+ 170 69.41176471
181
+ 171 69.59064327
182
+ 172 69.76744186
183
+ 173 69.36416185
184
+ 174 68.96551724
185
+ 175 69.14285714
186
+ 176 69.31818182
187
+ 177 69.49152542
188
+ 178 69.66292135
189
+ 179 69.27374302
190
+ 180 69.44444444
191
+ 181 69.06077348
192
+ 182 69.23076923
193
+ 183 69.39890710
194
+ 184 69.02173913
195
+ 185 69.18918919
196
+ 186 68.81720430
197
+ 187 68.44919786
198
+ 188 68.08510638
199
+ 189 67.72486772
200
+ 190 67.89473684
201
+ 191 68.06282723
202
+ 192 68.22916667
203
+ 193 67.87564767
204
+ 194 68.04123711
205
+ 195 67.69230769
206
+ 196 67.34693878
207
+ 197 67.00507614
208
+ 198 66.66666667
209
+ 199 66.83417085
210
+ 200 67.00000000
211
+ 201 67.16417910
212
+ 202 67.32673267
213
+ 203 66.99507389
214
+ 204 67.15686275
215
+ 205 66.82926829
216
+ 206 66.50485437
217
+ 207 66.66666667
218
+ 208 66.34615385
219
+ 209 66.50717703
220
+ 210 66.66666667
221
+ 211 66.82464455
222
+ 212 66.98113208
223
+ 213 67.13615023
224
+ 214 67.28971963
225
+ 215 66.97674419
226
+ 216 67.12962963
227
+ 217 66.82027650
228
+ 218 66.51376147
229
+ 219 66.66666667
230
+ 220 66.81818182
231
+ 221 66.51583710
232
+ 222 66.66666667
233
+ 223 66.81614350
234
+ 224 66.96428571
235
+ 225 67.11111111
236
+ 226 67.25663717
237
+ 227 66.96035242
238
+ 228 67.10526316
239
+ 229 67.24890830
240
+ 230 66.95652174
241
+ 231 66.66666667
242
+ 232 66.37931034
243
+ 233 66.09442060
244
+ 234 66.23931624
245
+ 235 65.95744681
246
+ 236 65.67796610
247
+ 237 65.82278481
248
+ 238 65.96638655
249
+ 239 66.10878661
250
+ 240 66.25000000
251
+ 241 66.39004149
252
+ 242 66.52892562
253
+ 243 66.66666667
254
+ 244 66.39344262
255
+ 245 66.12244898
256
+ 246 66.26016260
257
+ 247 66.39676113
258
+ 248 66.53225806
259
+ 249 66.66666667
260
+ 250 66.80000000
261
+ 251 66.93227092
262
+ 252 67.06349206
263
+ 253 66.79841897
264
+ 254 66.92913386
265
+ 255 67.05882353
266
+ 256 67.18750000
267
+ 257 66.92607004
268
+ 258 67.05426357
269
+ 259 67.18146718
270
+ 260 67.30769231
271
+ 261 67.43295019
272
+ 262 67.17557252
273
+ 263 67.30038023
274
+ 264 67.42424242
275
+ 265 67.54716981
276
+ 266 67.29323308
277
+ 267 67.41573034
278
+ 268 67.53731343
279
+ 269 67.65799257
280
+ 270 67.77777778
281
+ 271 67.52767528
282
+ 272 67.64705882
283
+ 273 67.39926740
284
+ 274 67.51824818
285
+ 275 67.27272727
286
+ 276 67.39130435
287
+ 277 67.50902527
288
+ 278 67.62589928
289
+ 279 67.74193548
290
+ 280 67.85714286
291
+ 281 67.97153025
292
+ 282 68.08510638
293
+ 283 67.84452297
294
+ 284 67.95774648
295
+ 285 67.71929825
296
+ 286 67.83216783
297
+ 287 67.94425087
298
+ 288 68.05555556
299
+ 289 68.16608997
300
+ 290 68.27586207
301
+ 291 68.38487973
302
+ 292 68.15068493
303
+ 293 68.25938567
304
+ 294 68.36734694
305
+ 295 68.47457627
306
+ 296 68.58108108
307
+ 297 68.68686869
308
+ 298 68.79194631
309
+ 299 68.56187291
310
+ 300 68.33333333
311
+ 301 68.10631229
312
+ 302 68.21192053
313
+ 303 67.98679868
314
+ 304 67.76315789
315
+ 305 67.86885246
316
+ 306 67.97385621
317
+ 307 67.75244300
318
+ 308 67.85714286
319
+ 309 67.96116505
320
+ 310 68.06451613
321
+ 311 68.16720257
322
+ 312 68.26923077
323
+ 313 68.37060703
324
+ 314 68.15286624
325
+ 315 68.25396825
326
+ 316 68.35443038
327
+ 317 68.45425868
328
+ 318 68.23899371
329
+ 319 68.02507837
330
+ 320 68.12500000
331
+ 321 67.91277259
332
+ 322 67.70186335
333
+ 323 67.80185759
334
+ 324 67.90123457
335
+ 325 68.00000000
336
+ 326 67.79141104
337
+ 327 67.58409786
338
+ 328 67.68292683
339
+ 329 67.78115502
340
+ 330 67.87878788
341
+ 331 67.97583082
342
+ 332 67.77108434
343
+ 333 67.86786787
344
+ 334 67.66467066
345
+ 335 67.76119403
346
+ 336 67.85714286
347
+ 337 67.65578635
348
+ 338 67.45562130
349
+ 339 67.55162242
350
+ 340 67.64705882
351
+ 341 67.74193548
352
+ 342 67.83625731
353
+ 343 67.93002915
354
+ 344 67.73255814
355
+ 345 67.82608696
356
+ 346 67.63005780
357
+ 347 67.72334294
358
+ 348 67.52873563
359
+ 349 67.33524355
360
+ 350 67.42857143
361
+ 351 67.52136752
362
+ 352 67.61363636
363
+ 353 67.42209632
364
+ 354 67.51412429
365
+ 355 67.60563380
366
+ 356 67.69662921
367
+ 357 67.78711485
368
+ 358 67.87709497
369
+ 359 67.96657382
370
+ 360 67.77777778
371
+ 361 67.86703601
372
+ 362 67.95580110
373
+ 363 68.04407713
374
+ 364 68.13186813
375
+ 365 68.21917808
376
+ 366 68.30601093
377
+ 367 68.39237057
378
+ 368 68.47826087
379
+ 369 68.56368564
380
+ 370 68.37837838
381
+ 371 68.46361186
382
+ 372 68.54838710
383
+ 373 68.36461126
384
+ 374 68.44919786
385
+ 375 68.53333333
386
+ 376 68.61702128
387
+ 377 68.43501326
388
+ 378 68.51851852
389
+ 379 68.60158311
390
+ 380 68.68421053
391
+ 381 68.76640420
392
+ 382 68.58638743
393
+ 383 68.66840731
394
+ 384 68.75000000
395
+ 385 68.83116883
396
+ 386 68.91191710
397
+ 387 68.99224806
398
+ 388 68.81443299
399
+ 389 68.89460154
400
+ 390 68.97435897
401
+ 391 68.79795396
402
+ 392 68.87755102
403
+ 393 68.95674300
404
+ 394 69.03553299
405
+ 395 69.11392405
406
+ 396 69.19191919
407
+ 397 69.26952141
408
+ 398 69.34673367
409
+ 399 69.42355890
410
+ 400 69.25000000
411
+ 401 69.32668329
412
+ 402 69.15422886
413
+ 403 68.98263027
414
+ 404 69.05940594
415
+ 405 68.88888889
416
+ 406 68.96551724
417
+ 407 69.04176904
418
+ 408 69.11764706
419
+ 409 69.19315403
420
+ 410 69.02439024
421
+ 411 68.85644769
422
+ 412 68.68932039
423
+ 413 68.76513317
424
+ 414 68.84057971
425
+ 415 68.67469880
426
+ 416 68.75000000
427
+ 417 68.82494005
428
+ 418 68.89952153
429
+ 419 68.97374702
430
+ 420 69.04761905
431
+ 421 68.88361045
432
+ 422 68.95734597
433
+ 423 69.03073286
434
+ 424 69.10377358
435
+ 425 69.17647059
436
+ 426 69.24882629
437
+ 427 69.08665105
438
+ 428 69.15887850
439
+ 429 68.99766900
440
+ 430 69.06976744
441
+ 431 69.14153132
442
+ 432 69.21296296
443
+ 433 69.28406467
444
+ 434 69.35483871
445
+ 435 69.19540230
446
+ 436 69.26605505
447
+ 437 69.10755149
448
+ 438 69.17808219
449
+ 439 69.24829157
450
+ 440 69.31818182
451
+ 441 69.38775510
452
+ 442 69.45701357
453
+ 443 69.52595937
454
+ 444 69.59459459
455
+ 445 69.66292135
456
+ 446 69.73094170
457
+ 447 69.79865772
458
+ 448 69.86607143
459
+ 449 69.71046771
460
+ 450 69.77777778
461
+ 451 69.62305987
462
+ 452 69.69026549
463
+ 453 69.53642384
464
+ 454 69.60352423
465
+ 455 69.67032967
466
+ 456 69.73684211
467
+ 457 69.80306346
468
+ 458 69.86899563
469
+ 459 69.93464052
470
+ 460 70.00000000
471
+ 461 70.06507592
472
+ 462 70.12987013
473
+ 463 70.19438445
474
+ 464 70.25862069
475
+ 465 70.10752688
476
+ 466 70.17167382
477
+ 467 70.23554604
478
+ 468 70.08547009
479
+ 469 69.93603412
480
+ 470 69.78723404
481
+ 471 69.85138004
482
+ 472 69.70338983
483
+ 473 69.76744186
484
+ 474 69.83122363
485
+ 475 69.89473684
486
+ 476 69.74789916
487
+ 477 69.60167715
488
+ 478 69.66527197
489
+ 479 69.72860125
490
+ 480 69.79166667
491
+ 481 69.64656965
492
+ 482 69.50207469
493
+ 483 69.56521739
494
+ 484 69.42148760
495
+ 485 69.27835052
496
+ 486 69.34156379
497
+ 487 69.40451745
498
+ 488 69.26229508
499
+ 489 69.32515337
500
+ 490 69.38775510
501
+ 491 69.45010183
502
+ 492 69.51219512
503
+ 493 69.57403651
504
+ 494 69.63562753
505
+ 495 69.69696970
506
+ 496 69.75806452
507
+ 497 69.81891348
508
+ 498 69.67871486
509
+ 499 69.53907816
510
+ 500 69.60000000
511
+ 501 69.66067864
512
+ 502 69.72111554
513
+ 503 69.78131213
514
+ 504 69.84126984
515
+ 505 69.90099010
516
+ 506 69.96047431
517
+ 507 69.82248521
518
+ 508 69.68503937
519
+ 509 69.74459725
520
+ 510 69.80392157
521
+ 511 69.66731898
522
+ 512 69.53125000
523
+ 513 69.39571150
524
+ 514 69.45525292
525
+ 515 69.51456311
526
+ 516 69.37984496
527
+ 517 69.24564797
528
+ 518 69.30501931
529
+ 519 69.17148362
530
+ 520 69.23076923
531
+ 521 69.09788868
532
+ 522 68.96551724
533
+ 523 69.02485660
534
+ 524 69.08396947
535
+ 525 69.14285714
536
+ 526 69.01140684
537
+ 527 68.88045541
538
+ 528 68.93939394
539
+ 529 68.80907372
540
+ 530 68.86792453
541
+ 531 68.92655367
542
+ 532 68.98496241
543
+ 533 68.85553471
544
+ 534 68.91385768
545
+ 535 68.97196262
546
+ 536 69.02985075
547
+ 537 69.08752328
548
+ 538 69.14498141
549
+ 539 69.01669759
550
+ 540 69.07407407
551
+ 541 69.13123845
552
+ 542 69.18819188
553
+ 543 69.24493554
554
+ 544 69.30147059
555
+ 545 69.35779817
556
+ 546 69.41391941
557
+ 547 69.46983547
558
+ 548 69.52554745
559
+ 549 69.58105647
560
+ 550 69.45454545
561
+ 551 69.50998185
562
+ 552 69.38405797
563
+ 553 69.43942134
564
+ 554 69.49458484
565
+ 555 69.36936937
566
+ 556 69.42446043
567
+ 557 69.29982047
568
+ 558 69.17562724
569
+ 559 69.05187835
570
+ 560 69.10714286
571
+ 561 69.16221034
572
+ 562 69.03914591
573
+ 563 68.91651865
574
+ 564 68.79432624
575
+ 565 68.67256637
576
+ 566 68.55123675
577
+ 567 68.60670194
578
+ 568 68.66197183
579
+ 569 68.54130053
580
+ 570 68.59649123
581
+ 571 68.65148862
582
+ 572 68.70629371
583
+ 573 68.76090750
584
+ 574 68.81533101
585
+ 575 68.86956522
586
+ 576 68.92361111
587
+ 577 68.80415945
588
+ 578 68.85813149
589
+ 579 68.91191710
590
+ 580 68.79310345
591
+ 581 68.84681583
592
+ 582 68.90034364
593
+ 583 68.95368782
594
+ 584 69.00684932
595
+ 585 69.05982906
596
+ 586 68.94197952
597
+ 587 68.99488927
598
+ 588 68.87755102
599
+ 589 68.76061121
600
+ 590 68.64406780
601
+ 591 68.69712352
602
+ 592 68.75000000
603
+ 593 68.63406408
604
+ 594 68.51851852
605
+ 595 68.57142857
606
+ 596 68.62416107
607
+ 597 68.67671692
608
+ 598 68.72909699
609
+ 599 68.61435726
610
+ 600 68.66666667
611
+ 601 68.71880200
612
+ 602 68.77076412
613
+ 603 68.65671642
614
+ 604 68.70860927
615
+ 605 68.76033058
616
+ 606 68.64686469
617
+ 607 68.69851730
618
+ 608 68.58552632
619
+ 609 68.63711002
620
+ 610 68.68852459
621
+ 611 68.73977087
622
+ 612 68.62745098
623
+ 613 68.67862969
624
+ 614 68.56677524
625
+ 615 68.61788618
626
+ 616 68.50649351
627
+ 617 68.55753647
628
+ 618 68.60841424
629
+ 619 68.49757674
630
+ 620 68.54838710
631
+ 621 68.43800322
632
+ 622 68.48874598
633
+ 623 68.53932584
634
+ 624 68.58974359
635
+ 625 68.64000000
636
+ 626 68.53035144
637
+ 627 68.58054226
638
+ 628 68.63057325
639
+ 629 68.52146264
640
+ 630 68.57142857
641
+ 631 68.62123613
642
+ 632 68.51265823
643
+ 633 68.56240126
644
+ 634 68.61198738
645
+ 635 68.66141732
646
+ 636 68.71069182
647
+ 637 68.75981162
648
+ 638 68.80877743
649
+ 639 68.85758998
650
+ 640 68.75000000
651
+ 641 68.79875195
652
+ 642 68.69158879
653
+ 643 68.74027994
654
+ 644 68.63354037
655
+ 645 68.68217054
656
+ 646 68.73065015
657
+ 647 68.77897991
658
+ 648 68.82716049
659
+ 649 68.72110940
660
+ 650 68.76923077
661
+ 651 68.81720430
662
+ 652 68.86503067
663
+ 653 68.91271057
664
+ 654 68.96024465
665
+ 655 69.00763359
666
+ 656 69.05487805
667
+ 657 69.10197869
668
+ 658 69.14893617
669
+ 659 69.04400607
670
+ 660 68.93939394
671
+ 661 68.98638427
672
+ 662 69.03323263
673
+ 663 69.07993967
674
+ 664 69.12650602
675
+ 665 69.17293233
676
+ 666 69.21921922
677
+ 667 69.26536732
678
+ 668 69.31137725
679
+ 669 69.35724963
680
+ 670 69.40298507
681
+ 671 69.44858420
682
+ 672 69.49404762
683
+ 673 69.53937593
684
+ 674 69.43620178
685
+ 675 69.48148148
686
+ 676 69.37869822
687
+ 677 69.42392910
688
+ 678 69.46902655
689
+ 679 69.51399116
690
+ 680 69.55882353
691
+ 681 69.45668135
692
+ 682 69.50146628
693
+ 683 69.54612006
694
+ 684 69.59064327
695
+ 685 69.48905109
696
+ 686 69.53352770
697
+ 687 69.57787482
698
+ 688 69.62209302
699
+ 689 69.66618287
700
+ 690 69.71014493
701
+ 691 69.75397974
702
+ 692 69.79768786
703
+ 693 69.84126984
704
+ 694 69.88472622
705
+ 695 69.92805755
706
+ 696 69.97126437
707
+ 697 69.87087518
708
+ 698 69.91404011
709
+ 699 69.81402003
710
+ 700 69.85714286
711
+ 701 69.90014265
712
+ 702 69.94301994
713
+ 703 69.98577525
714
+ 704 70.02840909
715
+ 705 70.07092199
716
+ 706 70.11331445
717
+ 707 70.15558699
718
+ 708 70.19774011
719
+ 709 70.23977433
720
+ 710 70.28169014
721
+ 711 70.32348805
722
+ 712 70.36516854
723
+ 713 70.40673212
724
+ 714 70.44817927
725
+ 715 70.48951049
726
+ 716 70.53072626
727
+ 717 70.57182706
728
+ 718 70.61281337
729
+ 719 70.51460362
730
+ 720 70.55555556
731
+ 721 70.45769764
732
+ 722 70.49861496
733
+ 723 70.53941909
734
+ 724 70.58011050
735
+ 725 70.62068966
736
+ 726 70.66115702
737
+ 727 70.70151307
738
+ 728 70.74175824
739
+ 729 70.78189300
740
+ 730 70.68493151
741
+ 731 70.72503420
742
+ 732 70.62841530
743
+ 733 70.66848568
744
+ 734 70.70844687
745
+ 735 70.74829932
746
+ 736 70.78804348
747
+ 737 70.69199457
748
+ 738 70.59620596
749
+ 739 70.50067659
750
+ 740 70.54054054
751
+ 741 70.58029690
752
+ 742 70.61994609
753
+ 743 70.65948856
754
+ 744 70.69892473
755
+ 745 70.60402685
756
+ 746 70.64343164
757
+ 747 70.54886212
758
+ 748 70.58823529
759
+ 749 70.62750334
760
+ 750 70.66666667
scores/Hammer2.1-7b-q3_k_l.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 0.00000000
13
+ 3 0.00000000
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 28.57142857
18
+ 8 37.50000000
19
+ 9 44.44444444
20
+ 10 40.00000000
21
+ 11 36.36363636
22
+ 12 41.66666667
23
+ 13 46.15384615
24
+ 14 42.85714286
25
+ 15 46.66666667
26
+ 16 50.00000000
27
+ 17 52.94117647
28
+ 18 50.00000000
29
+ 19 52.63157895
30
+ 20 50.00000000
31
+ 21 47.61904762
32
+ 22 45.45454545
33
+ 23 43.47826087
34
+ 24 45.83333333
35
+ 25 48.00000000
36
+ 26 46.15384615
37
+ 27 48.14814815
38
+ 28 46.42857143
39
+ 29 48.27586207
40
+ 30 50.00000000
41
+ 31 51.61290323
42
+ 32 53.12500000
43
+ 33 51.51515152
44
+ 34 52.94117647
45
+ 35 54.28571429
46
+ 36 52.77777778
47
+ 37 54.05405405
48
+ 38 52.63157895
49
+ 39 51.28205128
50
+ 40 50.00000000
51
+ 41 48.78048780
52
+ 42 47.61904762
53
+ 43 48.83720930
54
+ 44 47.72727273
55
+ 45 46.66666667
56
+ 46 45.65217391
57
+ 47 44.68085106
58
+ 48 45.83333333
59
+ 49 44.89795918
60
+ 50 44.00000000
61
+ 51 43.13725490
62
+ 52 42.30769231
63
+ 53 41.50943396
64
+ 54 40.74074074
65
+ 55 41.81818182
66
+ 56 41.07142857
67
+ 57 40.35087719
68
+ 58 41.37931034
69
+ 59 42.37288136
70
+ 60 41.66666667
71
+ 61 40.98360656
72
+ 62 40.32258065
73
+ 63 39.68253968
74
+ 64 39.06250000
75
+ 65 38.46153846
76
+ 66 39.39393939
77
+ 67 38.80597015
78
+ 68 38.23529412
79
+ 69 37.68115942
80
+ 70 38.57142857
81
+ 71 39.43661972
82
+ 72 38.88888889
83
+ 73 38.35616438
84
+ 74 39.18918919
85
+ 75 40.00000000
86
+ 76 40.78947368
87
+ 77 41.55844156
88
+ 78 42.30769231
89
+ 79 41.77215190
90
+ 80 42.50000000
91
+ 81 43.20987654
92
+ 82 42.68292683
93
+ 83 43.37349398
94
+ 84 44.04761905
95
+ 85 43.52941176
96
+ 86 43.02325581
97
+ 87 43.67816092
98
+ 88 44.31818182
99
+ 89 43.82022472
100
+ 90 43.33333333
101
+ 91 42.85714286
102
+ 92 42.39130435
103
+ 93 41.93548387
104
+ 94 42.55319149
105
+ 95 42.10526316
106
+ 96 42.70833333
107
+ 97 42.26804124
108
+ 98 41.83673469
109
+ 99 41.41414141
110
+ 100 41.00000000
111
+ 101 41.58415842
112
+ 102 41.17647059
113
+ 103 40.77669903
114
+ 104 41.34615385
115
+ 105 41.90476190
116
+ 106 42.45283019
117
+ 107 42.05607477
118
+ 108 41.66666667
119
+ 109 41.28440367
120
+ 110 40.90909091
121
+ 111 40.54054054
122
+ 112 41.07142857
123
+ 113 40.70796460
124
+ 114 41.22807018
125
+ 115 41.73913043
126
+ 116 41.37931034
127
+ 117 41.02564103
128
+ 118 40.67796610
129
+ 119 41.17647059
130
+ 120 41.66666667
131
+ 121 42.14876033
132
+ 122 41.80327869
133
+ 123 41.46341463
134
+ 124 41.12903226
135
+ 125 41.60000000
136
+ 126 41.26984127
137
+ 127 40.94488189
138
+ 128 40.62500000
139
+ 129 40.31007752
140
+ 130 40.00000000
141
+ 131 40.45801527
142
+ 132 40.15151515
143
+ 133 39.84962406
144
+ 134 39.55223881
145
+ 135 39.25925926
146
+ 136 38.97058824
147
+ 137 39.41605839
148
+ 138 39.13043478
149
+ 139 39.56834532
150
+ 140 40.00000000
151
+ 141 39.71631206
152
+ 142 40.14084507
153
+ 143 40.55944056
154
+ 144 40.27777778
155
+ 145 40.00000000
156
+ 146 40.41095890
157
+ 147 40.13605442
158
+ 148 40.54054054
159
+ 149 40.93959732
160
+ 150 40.66666667
161
+ 151 40.39735099
162
+ 152 40.78947368
163
+ 153 40.52287582
164
+ 154 40.25974026
165
+ 155 40.00000000
166
+ 156 39.74358974
167
+ 157 40.12738854
168
+ 158 40.50632911
169
+ 159 40.25157233
170
+ 160 40.62500000
171
+ 161 40.37267081
172
+ 162 40.12345679
173
+ 163 39.87730061
174
+ 164 39.63414634
175
+ 165 40.00000000
176
+ 166 39.75903614
177
+ 167 40.11976048
178
+ 168 39.88095238
179
+ 169 40.23668639
180
+ 170 40.00000000
181
+ 171 39.76608187
182
+ 172 39.53488372
183
+ 173 39.30635838
184
+ 174 39.65517241
185
+ 175 39.42857143
186
+ 176 39.20454545
187
+ 177 38.98305085
188
+ 178 39.32584270
189
+ 179 39.66480447
190
+ 180 39.44444444
191
+ 181 39.22651934
192
+ 182 39.01098901
193
+ 183 39.34426230
194
+ 184 39.13043478
195
+ 185 39.45945946
196
+ 186 39.78494624
197
+ 187 39.57219251
198
+ 188 39.36170213
199
+ 189 39.68253968
200
+ 190 39.47368421
201
+ 191 39.79057592
202
+ 192 39.58333333
203
+ 193 39.37823834
204
+ 194 39.17525773
205
+ 195 39.48717949
206
+ 196 39.28571429
207
+ 197 39.59390863
208
+ 198 39.39393939
209
+ 199 39.19597990
210
+ 200 39.00000000
211
+ 201 38.80597015
212
+ 202 38.61386139
213
+ 203 38.42364532
214
+ 204 38.23529412
215
+ 205 38.53658537
216
+ 206 38.34951456
217
+ 207 38.64734300
218
+ 208 38.46153846
219
+ 209 38.27751196
220
+ 210 38.09523810
221
+ 211 38.38862559
222
+ 212 38.20754717
223
+ 213 38.02816901
224
+ 214 37.85046729
225
+ 215 38.13953488
226
+ 216 38.42592593
227
+ 217 38.24884793
228
+ 218 38.53211009
229
+ 219 38.81278539
230
+ 220 39.09090909
231
+ 221 39.36651584
232
+ 222 39.63963964
233
+ 223 39.91031390
234
+ 224 40.17857143
235
+ 225 40.44444444
236
+ 226 40.70796460
237
+ 227 40.52863436
238
+ 228 40.78947368
239
+ 229 40.61135371
240
+ 230 40.43478261
241
+ 231 40.25974026
242
+ 232 40.08620690
243
+ 233 39.91416309
244
+ 234 40.17094017
245
+ 235 40.00000000
246
+ 236 39.83050847
247
+ 237 39.66244726
248
+ 238 39.49579832
249
+ 239 39.33054393
250
+ 240 39.16666667
251
+ 241 39.41908714
252
+ 242 39.66942149
253
+ 243 39.91769547
254
+ 244 39.75409836
255
+ 245 40.00000000
256
+ 246 39.83739837
257
+ 247 40.08097166
258
+ 248 40.32258065
259
+ 249 40.16064257
260
+ 250 40.00000000
261
+ 251 39.84063745
262
+ 252 39.68253968
263
+ 253 39.92094862
264
+ 254 39.76377953
265
+ 255 39.60784314
266
+ 256 39.84375000
267
+ 257 40.07782101
268
+ 258 39.92248062
269
+ 259 39.76833977
270
+ 260 39.61538462
271
+ 261 39.46360153
272
+ 262 39.31297710
273
+ 263 39.16349810
274
+ 264 39.39393939
275
+ 265 39.24528302
276
+ 266 39.47368421
277
+ 267 39.32584270
278
+ 268 39.17910448
279
+ 269 39.40520446
280
+ 270 39.62962963
281
+ 271 39.48339483
282
+ 272 39.33823529
283
+ 273 39.19413919
284
+ 274 39.05109489
285
+ 275 38.90909091
286
+ 276 39.13043478
287
+ 277 39.35018051
288
+ 278 39.56834532
289
+ 279 39.42652330
290
+ 280 39.64285714
291
+ 281 39.50177936
292
+ 282 39.36170213
293
+ 283 39.22261484
294
+ 284 39.08450704
295
+ 285 38.94736842
296
+ 286 38.81118881
297
+ 287 39.02439024
298
+ 288 38.88888889
299
+ 289 38.75432526
300
+ 290 38.96551724
301
+ 291 39.17525773
302
+ 292 39.38356164
303
+ 293 39.59044369
304
+ 294 39.45578231
305
+ 295 39.32203390
306
+ 296 39.18918919
307
+ 297 39.05723906
308
+ 298 39.26174497
309
+ 299 39.46488294
310
+ 300 39.66666667
311
+ 301 39.53488372
312
+ 302 39.73509934
313
+ 303 39.60396040
314
+ 304 39.47368421
315
+ 305 39.67213115
316
+ 306 39.54248366
317
+ 307 39.73941368
318
+ 308 39.61038961
319
+ 309 39.48220065
320
+ 310 39.67741935
321
+ 311 39.87138264
322
+ 312 40.06410256
323
+ 313 40.25559105
324
+ 314 40.44585987
325
+ 315 40.63492063
326
+ 316 40.50632911
327
+ 317 40.69400631
328
+ 318 40.56603774
329
+ 319 40.75235110
330
+ 320 40.62500000
331
+ 321 40.49844237
332
+ 322 40.37267081
333
+ 323 40.24767802
334
+ 324 40.12345679
335
+ 325 40.30769231
336
+ 326 40.18404908
337
+ 327 40.36697248
338
+ 328 40.54878049
339
+ 329 40.72948328
340
+ 330 40.60606061
341
+ 331 40.48338369
342
+ 332 40.36144578
343
+ 333 40.24024024
344
+ 334 40.11976048
345
+ 335 40.29850746
346
+ 336 40.47619048
347
+ 337 40.35608309
348
+ 338 40.23668639
349
+ 339 40.11799410
350
+ 340 40.29411765
351
+ 341 40.46920821
352
+ 342 40.64327485
353
+ 343 40.52478134
354
+ 344 40.40697674
355
+ 345 40.28985507
356
+ 346 40.46242775
357
+ 347 40.63400576
358
+ 348 40.51724138
359
+ 349 40.68767908
360
+ 350 40.85714286
361
+ 351 40.74074074
362
+ 352 40.62500000
363
+ 353 40.79320113
364
+ 354 40.67796610
365
+ 355 40.56338028
366
+ 356 40.73033708
367
+ 357 40.61624650
368
+ 358 40.50279330
369
+ 359 40.38997214
370
+ 360 40.55555556
371
+ 361 40.44321330
372
+ 362 40.60773481
373
+ 363 40.77134986
374
+ 364 40.65934066
375
+ 365 40.54794521
376
+ 366 40.71038251
377
+ 367 40.59945504
378
+ 368 40.48913043
379
+ 369 40.37940379
380
+ 370 40.54054054
381
+ 371 40.43126685
382
+ 372 40.32258065
383
+ 373 40.21447721
384
+ 374 40.37433155
385
+ 375 40.53333333
386
+ 376 40.42553191
387
+ 377 40.31830239
388
+ 378 40.47619048
389
+ 379 40.36939314
390
+ 380 40.26315789
391
+ 381 40.41994751
392
+ 382 40.57591623
393
+ 383 40.73107050
394
+ 384 40.62500000
395
+ 385 40.77922078
396
+ 386 40.67357513
397
+ 387 40.56847545
398
+ 388 40.46391753
399
+ 389 40.61696658
400
+ 390 40.76923077
401
+ 391 40.92071611
402
+ 392 41.07142857
403
+ 393 40.96692112
404
+ 394 40.86294416
405
+ 395 41.01265823
406
+ 396 41.16161616
407
+ 397 41.05793451
408
+ 398 40.95477387
409
+ 399 41.10275689
410
+ 400 41.00000000
411
+ 401 41.14713217
412
+ 402 41.04477612
413
+ 403 40.94292804
414
+ 404 40.84158416
415
+ 405 40.74074074
416
+ 406 40.64039409
417
+ 407 40.54054054
418
+ 408 40.44117647
419
+ 409 40.58679707
420
+ 410 40.48780488
421
+ 411 40.38929440
422
+ 412 40.29126214
423
+ 413 40.19370460
424
+ 414 40.09661836
425
+ 415 40.24096386
426
+ 416 40.14423077
427
+ 417 40.04796163
428
+ 418 39.95215311
429
+ 419 40.09546539
430
+ 420 40.23809524
431
+ 421 40.14251781
432
+ 422 40.04739336
433
+ 423 39.95271868
434
+ 424 40.09433962
435
+ 425 40.23529412
436
+ 426 40.37558685
437
+ 427 40.28103044
438
+ 428 40.18691589
439
+ 429 40.32634033
440
+ 430 40.23255814
441
+ 431 40.37122970
442
+ 432 40.50925926
443
+ 433 40.41570439
444
+ 434 40.32258065
445
+ 435 40.22988506
446
+ 436 40.13761468
447
+ 437 40.04576659
448
+ 438 40.18264840
449
+ 439 40.31890661
450
+ 440 40.22727273
451
+ 441 40.13605442
452
+ 442 40.04524887
453
+ 443 39.95485327
454
+ 444 39.86486486
455
+ 445 39.77528090
456
+ 446 39.68609865
457
+ 447 39.82102908
458
+ 448 39.73214286
459
+ 449 39.64365256
460
+ 450 39.55555556
461
+ 451 39.46784922
462
+ 452 39.60176991
463
+ 453 39.73509934
464
+ 454 39.64757709
465
+ 455 39.56043956
466
+ 456 39.69298246
467
+ 457 39.82494530
468
+ 458 39.73799127
469
+ 459 39.65141612
470
+ 460 39.78260870
471
+ 461 39.69631236
472
+ 462 39.61038961
473
+ 463 39.52483801
474
+ 464 39.65517241
475
+ 465 39.56989247
476
+ 466 39.48497854
477
+ 467 39.61456103
478
+ 468 39.52991453
479
+ 469 39.65884861
480
+ 470 39.57446809
481
+ 471 39.49044586
482
+ 472 39.40677966
483
+ 473 39.32346723
484
+ 474 39.24050633
485
+ 475 39.15789474
486
+ 476 39.28571429
487
+ 477 39.20335430
488
+ 478 39.33054393
489
+ 479 39.45720251
490
+ 480 39.58333333
491
+ 481 39.50103950
492
+ 482 39.41908714
493
+ 483 39.33747412
494
+ 484 39.25619835
495
+ 485 39.17525773
496
+ 486 39.09465021
497
+ 487 39.21971253
498
+ 488 39.13934426
499
+ 489 39.05930470
500
+ 490 39.18367347
501
+ 491 39.10386965
502
+ 492 39.02439024
503
+ 493 38.94523327
504
+ 494 39.06882591
505
+ 495 39.19191919
506
+ 496 39.31451613
507
+ 497 39.23541247
508
+ 498 39.15662651
509
+ 499 39.07815631
510
+ 500 39.00000000
511
+ 501 38.92215569
512
+ 502 38.84462151
513
+ 503 38.76739563
514
+ 504 38.88888889
515
+ 505 38.81188119
516
+ 506 38.93280632
517
+ 507 38.85601578
518
+ 508 38.97637795
519
+ 509 38.89980354
520
+ 510 39.01960784
521
+ 511 38.94324853
522
+ 512 38.86718750
523
+ 513 38.79142300
524
+ 514 38.71595331
525
+ 515 38.64077670
526
+ 516 38.75968992
527
+ 517 38.87814313
528
+ 518 38.99613900
529
+ 519 38.92100193
530
+ 520 39.03846154
531
+ 521 38.96353167
532
+ 522 38.88888889
533
+ 523 38.81453155
534
+ 524 38.93129771
535
+ 525 38.85714286
536
+ 526 38.78326996
537
+ 527 38.89943074
538
+ 528 39.01515152
539
+ 529 38.94139887
540
+ 530 39.05660377
541
+ 531 38.98305085
542
+ 532 38.90977444
543
+ 533 38.83677298
544
+ 534 38.76404494
545
+ 535 38.87850467
546
+ 536 38.80597015
547
+ 537 38.91992551
548
+ 538 39.03345725
549
+ 539 38.96103896
550
+ 540 38.88888889
551
+ 541 38.81700555
552
+ 542 38.92988930
553
+ 543 38.85819521
554
+ 544 38.78676471
555
+ 545 38.71559633
556
+ 546 38.64468864
557
+ 547 38.57404022
558
+ 548 38.68613139
559
+ 549 38.61566485
560
+ 550 38.54545455
561
+ 551 38.65698730
562
+ 552 38.58695652
563
+ 553 38.51717902
564
+ 554 38.44765343
565
+ 555 38.37837838
566
+ 556 38.48920863
567
+ 557 38.42010772
568
+ 558 38.53046595
569
+ 559 38.46153846
570
+ 560 38.39285714
571
+ 561 38.32442068
572
+ 562 38.43416370
573
+ 563 38.54351687
574
+ 564 38.47517730
575
+ 565 38.58407080
576
+ 566 38.69257951
577
+ 567 38.62433862
578
+ 568 38.55633803
579
+ 569 38.48857645
580
+ 570 38.59649123
581
+ 571 38.52889667
582
+ 572 38.63636364
583
+ 573 38.56893543
584
+ 574 38.67595819
585
+ 575 38.60869565
586
+ 576 38.71527778
587
+ 577 38.82149047
588
+ 578 38.75432526
589
+ 579 38.68739206
590
+ 580 38.62068966
591
+ 581 38.55421687
592
+ 582 38.48797251
593
+ 583 38.59348199
594
+ 584 38.69863014
595
+ 585 38.63247863
596
+ 586 38.73720137
597
+ 587 38.67120954
598
+ 588 38.60544218
599
+ 589 38.53989813
600
+ 590 38.47457627
601
+ 591 38.40947547
602
+ 592 38.34459459
603
+ 593 38.27993255
604
+ 594 38.38383838
605
+ 595 38.31932773
606
+ 596 38.42281879
607
+ 597 38.35845896
608
+ 598 38.29431438
609
+ 599 38.39732888
610
+ 600 38.33333333
611
+ 601 38.43594010
612
+ 602 38.37209302
613
+ 603 38.47429519
614
+ 604 38.41059603
615
+ 605 38.34710744
616
+ 606 38.28382838
617
+ 607 38.38550247
618
+ 608 38.32236842
619
+ 609 38.25944171
620
+ 610 38.19672131
621
+ 611 38.29787234
622
+ 612 38.23529412
623
+ 613 38.17292007
624
+ 614 38.11074919
625
+ 615 38.04878049
626
+ 616 38.14935065
627
+ 617 38.08752026
628
+ 618 38.02588997
629
+ 619 37.96445880
630
+ 620 37.90322581
631
+ 621 37.84219002
632
+ 622 37.78135048
633
+ 623 37.72070626
634
+ 624 37.66025641
635
+ 625 37.60000000
636
+ 626 37.69968051
637
+ 627 37.63955343
638
+ 628 37.57961783
639
+ 629 37.67885533
640
+ 630 37.77777778
641
+ 631 37.87638669
642
+ 632 37.97468354
643
+ 633 37.91469194
644
+ 634 37.85488959
645
+ 635 37.79527559
646
+ 636 37.73584906
647
+ 637 37.83359498
648
+ 638 37.77429467
649
+ 639 37.87167449
650
+ 640 37.96875000
651
+ 641 37.90951638
652
+ 642 38.00623053
653
+ 643 37.94712286
654
+ 644 38.04347826
655
+ 645 37.98449612
656
+ 646 37.92569659
657
+ 647 37.86707883
658
+ 648 37.80864198
659
+ 649 37.75038521
660
+ 650 37.84615385
661
+ 651 37.78801843
662
+ 652 37.73006135
663
+ 653 37.67228178
664
+ 654 37.76758410
665
+ 655 37.70992366
666
+ 656 37.65243902
667
+ 657 37.59512938
668
+ 658 37.68996960
669
+ 659 37.63277693
670
+ 660 37.57575758
671
+ 661 37.51891074
672
+ 662 37.46223565
673
+ 663 37.40573152
674
+ 664 37.50000000
675
+ 665 37.59398496
676
+ 666 37.53753754
677
+ 667 37.63118441
678
+ 668 37.72455090
679
+ 669 37.81763827
680
+ 670 37.76119403
681
+ 671 37.70491803
682
+ 672 37.64880952
683
+ 673 37.74145617
684
+ 674 37.68545994
685
+ 675 37.62962963
686
+ 676 37.57396450
687
+ 677 37.51846381
688
+ 678 37.46312684
689
+ 679 37.40795287
690
+ 680 37.50000000
691
+ 681 37.59177680
692
+ 682 37.53665689
693
+ 683 37.48169839
694
+ 684 37.42690058
695
+ 685 37.37226277
696
+ 686 37.31778426
697
+ 687 37.26346434
698
+ 688 37.20930233
699
+ 689 37.30043541
700
+ 690 37.24637681
701
+ 691 37.33719247
702
+ 692 37.28323699
703
+ 693 37.22943723
704
+ 694 37.17579251
705
+ 695 37.12230216
706
+ 696 37.21264368
707
+ 697 37.30272597
708
+ 698 37.24928367
709
+ 699 37.33905579
710
+ 700 37.28571429
711
+ 701 37.23252496
712
+ 702 37.32193732
713
+ 703 37.41109531
714
+ 704 37.35795455
715
+ 705 37.30496454
716
+ 706 37.25212465
717
+ 707 37.19943423
718
+ 708 37.14689266
719
+ 709 37.23554302
720
+ 710 37.32394366
721
+ 711 37.41209564
722
+ 712 37.35955056
723
+ 713 37.44740533
724
+ 714 37.53501401
725
+ 715 37.48251748
726
+ 716 37.56983240
727
+ 717 37.51743375
728
+ 718 37.46518106
729
+ 719 37.55215577
730
+ 720 37.50000000
731
+ 721 37.44798890
732
+ 722 37.53462604
733
+ 723 37.62102351
734
+ 724 37.56906077
735
+ 725 37.65517241
736
+ 726 37.60330579
737
+ 727 37.55158184
738
+ 728 37.50000000
739
+ 729 37.44855967
740
+ 730 37.39726027
741
+ 731 37.34610123
742
+ 732 37.43169399
743
+ 733 37.51705321
744
+ 734 37.60217984
745
+ 735 37.55102041
746
+ 736 37.50000000
747
+ 737 37.58480326
748
+ 738 37.53387534
749
+ 739 37.61840325
750
+ 740 37.56756757
751
+ 741 37.51686910
752
+ 742 37.60107817
753
+ 743 37.68506057
754
+ 744 37.76881720
755
+ 745 37.71812081
756
+ 746 37.66756032
757
+ 747 37.61713521
758
+ 748 37.56684492
759
+ 749 37.51668892
760
+ 750 37.60000000
761
+
762
+ Final result: 37.6000 +/- 1.7699
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-q3_k_l.ppx ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ====== Perplexity statistics ======
2
+ Mean PPL(Q) : 10.521151 ± 0.076540
3
+ Mean PPL(base) : 9.366577 ± 0.066397
4
+ Cor(ln(PPL(Q)), ln(PPL(base))): 98.01%
5
+ Mean ln(PPL(Q)/PPL(base)) : 0.116240 ± 0.001446
6
+ Mean PPL(Q)/PPL(base) : 1.123265 ± 0.001624
7
+ Mean PPL(Q)-PPL(base) : 1.154574 ± 0.017475
8
+
9
+ ====== KL divergence statistics ======
10
+ Mean KLD: 0.123969 ± 0.000369
11
+ Maximum KLD: 5.081474
12
+ 99.9% KLD: 1.300398
13
+ 99.0% KLD: 0.639832
14
+ 99.0% KLD: 0.639832
15
+ Median KLD: 0.089902
16
+ 10.0% KLD: 0.001788
17
+ 5.0% KLD: 0.000368
18
+ 1.0% KLD: 0.000029
19
+ Minimum KLD: -0.000038
20
+
21
+ ====== Token probability statistics ======
22
+ Mean Δp: -1.696 ± 0.023 %
23
+ Maximum Δp: 86.345%
24
+ 99.9% Δp: 39.410%
25
+ 99.0% Δp: 22.830%
26
+ 95.0% Δp: 10.667%
27
+ 90.0% Δp: 5.530%
28
+ 75.0% Δp: 0.509%
29
+ Median Δp: -0.103%
30
+ 25.0% Δp: -3.310%
31
+ 10.0% Δp: -11.363%
32
+ 5.0% Δp: -17.740%
33
+ 1.0% Δp: -32.927%
34
+ 0.1% Δp: -57.176%
35
+ Minimum Δp: -92.270%
36
+ RMS Δp : 9.140 ± 0.039 %
37
+ Same top p: 81.206 ± 0.101 %
scores/Hammer2.1-7b-q3_k_l.tqa ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 817 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 817 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 14.28571429
18
+ 8 12.50000000
19
+ 9 11.11111111
20
+ 10 10.00000000
21
+ 11 9.09090909
22
+ 12 16.66666667
23
+ 13 23.07692308
24
+ 14 21.42857143
25
+ 15 20.00000000
26
+ 16 18.75000000
27
+ 17 17.64705882
28
+ 18 16.66666667
29
+ 19 21.05263158
30
+ 20 20.00000000
31
+ 21 19.04761905
32
+ 22 18.18181818
33
+ 23 17.39130435
34
+ 24 20.83333333
35
+ 25 20.00000000
36
+ 26 19.23076923
37
+ 27 18.51851852
38
+ 28 17.85714286
39
+ 29 20.68965517
40
+ 30 20.00000000
41
+ 31 19.35483871
42
+ 32 21.87500000
43
+ 33 24.24242424
44
+ 34 23.52941176
45
+ 35 22.85714286
46
+ 36 22.22222222
47
+ 37 24.32432432
48
+ 38 23.68421053
49
+ 39 25.64102564
50
+ 40 25.00000000
51
+ 41 24.39024390
52
+ 42 23.80952381
53
+ 43 25.58139535
54
+ 44 25.00000000
55
+ 45 24.44444444
56
+ 46 26.08695652
57
+ 47 25.53191489
58
+ 48 27.08333333
59
+ 49 26.53061224
60
+ 50 28.00000000
61
+ 51 29.41176471
62
+ 52 28.84615385
63
+ 53 28.30188679
64
+ 54 29.62962963
65
+ 55 30.90909091
66
+ 56 30.35714286
67
+ 57 29.82456140
68
+ 58 29.31034483
69
+ 59 28.81355932
70
+ 60 28.33333333
71
+ 61 27.86885246
72
+ 62 29.03225806
73
+ 63 28.57142857
74
+ 64 29.68750000
75
+ 65 29.23076923
76
+ 66 28.78787879
77
+ 67 28.35820896
78
+ 68 27.94117647
79
+ 69 28.98550725
80
+ 70 30.00000000
81
+ 71 29.57746479
82
+ 72 30.55555556
83
+ 73 30.13698630
84
+ 74 29.72972973
85
+ 75 29.33333333
86
+ 76 28.94736842
87
+ 77 28.57142857
88
+ 78 28.20512821
89
+ 79 27.84810127
90
+ 80 28.75000000
91
+ 81 28.39506173
92
+ 82 29.26829268
93
+ 83 28.91566265
94
+ 84 28.57142857
95
+ 85 28.23529412
96
+ 86 27.90697674
97
+ 87 27.58620690
98
+ 88 27.27272727
99
+ 89 28.08988764
100
+ 90 27.77777778
101
+ 91 28.57142857
102
+ 92 28.26086957
103
+ 93 27.95698925
104
+ 94 27.65957447
105
+ 95 28.42105263
106
+ 96 29.16666667
107
+ 97 29.89690722
108
+ 98 29.59183673
109
+ 99 30.30303030
110
+ 100 31.00000000
111
+ 101 31.68316832
112
+ 102 31.37254902
113
+ 103 31.06796117
114
+ 104 30.76923077
115
+ 105 30.47619048
116
+ 106 30.18867925
117
+ 107 30.84112150
118
+ 108 31.48148148
119
+ 109 31.19266055
120
+ 110 31.81818182
121
+ 111 31.53153153
122
+ 112 31.25000000
123
+ 113 30.97345133
124
+ 114 31.57894737
125
+ 115 31.30434783
126
+ 116 31.03448276
127
+ 117 30.76923077
128
+ 118 30.50847458
129
+ 119 30.25210084
130
+ 120 30.83333333
131
+ 121 31.40495868
132
+ 122 31.14754098
133
+ 123 30.89430894
134
+ 124 31.45161290
135
+ 125 32.00000000
136
+ 126 31.74603175
137
+ 127 31.49606299
138
+ 128 31.25000000
139
+ 129 31.00775194
140
+ 130 30.76923077
141
+ 131 30.53435115
142
+ 132 30.30303030
143
+ 133 30.07518797
144
+ 134 30.59701493
145
+ 135 31.11111111
146
+ 136 30.88235294
147
+ 137 31.38686131
148
+ 138 31.88405797
149
+ 139 32.37410072
150
+ 140 32.14285714
151
+ 141 31.91489362
152
+ 142 31.69014085
153
+ 143 31.46853147
154
+ 144 31.94444444
155
+ 145 31.72413793
156
+ 146 31.50684932
157
+ 147 31.29251701
158
+ 148 31.08108108
159
+ 149 30.87248322
160
+ 150 30.66666667
161
+ 151 31.12582781
162
+ 152 30.92105263
163
+ 153 30.71895425
164
+ 154 30.51948052
165
+ 155 30.96774194
166
+ 156 31.41025641
167
+ 157 31.21019108
168
+ 158 31.64556962
169
+ 159 31.44654088
170
+ 160 31.25000000
171
+ 161 31.05590062
172
+ 162 30.86419753
173
+ 163 30.67484663
174
+ 164 30.48780488
175
+ 165 30.90909091
176
+ 166 31.32530120
177
+ 167 31.13772455
178
+ 168 30.95238095
179
+ 169 30.76923077
180
+ 170 30.58823529
181
+ 171 30.40935673
182
+ 172 30.23255814
183
+ 173 30.05780347
184
+ 174 30.45977011
185
+ 175 30.28571429
186
+ 176 30.11363636
187
+ 177 29.94350282
188
+ 178 29.77528090
189
+ 179 29.60893855
190
+ 180 29.44444444
191
+ 181 29.28176796
192
+ 182 29.12087912
193
+ 183 28.96174863
194
+ 184 28.80434783
195
+ 185 28.64864865
196
+ 186 28.49462366
197
+ 187 28.34224599
198
+ 188 28.72340426
199
+ 189 29.10052910
200
+ 190 28.94736842
201
+ 191 28.79581152
202
+ 192 29.16666667
203
+ 193 29.01554404
204
+ 194 29.38144330
205
+ 195 29.23076923
206
+ 196 29.08163265
207
+ 197 29.44162437
208
+ 198 29.79797980
209
+ 199 29.64824121
210
+ 200 29.50000000
211
+ 201 29.35323383
212
+ 202 29.70297030
213
+ 203 29.55665025
214
+ 204 29.41176471
215
+ 205 29.75609756
216
+ 206 29.61165049
217
+ 207 29.95169082
218
+ 208 29.80769231
219
+ 209 29.66507177
220
+ 210 29.52380952
221
+ 211 29.38388626
222
+ 212 29.24528302
223
+ 213 29.10798122
224
+ 214 28.97196262
225
+ 215 28.83720930
226
+ 216 29.16666667
227
+ 217 29.03225806
228
+ 218 29.35779817
229
+ 219 29.22374429
230
+ 220 29.09090909
231
+ 221 28.95927602
232
+ 222 28.82882883
233
+ 223 28.69955157
234
+ 224 29.01785714
235
+ 225 28.88888889
236
+ 226 28.76106195
237
+ 227 29.07488987
238
+ 228 28.94736842
239
+ 229 28.82096070
240
+ 230 28.69565217
241
+ 231 28.57142857
242
+ 232 28.87931034
243
+ 233 29.18454936
244
+ 234 29.05982906
245
+ 235 28.93617021
246
+ 236 29.23728814
247
+ 237 29.53586498
248
+ 238 29.83193277
249
+ 239 29.70711297
250
+ 240 29.58333333
251
+ 241 29.46058091
252
+ 242 29.33884298
253
+ 243 29.21810700
254
+ 244 29.09836066
255
+ 245 28.97959184
256
+ 246 28.86178862
257
+ 247 28.74493927
258
+ 248 28.62903226
259
+ 249 28.51405622
260
+ 250 28.40000000
261
+ 251 28.68525896
262
+ 252 28.57142857
263
+ 253 28.45849802
264
+ 254 28.34645669
265
+ 255 28.62745098
266
+ 256 28.51562500
267
+ 257 28.40466926
268
+ 258 28.29457364
269
+ 259 28.18532819
270
+ 260 28.07692308
271
+ 261 28.35249042
272
+ 262 28.24427481
273
+ 263 28.13688213
274
+ 264 28.40909091
275
+ 265 28.67924528
276
+ 266 28.57142857
277
+ 267 28.83895131
278
+ 268 28.73134328
279
+ 269 28.62453532
280
+ 270 28.51851852
281
+ 271 28.41328413
282
+ 272 28.30882353
283
+ 273 28.20512821
284
+ 274 28.46715328
285
+ 275 28.36363636
286
+ 276 28.26086957
287
+ 277 28.15884477
288
+ 278 28.05755396
289
+ 279 27.95698925
290
+ 280 28.21428571
291
+ 281 28.46975089
292
+ 282 28.36879433
293
+ 283 28.26855124
294
+ 284 28.52112676
295
+ 285 28.42105263
296
+ 286 28.67132867
297
+ 287 28.91986063
298
+ 288 28.81944444
299
+ 289 28.71972318
300
+ 290 28.62068966
301
+ 291 28.86597938
302
+ 292 28.76712329
303
+ 293 29.01023891
304
+ 294 28.91156463
305
+ 295 29.15254237
306
+ 296 29.39189189
307
+ 297 29.62962963
308
+ 298 29.53020134
309
+ 299 29.43143813
310
+ 300 29.33333333
311
+ 301 29.23588040
312
+ 302 29.13907285
313
+ 303 29.04290429
314
+ 304 28.94736842
315
+ 305 28.85245902
316
+ 306 28.75816993
317
+ 307 28.66449511
318
+ 308 28.89610390
319
+ 309 28.80258900
320
+ 310 28.70967742
321
+ 311 28.61736334
322
+ 312 28.84615385
323
+ 313 28.75399361
324
+ 314 28.98089172
325
+ 315 28.88888889
326
+ 316 28.79746835
327
+ 317 28.70662461
328
+ 318 28.61635220
329
+ 319 28.52664577
330
+ 320 28.43750000
331
+ 321 28.34890966
332
+ 322 28.57142857
333
+ 323 28.48297214
334
+ 324 28.39506173
335
+ 325 28.30769231
336
+ 326 28.22085890
337
+ 327 28.44036697
338
+ 328 28.65853659
339
+ 329 28.57142857
340
+ 330 28.78787879
341
+ 331 28.70090634
342
+ 332 28.91566265
343
+ 333 28.82882883
344
+ 334 28.74251497
345
+ 335 28.95522388
346
+ 336 29.16666667
347
+ 337 29.08011869
348
+ 338 28.99408284
349
+ 339 29.20353982
350
+ 340 29.11764706
351
+ 341 29.32551320
352
+ 342 29.23976608
353
+ 343 29.15451895
354
+ 344 29.36046512
355
+ 345 29.56521739
356
+ 346 29.47976879
357
+ 347 29.68299712
358
+ 348 29.59770115
359
+ 349 29.51289398
360
+ 350 29.42857143
361
+ 351 29.34472934
362
+ 352 29.26136364
363
+ 353 29.17847025
364
+ 354 29.09604520
365
+ 355 29.29577465
366
+ 356 29.21348315
367
+ 357 29.13165266
368
+ 358 29.05027933
369
+ 359 28.96935933
370
+ 360 28.88888889
371
+ 361 29.08587258
372
+ 362 29.00552486
373
+ 363 29.20110193
374
+ 364 29.12087912
375
+ 365 29.31506849
376
+ 366 29.23497268
377
+ 367 29.42779292
378
+ 368 29.34782609
379
+ 369 29.26829268
380
+ 370 29.18918919
381
+ 371 29.11051213
382
+ 372 29.03225806
383
+ 373 28.95442359
384
+ 374 29.14438503
385
+ 375 29.06666667
386
+ 376 28.98936170
387
+ 377 28.91246684
388
+ 378 28.83597884
389
+ 379 28.75989446
390
+ 380 28.68421053
391
+ 381 28.60892388
392
+ 382 28.79581152
393
+ 383 28.72062663
394
+ 384 28.64583333
395
+ 385 28.57142857
396
+ 386 28.49740933
397
+ 387 28.42377261
398
+ 388 28.60824742
399
+ 389 28.53470437
400
+ 390 28.46153846
401
+ 391 28.38874680
402
+ 392 28.31632653
403
+ 393 28.24427481
404
+ 394 28.42639594
405
+ 395 28.35443038
406
+ 396 28.28282828
407
+ 397 28.46347607
408
+ 398 28.39195980
409
+ 399 28.32080201
410
+ 400 28.25000000
411
+ 401 28.17955112
412
+ 402 28.35820896
413
+ 403 28.53598015
414
+ 404 28.46534653
415
+ 405 28.39506173
416
+ 406 28.32512315
417
+ 407 28.25552826
418
+ 408 28.18627451
419
+ 409 28.11735941
420
+ 410 28.04878049
421
+ 411 27.98053528
422
+ 412 28.15533981
423
+ 413 28.08716707
424
+ 414 28.26086957
425
+ 415 28.43373494
426
+ 416 28.36538462
427
+ 417 28.29736211
428
+ 418 28.22966507
429
+ 419 28.16229117
430
+ 420 28.33333333
431
+ 421 28.50356295
432
+ 422 28.43601896
433
+ 423 28.36879433
434
+ 424 28.30188679
435
+ 425 28.23529412
436
+ 426 28.16901408
437
+ 427 28.10304450
438
+ 428 28.03738318
439
+ 429 28.20512821
440
+ 430 28.13953488
441
+ 431 28.07424594
442
+ 432 28.24074074
443
+ 433 28.40646651
444
+ 434 28.57142857
445
+ 435 28.50574713
446
+ 436 28.44036697
447
+ 437 28.37528604
448
+ 438 28.31050228
449
+ 439 28.24601367
450
+ 440 28.18181818
451
+ 441 28.11791383
452
+ 442 28.05429864
453
+ 443 27.99097065
454
+ 444 28.15315315
455
+ 445 28.08988764
456
+ 446 28.02690583
457
+ 447 27.96420582
458
+ 448 27.90178571
459
+ 449 27.83964365
460
+ 450 28.00000000
461
+ 451 27.93791574
462
+ 452 27.87610619
463
+ 453 28.03532009
464
+ 454 28.19383260
465
+ 455 28.13186813
466
+ 456 28.07017544
467
+ 457 28.00875274
468
+ 458 27.94759825
469
+ 459 27.88671024
470
+ 460 27.82608696
471
+ 461 27.76572668
472
+ 462 27.70562771
473
+ 463 27.86177106
474
+ 464 28.01724138
475
+ 465 28.17204301
476
+ 466 28.11158798
477
+ 467 28.05139186
478
+ 468 28.20512821
479
+ 469 28.14498934
480
+ 470 28.08510638
481
+ 471 28.23779193
482
+ 472 28.17796610
483
+ 473 28.11839323
484
+ 474 28.05907173
485
+ 475 28.21052632
486
+ 476 28.15126050
487
+ 477 28.30188679
488
+ 478 28.24267782
489
+ 479 28.18371608
490
+ 480 28.12500000
491
+ 481 28.06652807
492
+ 482 28.21576763
493
+ 483 28.15734990
494
+ 484 28.30578512
495
+ 485 28.45360825
496
+ 486 28.39506173
497
+ 487 28.33675565
498
+ 488 28.27868852
499
+ 489 28.42535787
500
+ 490 28.57142857
501
+ 491 28.71690428
502
+ 492 28.86178862
503
+ 493 28.80324544
504
+ 494 28.74493927
505
+ 495 28.88888889
506
+ 496 28.83064516
507
+ 497 28.77263581
508
+ 498 28.71485944
509
+ 499 28.85771543
510
+ 500 28.80000000
511
+ 501 28.94211577
512
+ 502 28.88446215
513
+ 503 28.82703777
514
+ 504 28.96825397
515
+ 505 28.91089109
516
+ 506 28.85375494
517
+ 507 28.99408284
518
+ 508 28.93700787
519
+ 509 29.07662083
520
+ 510 29.21568627
521
+ 511 29.15851272
522
+ 512 29.10156250
523
+ 513 29.04483431
524
+ 514 28.98832685
525
+ 515 29.12621359
526
+ 516 29.06976744
527
+ 517 29.20696325
528
+ 518 29.15057915
529
+ 519 29.09441233
530
+ 520 29.03846154
531
+ 521 28.98272553
532
+ 522 28.92720307
533
+ 523 28.87189293
534
+ 524 28.81679389
535
+ 525 28.76190476
536
+ 526 28.70722433
537
+ 527 28.65275142
538
+ 528 28.59848485
539
+ 529 28.54442344
540
+ 530 28.49056604
541
+ 531 28.43691149
542
+ 532 28.57142857
543
+ 533 28.70544090
544
+ 534 28.65168539
545
+ 535 28.59813084
546
+ 536 28.54477612
547
+ 537 28.67783985
548
+ 538 28.62453532
549
+ 539 28.57142857
550
+ 540 28.70370370
551
+ 541 28.65064695
552
+ 542 28.59778598
553
+ 543 28.54511971
554
+ 544 28.49264706
555
+ 545 28.44036697
556
+ 546 28.57142857
557
+ 547 28.70201097
558
+ 548 28.83211679
559
+ 549 28.77959927
560
+ 550 28.72727273
561
+ 551 28.67513612
562
+ 552 28.80434783
563
+ 553 28.93309222
564
+ 554 28.88086643
565
+ 555 28.82882883
566
+ 556 28.77697842
567
+ 557 28.72531418
568
+ 558 28.67383513
569
+ 559 28.62254025
570
+ 560 28.75000000
571
+ 561 28.87700535
572
+ 562 28.82562278
573
+ 563 28.77442274
574
+ 564 28.90070922
575
+ 565 29.02654867
576
+ 566 29.15194346
577
+ 567 29.10052910
578
+ 568 29.04929577
579
+ 569 29.17398946
580
+ 570 29.12280702
581
+ 571 29.07180385
582
+ 572 29.02097902
583
+ 573 28.97033159
584
+ 574 29.09407666
585
+ 575 29.21739130
586
+ 576 29.16666667
587
+ 577 29.28942808
588
+ 578 29.41176471
589
+ 579 29.53367876
590
+ 580 29.65517241
591
+ 581 29.60413081
592
+ 582 29.55326460
593
+ 583 29.50257290
594
+ 584 29.62328767
595
+ 585 29.57264957
596
+ 586 29.69283276
597
+ 587 29.64224872
598
+ 588 29.59183673
599
+ 589 29.54159593
600
+ 590 29.66101695
601
+ 591 29.78003384
602
+ 592 29.72972973
603
+ 593 29.67959528
604
+ 594 29.79797980
605
+ 595 29.74789916
606
+ 596 29.69798658
607
+ 597 29.81574539
608
+ 598 29.76588629
609
+ 599 29.71619366
610
+ 600 29.66666667
611
+ 601 29.78369384
612
+ 602 29.73421927
613
+ 603 29.85074627
614
+ 604 29.80132450
615
+ 605 29.75206612
616
+ 606 29.86798680
617
+ 607 29.81878089
618
+ 608 29.76973684
619
+ 609 29.88505747
620
+ 610 29.83606557
621
+ 611 29.78723404
622
+ 612 29.90196078
623
+ 613 29.85318108
624
+ 614 29.80456026
625
+ 615 29.75609756
626
+ 616 29.87012987
627
+ 617 29.98379254
628
+ 618 29.93527508
629
+ 619 30.04846527
630
+ 620 30.00000000
631
+ 621 29.95169082
632
+ 622 30.06430868
633
+ 623 30.01605136
634
+ 624 29.96794872
635
+ 625 29.92000000
636
+ 626 29.87220447
637
+ 627 29.82456140
638
+ 628 29.77707006
639
+ 629 29.72972973
640
+ 630 29.68253968
641
+ 631 29.79397781
642
+ 632 29.74683544
643
+ 633 29.69984202
644
+ 634 29.81072555
645
+ 635 29.76377953
646
+ 636 29.71698113
647
+ 637 29.67032967
648
+ 638 29.62382445
649
+ 639 29.57746479
650
+ 640 29.53125000
651
+ 641 29.48517941
652
+ 642 29.59501558
653
+ 643 29.54898911
654
+ 644 29.50310559
655
+ 645 29.61240310
656
+ 646 29.56656347
657
+ 647 29.52086553
658
+ 648 29.62962963
659
+ 649 29.58397535
660
+ 650 29.69230769
661
+ 651 29.80030722
662
+ 652 29.90797546
663
+ 653 30.01531394
664
+ 654 29.96941896
665
+ 655 29.92366412
666
+ 656 30.03048780
667
+ 657 30.13698630
668
+ 658 30.09118541
669
+ 659 30.04552352
670
+ 660 30.00000000
671
+ 661 29.95461422
672
+ 662 29.90936556
673
+ 663 29.86425339
674
+ 664 29.81927711
675
+ 665 29.77443609
676
+ 666 29.87987988
677
+ 667 29.83508246
678
+ 668 29.79041916
679
+ 669 29.74588939
680
+ 670 29.85074627
681
+ 671 29.80625931
682
+ 672 29.76190476
683
+ 673 29.71768202
684
+ 674 29.67359050
685
+ 675 29.77777778
686
+ 676 29.88165680
687
+ 677 29.83751846
688
+ 678 29.79351032
689
+ 679 29.74963181
690
+ 680 29.70588235
691
+ 681 29.66226138
692
+ 682 29.61876833
693
+ 683 29.57540264
694
+ 684 29.67836257
695
+ 685 29.63503650
696
+ 686 29.59183673
697
+ 687 29.54876274
698
+ 688 29.50581395
699
+ 689 29.60812772
700
+ 690 29.56521739
701
+ 691 29.66714906
702
+ 692 29.76878613
703
+ 693 29.87012987
704
+ 694 29.82708934
705
+ 695 29.78417266
706
+ 696 29.88505747
707
+ 697 29.84218077
708
+ 698 29.79942693
709
+ 699 29.75679542
710
+ 700 29.85714286
711
+ 701 29.81455064
712
+ 702 29.77207977
713
+ 703 29.72972973
714
+ 704 29.68750000
715
+ 705 29.64539007
716
+ 706 29.60339943
717
+ 707 29.70297030
718
+ 708 29.66101695
719
+ 709 29.61918195
720
+ 710 29.57746479
721
+ 711 29.53586498
722
+ 712 29.49438202
723
+ 713 29.45301543
724
+ 714 29.55182073
725
+ 715 29.65034965
726
+ 716 29.60893855
727
+ 717 29.56764296
728
+ 718 29.52646240
729
+ 719 29.48539638
730
+ 720 29.44444444
731
+ 721 29.40360610
732
+ 722 29.36288089
733
+ 723 29.32226833
734
+ 724 29.41988950
735
+ 725 29.37931034
736
+ 726 29.47658402
737
+ 727 29.43603851
738
+ 728 29.39560440
739
+ 729 29.49245542
740
+ 730 29.45205479
741
+ 731 29.41176471
742
+ 732 29.50819672
743
+ 733 29.46793997
744
+ 734 29.42779292
745
+ 735 29.38775510
746
+ 736 29.34782609
747
+ 737 29.44369064
748
+ 738 29.40379404
749
+ 739 29.49932341
750
+ 740 29.45945946
751
+ 741 29.55465587
752
+ 742 29.51482480
753
+ 743 29.47510094
754
+ 744 29.56989247
755
+ 745 29.53020134
756
+ 746 29.62466488
757
+ 747 29.71887550
758
+ 748 29.67914439
759
+ 749 29.77303071
760
+ 750 29.86666667
761
+
762
+ Final result: 29.8667 +/- 1.6723
763
+ Random chance: 19.8992 +/- 1.4588
scores/Hammer2.1-7b-q3_k_l.wng ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ winogrande_score : loaded 1267 tasks from prompt.
6
+ winogrande_score : selecting 750 random tasks
7
+ winogrande_score : tokenizing selected tasks
8
+ winogrande_score : calculating winogrande score over selected tasks.
9
+ 1 100.0000 -4.258393 -4.922616 1 1
10
+ 2 100.0000 -2.087463 -2.623787 1 1
11
+ 3 100.0000 -2.812181 -2.298521 2 2
12
+ 4 75.0000 -4.868878 -4.359646 2 1
13
+ 5 60.0000 -2.355327 -3.245432 1 2
14
+ 6 66.6667 -4.708636 -4.731373 1 1
15
+ 7 71.4286 -4.263852 -3.082357 2 2
16
+ 8 62.5000 -1.825159 -1.985436 1 2
17
+ 9 66.6667 -3.495538 -5.652018 1 1
18
+ 10 70.0000 -1.809094 -1.429981 2 2
19
+ 11 72.7273 -11.224876 -5.450241 2 2
20
+ 12 75.0000 -4.303400 -3.880177 2 2
21
+ 13 76.9231 -3.086766 -6.401550 1 1
22
+ 14 71.4286 -3.912080 -3.859712 2 1
23
+ 15 66.6667 -3.861654 -5.618620 1 2
24
+ 16 68.7500 -4.462656 -4.027624 2 2
25
+ 17 64.7059 -6.079226 -6.791906 1 2
26
+ 18 66.6667 -1.365568 -2.834053 1 1
27
+ 19 68.4211 -4.373370 -5.304165 1 1
28
+ 20 70.0000 -3.197187 -4.105173 1 1
29
+ 21 71.4286 -2.733500 -3.257509 1 1
30
+ 22 72.7273 -2.890539 -2.354089 2 2
31
+ 23 73.9130 -5.309480 -5.392314 1 1
32
+ 24 70.8333 -2.752423 -2.480168 2 1
33
+ 25 72.0000 -2.767115 -2.539575 2 2
34
+ 26 73.0769 -3.291131 -4.359396 1 1
35
+ 27 74.0741 -5.384951 -5.382199 2 2
36
+ 28 75.0000 -4.453629 -5.247103 1 1
37
+ 29 75.8621 -4.008163 -4.286048 1 1
38
+ 30 73.3333 -1.478904 -1.817019 1 2
39
+ 31 74.1935 -5.739226 -5.741822 1 1
40
+ 32 75.0000 -4.337257 -5.611024 1 1
41
+ 33 75.7576 -3.588931 -6.266914 1 1
42
+ 34 73.5294 -2.358663 -2.286152 2 1
43
+ 35 71.4286 -4.711539 -4.730115 1 2
44
+ 36 72.2222 -5.097197 -4.150145 2 2
45
+ 37 70.2703 -3.538671 -3.287072 2 1
46
+ 38 71.0526 -3.310911 -3.548758 1 1
47
+ 39 71.7949 -4.328108 -4.054216 2 2
48
+ 40 70.0000 -4.301931 -5.513919 1 2
49
+ 41 70.7317 -2.555128 -2.924105 1 1
50
+ 42 71.4286 -1.806398 -2.569053 1 1
51
+ 43 69.7674 -4.090628 -4.113010 1 2
52
+ 44 68.1818 -2.032056 -2.053465 1 2
53
+ 45 68.8889 -0.095197 -14.702598 1 1
54
+ 46 69.5652 -2.646211 -2.838568 1 1
55
+ 47 68.0851 -4.322934 -4.926567 1 2
56
+ 48 68.7500 -5.118403 -4.995190 2 2
57
+ 49 67.3469 -2.870949 -0.903379 2 1
58
+ 50 66.0000 -2.522727 -2.546365 1 2
59
+ 51 66.6667 -2.441005 -2.916932 1 1
60
+ 52 67.3077 -3.135936 -2.390750 2 2
61
+ 53 67.9245 -3.838913 -4.803535 1 1
62
+ 54 68.5185 -4.134429 -3.761396 2 2
63
+ 55 69.0909 -3.271567 -2.954415 2 2
64
+ 56 67.8571 -5.008878 -5.425260 1 2
65
+ 57 66.6667 -2.097795 -3.229101 1 2
66
+ 58 67.2414 -3.360974 -4.329776 1 1
67
+ 59 67.7966 -3.098367 -2.960912 2 2
68
+ 60 68.3333 -4.915509 -4.497200 2 2
69
+ 61 68.8525 -4.096455 -4.515255 1 1
70
+ 62 67.7419 -7.517569 -7.379702 2 1
71
+ 63 66.6667 -3.906125 -4.793322 1 2
72
+ 64 67.1875 -4.197346 -4.406158 1 1
73
+ 65 67.6923 -4.699005 -4.252464 2 2
74
+ 66 68.1818 -2.536740 -3.720163 1 1
75
+ 67 68.6567 -2.365510 -2.086620 2 2
76
+ 68 69.1176 -4.850766 -4.615762 2 2
77
+ 69 69.5652 -4.382039 -8.095819 1 1
78
+ 70 70.0000 -3.437429 -2.934406 2 2
79
+ 71 69.0141 -3.466995 -3.078214 2 1
80
+ 72 68.0556 -2.829777 -3.198086 1 2
81
+ 73 68.4932 -3.165992 -3.558535 1 1
82
+ 74 68.9189 -2.477611 -2.479536 1 1
83
+ 75 69.3333 -1.552640 -2.278865 1 1
84
+ 76 69.7368 -1.956179 -4.023433 1 1
85
+ 77 68.8312 -2.822201 -3.208450 1 2
86
+ 78 69.2308 -1.416577 -1.623659 1 1
87
+ 79 68.3544 -2.230979 -2.454638 1 2
88
+ 80 68.7500 -2.399529 -3.718987 1 1
89
+ 81 67.9012 -3.615929 -3.681257 1 2
90
+ 82 67.0732 -2.555015 -2.825947 1 2
91
+ 83 66.2651 -6.326290 -6.558348 1 2
92
+ 84 66.6667 -2.498080 -2.438698 2 2
93
+ 85 67.0588 -5.080064 -5.561990 1 1
94
+ 86 66.2791 -5.936318 -6.199219 1 2
95
+ 87 65.5172 -1.729248 -1.427739 2 1
96
+ 88 64.7727 -4.557175 -4.241110 2 1
97
+ 89 65.1685 -3.263171 -3.575262 1 1
98
+ 90 65.5556 -5.921990 -4.741462 2 2
99
+ 91 64.8352 -4.876374 -4.289455 2 1
100
+ 92 65.2174 -5.504188 -4.332339 2 2
101
+ 93 65.5914 -2.807614 -2.455973 2 2
102
+ 94 65.9574 -3.732096 -3.047809 2 2
103
+ 95 66.3158 -5.414975 -6.034754 1 1
104
+ 96 66.6667 -2.000194 -3.054429 1 1
105
+ 97 67.0103 -5.263141 -5.377751 1 1
106
+ 98 67.3469 -5.150519 -3.772655 2 2
107
+ 99 66.6667 -3.044497 -2.380490 2 1
108
+ 100 67.0000 -7.703159 -6.930967 2 2
109
+ 101 66.3366 -4.688031 -4.768580 1 2
110
+ 102 66.6667 -3.919231 -5.120968 1 1
111
+ 103 66.0194 -2.927786 -3.304010 1 2
112
+ 104 66.3462 -8.570854 -8.437530 2 2
113
+ 105 66.6667 -1.310711 -2.354064 1 1
114
+ 106 66.9811 -3.193346 -2.718450 2 2
115
+ 107 67.2897 -3.366830 -2.622954 2 2
116
+ 108 67.5926 -3.959323 -3.141849 2 2
117
+ 109 66.9725 -5.686187 -5.919204 1 2
118
+ 110 67.2727 -4.111486 -3.672107 2 2
119
+ 111 67.5676 -4.057866 -3.633433 2 2
120
+ 112 66.9643 -3.417317 -3.362487 2 1
121
+ 113 66.3717 -4.662737 -4.722427 1 2
122
+ 114 66.6667 -3.828870 -4.426931 1 1
123
+ 115 66.9565 -2.317768 -3.109950 1 1
124
+ 116 67.2414 -3.479540 -2.558044 2 2
125
+ 117 66.6667 -4.633681 -5.207091 1 2
126
+ 118 66.9492 -0.964099 -2.742462 1 1
127
+ 119 67.2269 -2.677309 -3.076334 1 1
128
+ 120 66.6667 -2.671961 -2.396663 2 1
129
+ 121 66.9421 -1.314271 -0.878129 2 2
130
+ 122 67.2131 -6.053524 -4.672875 2 2
131
+ 123 66.6667 -3.344150 -3.144387 2 1
132
+ 124 66.9355 -2.723909 -2.430016 2 2
133
+ 125 67.2000 -3.850634 -3.122308 2 2
134
+ 126 66.6667 -6.778810 -7.287160 1 2
135
+ 127 66.9291 -2.169026 -1.190723 2 2
136
+ 128 67.1875 -2.156067 -2.492938 1 1
137
+ 129 67.4419 -2.162697 -1.388652 2 2
138
+ 130 67.6923 -5.699975 -5.083689 2 2
139
+ 131 67.1756 -3.650408 -3.934435 1 2
140
+ 132 66.6667 -3.112490 -3.128711 1 2
141
+ 133 66.9173 -1.466216 -3.933589 1 1
142
+ 134 67.1642 -5.154942 -4.516918 2 2
143
+ 135 67.4074 -4.491626 -5.274894 1 1
144
+ 136 67.6471 -5.187514 -4.468640 2 2
145
+ 137 67.1533 -6.699468 -6.630946 2 1
146
+ 138 66.6667 -4.136912 -4.554589 1 2
147
+ 139 66.1871 -4.697422 -4.176826 2 1
148
+ 140 66.4286 -1.306254 -1.189875 2 2
149
+ 141 66.6667 -4.488186 -3.930090 2 2
150
+ 142 66.9014 -2.895093 -2.811123 2 2
151
+ 143 67.1329 -1.606214 -5.602889 1 1
152
+ 144 66.6667 -2.043030 -2.020330 2 1
153
+ 145 66.2069 -1.206014 -14.140805 1 2
154
+ 146 66.4384 -4.506599 -4.092323 2 2
155
+ 147 65.9864 -5.001079 -5.421366 1 2
156
+ 148 66.2162 -2.086448 -2.528462 1 1
157
+ 149 66.4430 -3.797919 -3.532465 2 2
158
+ 150 66.6667 -4.376402 -3.410414 2 2
159
+ 151 66.2252 -2.382921 -2.302724 2 1
160
+ 152 66.4474 -4.029729 -4.028755 2 2
161
+ 153 66.0131 -4.235216 -4.039123 2 1
162
+ 154 66.2338 -3.435052 -3.728086 1 1
163
+ 155 66.4516 -2.058302 -1.988672 2 2
164
+ 156 66.0256 -2.787528 -2.440931 2 1
165
+ 157 66.2420 -3.733016 -5.040459 1 1
166
+ 158 66.4557 -4.410717 -3.957542 2 2
167
+ 159 66.6667 -3.684002 -5.221528 1 1
168
+ 160 66.8750 -3.773694 -3.052379 2 2
169
+ 161 67.0807 -2.775719 -2.844738 1 1
170
+ 162 66.6667 -0.419326 -15.198402 1 2
171
+ 163 66.8712 -1.820291 -1.833650 1 1
172
+ 164 67.0732 -5.044436 -4.283976 2 2
173
+ 165 66.6667 -4.654115 -4.655699 1 2
174
+ 166 66.8675 -4.139293 -3.729534 2 2
175
+ 167 66.4671 -4.787830 -4.432318 2 1
176
+ 168 66.0714 -4.341026 -4.653561 1 2
177
+ 169 65.6805 -3.681366 -3.993058 1 2
178
+ 170 65.2941 -4.496274 -4.195696 2 1
179
+ 171 65.4971 -3.742366 -4.719990 1 1
180
+ 172 65.6977 -4.573602 -5.638986 1 1
181
+ 173 65.8960 -3.423094 -2.971261 2 2
182
+ 174 66.0920 -7.040839 -6.451699 2 2
183
+ 175 65.7143 -4.913093 -4.660059 2 1
184
+ 176 65.9091 -1.820628 -3.174813 1 1
185
+ 177 66.1017 -5.526471 -5.241077 2 2
186
+ 178 65.7303 -2.505002 -2.290920 2 1
187
+ 179 65.9218 -2.974208 -3.165006 1 1
188
+ 180 66.1111 -5.901462 -5.332144 2 2
189
+ 181 66.2983 -4.202220 -2.971752 2 2
190
+ 182 66.4835 -4.571492 -3.892996 2 2
191
+ 183 66.6667 -6.599412 -6.099703 2 2
192
+ 184 66.8478 -2.652965 -3.680360 1 1
193
+ 185 66.4865 -5.874372 -5.934449 1 2
194
+ 186 66.6667 -3.452462 -3.721976 1 1
195
+ 187 66.8449 -5.105780 -4.852828 2 2
196
+ 188 66.4894 -3.510996 -3.373163 2 1
197
+ 189 66.1376 -5.514863 -5.980798 1 2
198
+ 190 66.3158 -4.852913 -4.552090 2 2
199
+ 191 66.4921 -3.847837 -4.231747 1 1
200
+ 192 66.6667 -6.602087 -4.079943 2 2
201
+ 193 66.8394 -4.267015 -4.178764 2 2
202
+ 194 67.0103 -2.168664 -1.360129 2 2
203
+ 195 66.6667 -3.992078 -3.933282 2 1
204
+ 196 66.8367 -4.202204 -2.963710 2 2
205
+ 197 67.0051 -0.755674 -2.965026 1 1
206
+ 198 67.1717 -5.468050 -6.967558 1 1
207
+ 199 67.3367 -5.282589 -3.883297 2 2
208
+ 200 67.5000 -3.433531 -3.206952 2 2
209
+ 201 67.6617 -2.994538 -3.692902 1 1
210
+ 202 67.8218 -7.850214 -7.759470 2 2
211
+ 203 67.4877 -3.337790 -3.341197 1 2
212
+ 204 67.6471 -3.261014 -3.550536 1 1
213
+ 205 67.8049 -3.923170 -3.309878 2 2
214
+ 206 67.9612 -6.057859 -6.049694 2 2
215
+ 207 68.1159 -2.902391 -3.093064 1 1
216
+ 208 68.2692 -4.576631 -3.154287 2 2
217
+ 209 68.4211 -4.261964 -3.870335 2 2
218
+ 210 68.0952 -8.227810 -9.239030 1 2
219
+ 211 67.7725 -6.168965 -5.703132 2 1
220
+ 212 67.9245 -4.649237 -3.504969 2 2
221
+ 213 67.6056 -5.154021 -6.015497 1 2
222
+ 214 67.7570 -3.219525 -3.455661 1 1
223
+ 215 67.9070 -5.572912 -5.167766 2 2
224
+ 216 67.5926 -3.724494 -3.504612 2 1
225
+ 217 67.7419 -1.367462 -1.962105 1 1
226
+ 218 67.8899 -2.385151 -2.316999 2 2
227
+ 219 68.0365 -2.831952 -3.137571 1 1
228
+ 220 68.1818 -3.449305 -3.385629 2 2
229
+ 221 68.3258 -2.624939 -7.943135 1 1
230
+ 222 68.0180 -0.160611 -7.060338 1 2
231
+ 223 68.1614 -4.510839 -4.325733 2 2
232
+ 224 67.8571 -6.348020 -6.477440 1 2
233
+ 225 68.0000 -3.949961 -2.972963 2 2
234
+ 226 68.1416 -3.185827 -3.651982 1 1
235
+ 227 67.8414 -4.393051 -3.621417 2 1
236
+ 228 67.9825 -2.110873 -3.765776 1 1
237
+ 229 67.6856 -4.714981 -4.787997 1 2
238
+ 230 67.8261 -3.230015 -3.536664 1 1
239
+ 231 67.9654 -2.193812 -2.399589 1 1
240
+ 232 68.1034 -2.434785 -1.404744 2 2
241
+ 233 68.2403 -1.978126 -2.397236 1 1
242
+ 234 68.3761 -4.120655 -3.468417 2 2
243
+ 235 68.5106 -4.096835 -3.540821 2 2
244
+ 236 68.2203 -2.577804 -2.265686 2 1
245
+ 237 68.3544 -1.141541 -1.399127 1 1
246
+ 238 68.4874 -3.429877 -2.944930 2 2
247
+ 239 68.6192 -2.831815 -2.646681 2 2
248
+ 240 68.3333 -8.035269 -8.192142 1 2
249
+ 241 68.4647 -4.555280 -3.724685 2 2
250
+ 242 68.5950 -6.167666 -5.819150 2 2
251
+ 243 68.7243 -4.204729 -4.327963 1 1
252
+ 244 68.8525 -1.944443 -3.615882 1 1
253
+ 245 68.9796 -4.674593 -5.869602 1 1
254
+ 246 69.1057 -2.466164 -3.182327 1 1
255
+ 247 68.8259 -4.872877 -5.477910 1 2
256
+ 248 68.9516 -3.379651 -3.111433 2 2
257
+ 249 69.0763 -3.096925 -2.031771 2 2
258
+ 250 69.2000 -3.347417 -3.877517 1 1
259
+ 251 69.3227 -0.866943 -0.880039 1 1
260
+ 252 69.4444 -6.489442 -6.099988 2 2
261
+ 253 69.1700 -5.439114 -5.316061 2 1
262
+ 254 69.2913 -4.284157 -3.209328 2 2
263
+ 255 69.4118 -3.494899 -4.868115 1 1
264
+ 256 69.5312 -3.844081 -4.659122 1 1
265
+ 257 69.2607 -2.831925 -2.265108 2 1
266
+ 258 68.9922 -3.591839 -4.076530 1 2
267
+ 259 68.7259 -4.494158 -4.422926 2 1
268
+ 260 68.8462 -4.525543 -5.343795 1 1
269
+ 261 68.9655 -3.802263 -3.777094 2 2
270
+ 262 69.0840 -5.134392 -5.044932 2 2
271
+ 263 69.2015 -3.714854 -4.764241 1 1
272
+ 264 69.3182 -4.404712 -4.188243 2 2
273
+ 265 69.4340 -1.435695 -1.928779 1 1
274
+ 266 69.5489 -6.293133 -8.157226 1 1
275
+ 267 69.6629 -3.899264 -4.008609 1 1
276
+ 268 69.4030 -2.866072 -2.607964 2 1
277
+ 269 69.5167 -3.510082 -3.179464 2 2
278
+ 270 69.6296 -2.468951 -5.058321 1 1
279
+ 271 69.7417 -3.246868 -3.081720 2 2
280
+ 272 69.8529 -4.303834 -3.635927 2 2
281
+ 273 69.9634 -4.020580 -3.655327 2 2
282
+ 274 70.0730 -2.954921 -2.156429 2 2
283
+ 275 70.1818 -1.439160 -2.030121 1 1
284
+ 276 69.9275 -2.509698 -2.340475 2 1
285
+ 277 70.0361 -2.884563 -3.059498 1 1
286
+ 278 70.1439 -2.928787 -3.006445 1 1
287
+ 279 70.2509 -4.667947 -3.051842 2 2
288
+ 280 70.3571 -4.084006 -3.826314 2 2
289
+ 281 70.4626 -2.186674 -1.621122 2 2
290
+ 282 70.5674 -4.070631 -3.732253 2 2
291
+ 283 70.3180 -3.202279 -3.224025 1 2
292
+ 284 70.0704 -2.908543 -2.671361 2 1
293
+ 285 70.1754 -5.540161 -6.209717 1 1
294
+ 286 70.2797 -3.681072 -3.771078 1 1
295
+ 287 70.3833 -2.160158 -4.267725 1 1
296
+ 288 70.4861 -3.804180 -2.611533 2 2
297
+ 289 70.5882 -2.430469 -2.909607 1 1
298
+ 290 70.6897 -5.404739 -5.901722 1 1
299
+ 291 70.7904 -1.740524 -3.222585 1 1
300
+ 292 70.5479 -2.004690 -2.456574 1 2
301
+ 293 70.6485 -3.656624 -2.971067 2 2
302
+ 294 70.4082 -5.175175 -3.842108 2 1
303
+ 295 70.5085 -1.806127 -2.119873 1 1
304
+ 296 70.6081 -4.940142 -4.820369 2 2
305
+ 297 70.3704 -1.035397 -1.288866 1 2
306
+ 298 70.1342 -5.905238 -7.380757 1 2
307
+ 299 70.2341 -2.402467 -2.016937 2 2
308
+ 300 70.3333 -3.063043 -4.080941 1 1
309
+ 301 70.4319 -4.558917 -5.564979 1 1
310
+ 302 70.5298 -4.445466 -4.322180 2 2
311
+ 303 70.2970 -9.503490 -8.174207 2 1
312
+ 304 70.3947 -3.521028 -3.451378 2 2
313
+ 305 70.1639 -2.912199 -3.210622 1 2
314
+ 306 70.2614 -2.601617 -2.218094 2 2
315
+ 307 70.3583 -3.147209 -3.353732 1 1
316
+ 308 70.4545 -4.720476 -4.273390 2 2
317
+ 309 70.5502 -2.990884 -3.635479 1 1
318
+ 310 70.6452 -5.068413 -5.057558 2 2
319
+ 311 70.7395 -1.329547 -4.914472 1 1
320
+ 312 70.5128 -5.894130 -4.602324 2 1
321
+ 313 70.2875 -4.302414 -5.327105 1 2
322
+ 314 70.3822 -5.932398 -5.141205 2 2
323
+ 315 70.4762 -2.547941 -3.924452 1 1
324
+ 316 70.5696 -6.649475 -5.074191 2 2
325
+ 317 70.3470 -4.621989 -4.245274 2 1
326
+ 318 70.4403 -5.111992 -4.849797 2 2
327
+ 319 70.5329 -2.236852 -3.277703 1 1
328
+ 320 70.6250 -3.890373 -5.240005 1 1
329
+ 321 70.4050 -2.913551 -2.960218 1 2
330
+ 322 70.1863 -2.270923 -2.373705 1 2
331
+ 323 70.2786 -2.612622 -3.263654 1 1
332
+ 324 70.3704 -4.190008 -4.414764 1 1
333
+ 325 70.4615 -5.213795 -4.428016 2 2
334
+ 326 70.5521 -2.934281 -1.930112 2 2
335
+ 327 70.6422 -2.697090 -3.638737 1 1
336
+ 328 70.7317 -4.065734 -4.505954 1 1
337
+ 329 70.8207 -3.905446 -10.170179 1 1
338
+ 330 70.9091 -1.288635 -2.009687 1 1
339
+ 331 70.6949 -5.690171 -3.970986 2 1
340
+ 332 70.7831 -1.957938 -3.184376 1 1
341
+ 333 70.8709 -3.123197 -1.964407 2 2
342
+ 334 70.9581 -2.002614 -6.074995 1 1
343
+ 335 70.7463 -4.527165 -4.458691 2 1
344
+ 336 70.5357 -3.261646 -4.061679 1 2
345
+ 337 70.6231 -1.958544 -1.926794 2 2
346
+ 338 70.4142 -4.019310 -3.427812 2 1
347
+ 339 70.5015 -6.318476 -5.166462 2 2
348
+ 340 70.5882 -5.559359 -5.796411 1 1
349
+ 341 70.6745 -2.892485 -4.956179 1 1
350
+ 342 70.7602 -6.046531 -5.851674 2 2
351
+ 343 70.8455 -4.033814 -4.410048 1 1
352
+ 344 70.9302 -2.638782 -2.676305 1 1
353
+ 345 71.0145 -1.862641 -3.650653 1 1
354
+ 346 70.8092 -4.429004 -7.153950 1 2
355
+ 347 70.8934 -4.419082 -3.893256 2 2
356
+ 348 70.9770 -0.978326 -2.771911 1 1
357
+ 349 70.7736 -3.427192 -5.794117 1 2
358
+ 350 70.5714 -1.018048 -4.027700 1 2
359
+ 351 70.6553 -2.376413 -3.537247 1 1
360
+ 352 70.7386 -4.106836 -2.951496 2 2
361
+ 353 70.5382 -2.360922 -1.849176 2 1
362
+ 354 70.6215 -5.417145 -3.983915 2 2
363
+ 355 70.7042 -3.328009 -2.534634 2 2
364
+ 356 70.5056 -3.383609 -3.461182 1 2
365
+ 357 70.5882 -4.294430 -4.213947 2 2
366
+ 358 70.6704 -2.438614 -2.384977 2 2
367
+ 359 70.7521 -5.996536 -3.492463 2 2
368
+ 360 70.5556 -2.613661 -3.098744 1 2
369
+ 361 70.6371 -3.152273 -1.647263 2 2
370
+ 362 70.7182 -4.987576 -4.594715 2 2
371
+ 363 70.5234 -2.639703 -2.390145 2 1
372
+ 364 70.6044 -6.250502 -6.738506 1 1
373
+ 365 70.6849 -5.710667 -6.163841 1 1
374
+ 366 70.4918 -4.375083 -9.567742 1 2
375
+ 367 70.5722 -5.421246 -5.989478 1 1
376
+ 368 70.6522 -3.317074 -1.314284 2 2
377
+ 369 70.4607 -3.991989 -4.058448 1 2
378
+ 370 70.5405 -4.951040 -5.062414 1 1
379
+ 371 70.6199 -4.416981 -4.898848 1 1
380
+ 372 70.4301 -2.819281 -2.498163 2 1
381
+ 373 70.5094 -5.393732 -5.690356 1 1
382
+ 374 70.5882 -1.978368 -3.589954 1 1
383
+ 375 70.6667 -3.267540 -2.429993 2 2
384
+ 376 70.7447 -5.714557 -4.177600 2 2
385
+ 377 70.5570 -3.880440 -4.110950 1 2
386
+ 378 70.3704 -3.421808 -3.677023 1 2
387
+ 379 70.1847 -1.521359 -2.327283 1 2
388
+ 380 70.0000 -7.909192 -7.314828 2 1
389
+ 381 70.0787 -12.041664 -9.182512 2 2
390
+ 382 69.8953 -2.286973 -2.295013 1 2
391
+ 383 69.9739 -1.764712 -0.781642 2 2
392
+ 384 69.7917 -4.016230 -2.889538 2 1
393
+ 385 69.6104 -1.263852 -1.064005 2 1
394
+ 386 69.6891 -1.702371 -9.012496 1 1
395
+ 387 69.7674 -1.502381 -1.884086 1 1
396
+ 388 69.5876 -5.148655 -6.122988 1 2
397
+ 389 69.6658 -2.436671 -1.897836 2 2
398
+ 390 69.7436 -2.660084 -3.723484 1 1
399
+ 391 69.5652 -3.687505 -5.228138 1 2
400
+ 392 69.3878 -6.233030 -4.404133 2 1
401
+ 393 69.4656 -6.961241 -6.048601 2 2
402
+ 394 69.5431 -1.860677 -4.582280 1 1
403
+ 395 69.3671 -4.387388 -4.543975 1 2
404
+ 396 69.1919 -7.023380 -6.292892 2 1
405
+ 397 69.0176 -7.534722 -7.102972 2 1
406
+ 398 69.0955 -3.988686 -7.114752 1 1
407
+ 399 68.9223 -5.148884 -5.033439 2 1
408
+ 400 69.0000 -1.949540 -4.789898 1 1
409
+ 401 69.0773 -2.764163 -2.941928 1 1
410
+ 402 68.9055 -4.086044 -4.310049 1 2
411
+ 403 68.9826 -7.178713 -7.085083 2 2
412
+ 404 69.0594 -4.333428 -2.637831 2 2
413
+ 405 69.1358 -3.177818 -4.199900 1 1
414
+ 406 69.2118 -8.230008 -9.270571 1 1
415
+ 407 69.2875 -5.300604 -5.055572 2 2
416
+ 408 69.3627 -3.361130 -3.158348 2 2
417
+ 409 69.4377 -3.949413 -4.237834 1 1
418
+ 410 69.5122 -5.473998 -6.152964 1 1
419
+ 411 69.5864 -5.284503 -5.955418 1 1
420
+ 412 69.6602 -4.256577 -2.762144 2 2
421
+ 413 69.7337 -4.167191 -2.992469 2 2
422
+ 414 69.5652 -4.820940 -4.991504 1 2
423
+ 415 69.6386 -2.674883 -2.647054 2 2
424
+ 416 69.7115 -5.490139 -3.078991 2 2
425
+ 417 69.7842 -5.065325 -5.866693 1 1
426
+ 418 69.8565 -5.703828 -6.840548 1 1
427
+ 419 69.6897 -4.536495 -5.133621 1 2
428
+ 420 69.5238 -3.317258 -3.404457 1 2
429
+ 421 69.5962 -4.313213 -3.543469 2 2
430
+ 422 69.4313 -2.611968 -2.115914 2 1
431
+ 423 69.2671 -3.633880 -3.418250 2 1
432
+ 424 69.3396 -5.123332 -5.736408 1 1
433
+ 425 69.4118 -3.896090 -2.444979 2 2
434
+ 426 69.2488 -1.952464 -2.569124 1 2
435
+ 427 69.3208 -3.820149 -5.199490 1 1
436
+ 428 69.1589 -3.493100 -2.732630 2 1
437
+ 429 69.2308 -4.134821 -4.102494 2 2
438
+ 430 69.0698 -6.733795 -5.763134 2 1
439
+ 431 69.1415 -5.441588 -6.052682 1 1
440
+ 432 68.9815 -4.247901 -4.636574 1 2
441
+ 433 69.0531 -1.752041 -1.657041 2 2
442
+ 434 69.1244 -7.368901 -8.049973 1 1
443
+ 435 69.1954 -3.210712 -3.828443 1 1
444
+ 436 69.2661 -0.624681 -2.418563 1 1
445
+ 437 69.3364 -3.823341 -3.802067 2 2
446
+ 438 69.4064 -3.871298 -3.721579 2 2
447
+ 439 69.2483 -3.692433 -3.108279 2 1
448
+ 440 69.3182 -4.171517 -5.026618 1 1
449
+ 441 69.3878 -3.931569 -5.396089 1 1
450
+ 442 69.4570 -5.050106 -5.486341 1 1
451
+ 443 69.5260 -1.567238 -3.294845 1 1
452
+ 444 69.3694 -5.614172 -7.262177 1 2
453
+ 445 69.4382 -1.710217 -2.967744 1 1
454
+ 446 69.5067 -5.826021 -5.029115 2 2
455
+ 447 69.3512 -1.907058 -1.841068 2 1
456
+ 448 69.4196 -4.150952 -3.496442 2 2
457
+ 449 69.4878 -3.101811 -2.898391 2 2
458
+ 450 69.5556 -4.059522 -3.223877 2 2
459
+ 451 69.6231 -4.138138 -3.372372 2 2
460
+ 452 69.6903 -4.618000 -5.499224 1 1
461
+ 453 69.7572 -5.406126 -4.703655 2 2
462
+ 454 69.8238 -3.111318 -3.449112 1 1
463
+ 455 69.6703 -1.769631 -8.909651 1 2
464
+ 456 69.7368 -4.730916 -4.620063 2 2
465
+ 457 69.5842 -11.008701 -10.545453 2 1
466
+ 458 69.6507 -4.277669 -3.633158 2 2
467
+ 459 69.7168 -4.537652 -4.297956 2 2
468
+ 460 69.7826 -0.084115 -7.037933 1 1
469
+ 461 69.8482 -2.588764 -3.751729 1 1
470
+ 462 69.6970 -5.484934 -5.291409 2 1
471
+ 463 69.7624 -1.004935 -1.197200 1 1
472
+ 464 69.6121 -2.814536 -2.630713 2 1
473
+ 465 69.6774 -3.218850 -3.021874 2 2
474
+ 466 69.7425 -5.424580 -5.075135 2 2
475
+ 467 69.8073 -5.054071 -5.574557 1 1
476
+ 468 69.8718 -5.552944 -4.401165 2 2
477
+ 469 69.7228 -1.933058 -1.725890 2 1
478
+ 470 69.7872 -1.894290 -1.617601 2 2
479
+ 471 69.6391 -2.465367 -2.058991 2 1
480
+ 472 69.7034 -4.363251 -3.727496 2 2
481
+ 473 69.7674 -4.522141 -3.129128 2 2
482
+ 474 69.8312 -5.370821 -6.134308 1 1
483
+ 475 69.8947 -1.476344 -1.149263 2 2
484
+ 476 69.9580 -4.714500 -4.494683 2 2
485
+ 477 69.8113 -4.920753 -5.403049 1 2
486
+ 478 69.6653 -4.633526 -4.191698 2 1
487
+ 479 69.7286 -3.579289 -3.688205 1 1
488
+ 480 69.7917 -5.464118 -5.355648 2 2
489
+ 481 69.8545 -2.276720 -2.874638 1 1
490
+ 482 69.7095 -4.575502 -5.365649 1 2
491
+ 483 69.5652 -5.437446 -4.674074 2 1
492
+ 484 69.4215 -2.590334 -1.522691 2 1
493
+ 485 69.4845 -1.639753 -2.628951 1 1
494
+ 486 69.5473 -2.673371 -3.337493 1 1
495
+ 487 69.6099 -2.492876 -1.796626 2 2
496
+ 488 69.6721 -3.223189 -2.214081 2 2
497
+ 489 69.7342 -2.172487 -2.673131 1 1
498
+ 490 69.7959 -3.437317 -3.885677 1 1
499
+ 491 69.8574 -5.496077 -4.719196 2 2
500
+ 492 69.9187 -5.781281 -5.324176 2 2
501
+ 493 69.9797 -1.453382 -2.877992 1 1
502
+ 494 69.8381 -4.802228 -4.969911 1 2
503
+ 495 69.8990 -4.081370 -4.261669 1 1
504
+ 496 69.9597 -3.730374 -3.656801 2 2
505
+ 497 69.8189 -5.279003 -4.900946 2 1
506
+ 498 69.6787 -1.428346 -1.708706 1 2
507
+ 499 69.7395 -2.823681 -3.274243 1 1
508
+ 500 69.8000 -5.336230 -4.330530 2 2
509
+ 501 69.8603 -1.170461 -1.994428 1 1
510
+ 502 69.9203 -1.745291 -1.841227 1 1
511
+ 503 69.9801 -3.042653 -2.911974 2 2
512
+ 504 69.8413 -2.902057 -2.846527 2 1
513
+ 505 69.7030 -7.537486 -8.425106 1 2
514
+ 506 69.5652 -5.986201 -5.925781 2 1
515
+ 507 69.6252 -4.787746 -4.838999 1 1
516
+ 508 69.6850 -6.596812 -2.884710 2 2
517
+ 509 69.7446 -3.733912 -8.344926 1 1
518
+ 510 69.6078 -4.225793 -3.519177 2 1
519
+ 511 69.6673 -5.082802 -3.930310 2 2
520
+ 512 69.7266 -3.708243 -4.268966 1 1
521
+ 513 69.7856 -3.642208 -3.521938 2 2
522
+ 514 69.6498 -5.974806 -5.588061 2 1
523
+ 515 69.5146 -7.455030 -7.178011 2 1
524
+ 516 69.3798 -4.216937 -4.216092 2 1
525
+ 517 69.4391 -1.519154 -1.544778 1 1
526
+ 518 69.4981 -2.975501 -2.034226 2 2
527
+ 519 69.5568 -9.974724 -9.294326 2 2
528
+ 520 69.6154 -2.801509 -2.321591 2 2
529
+ 521 69.6737 -1.277832 -1.410152 1 1
530
+ 522 69.7318 -3.704660 -2.957092 2 2
531
+ 523 69.5985 -3.827352 -3.935592 1 2
532
+ 524 69.4656 -6.885082 -6.850352 2 1
533
+ 525 69.5238 -4.922119 -1.847937 2 2
534
+ 526 69.5817 -0.296833 -17.669243 1 1
535
+ 527 69.6395 -6.763228 -5.099826 2 2
536
+ 528 69.6970 -6.550147 -5.711829 2 2
537
+ 529 69.7543 -5.087699 -5.942085 1 1
538
+ 530 69.8113 -5.600156 -4.099419 2 2
539
+ 531 69.6798 -4.004086 -3.621851 2 1
540
+ 532 69.7368 -2.282838 -2.999822 1 1
541
+ 533 69.7936 -3.781882 -3.099224 2 2
542
+ 534 69.8502 -2.994372 -3.539807 1 1
543
+ 535 69.7196 -3.305073 -3.361816 1 2
544
+ 536 69.5896 -2.326447 -2.103831 2 1
545
+ 537 69.6462 -6.100725 -6.401413 1 1
546
+ 538 69.7026 -2.609723 -2.779882 1 1
547
+ 539 69.7588 -4.617922 -2.762032 2 2
548
+ 540 69.6296 -2.643821 -3.099836 1 2
549
+ 541 69.6858 -2.662837 -2.693401 1 1
550
+ 542 69.5572 -3.924761 -4.574943 1 2
551
+ 543 69.6133 -5.497083 -4.986106 2 2
552
+ 544 69.6691 -2.749404 -4.288833 1 1
553
+ 545 69.5413 -1.214182 -4.805840 1 2
554
+ 546 69.5971 -2.690701 -3.289834 1 1
555
+ 547 69.6527 -6.348222 -4.989640 2 2
556
+ 548 69.7080 -4.513491 -4.750671 1 1
557
+ 549 69.5811 -5.687627 -5.521060 2 1
558
+ 550 69.6364 -3.513557 -3.386040 2 2
559
+ 551 69.5100 -1.687316 -7.014914 1 2
560
+ 552 69.5652 -2.681498 -3.403467 1 1
561
+ 553 69.6203 -5.898814 -5.772879 2 2
562
+ 554 69.4946 -4.597984 -4.974277 1 2
563
+ 555 69.5495 -2.956836 -2.104578 2 2
564
+ 556 69.6043 -5.997124 -4.710520 2 2
565
+ 557 69.6589 -4.470690 -5.588573 1 1
566
+ 558 69.7133 -2.182543 -3.647828 1 1
567
+ 559 69.7674 -2.482780 -2.298972 2 2
568
+ 560 69.6429 -2.612274 -2.048245 2 1
569
+ 561 69.6970 -2.065000 -2.570359 1 1
570
+ 562 69.7509 -0.683448 -1.556789 1 1
571
+ 563 69.6270 -1.571514 -2.703178 1 2
572
+ 564 69.6809 -3.135878 -3.746938 1 1
573
+ 565 69.5575 -3.618390 -3.638677 1 2
574
+ 566 69.6113 -4.283313 -5.323839 1 1
575
+ 567 69.6649 -2.759961 -2.749221 2 2
576
+ 568 69.7183 -7.377351 -5.199990 2 2
577
+ 569 69.5958 -5.671188 -5.278562 2 1
578
+ 570 69.6491 -4.784215 -6.489985 1 1
579
+ 571 69.7023 -4.742686 -3.295642 2 2
580
+ 572 69.7552 -2.943017 -2.706670 2 2
581
+ 573 69.6335 -4.389074 -4.839878 1 2
582
+ 574 69.6864 -3.786488 -2.628075 2 2
583
+ 575 69.7391 -5.005714 -4.688672 2 2
584
+ 576 69.7917 -2.716254 -3.366816 1 1
585
+ 577 69.8440 -4.363843 -4.161464 2 2
586
+ 578 69.8962 -3.238583 -2.852677 2 2
587
+ 579 69.7755 -5.417300 -5.477379 1 2
588
+ 580 69.8276 -1.846244 -3.471672 1 1
589
+ 581 69.8795 -5.813891 -7.865144 1 1
590
+ 582 69.7595 -5.481544 -5.825926 1 2
591
+ 583 69.8113 -4.597159 -4.524595 2 2
592
+ 584 69.6918 -4.776896 -4.708536 2 1
593
+ 585 69.7436 -0.398475 -1.358932 1 1
594
+ 586 69.7952 -3.938502 -4.461357 1 1
595
+ 587 69.6763 -5.118460 -4.860347 2 1
596
+ 588 69.7279 -4.893744 -2.643502 2 2
597
+ 589 69.6095 -3.201250 -3.556432 1 2
598
+ 590 69.6610 -4.477045 -3.841178 2 2
599
+ 591 69.5431 -4.049280 -4.269371 1 2
600
+ 592 69.5946 -3.278380 -4.294524 1 1
601
+ 593 69.6459 -4.475996 -4.452375 2 2
602
+ 594 69.5286 -4.969678 -3.910299 2 1
603
+ 595 69.5798 -1.935283 -1.908164 2 2
604
+ 596 69.6309 -4.759256 -4.380608 2 2
605
+ 597 69.5142 -2.339721 -2.334134 2 1
606
+ 598 69.5652 -3.265895 -3.621665 1 1
607
+ 599 69.6160 -2.843202 -4.238288 1 1
608
+ 600 69.6667 -3.820920 -3.601757 2 2
609
+ 601 69.7171 -3.316078 -3.017525 2 2
610
+ 602 69.7674 -3.290125 -5.267455 1 1
611
+ 603 69.8176 -6.992802 -5.343040 2 2
612
+ 604 69.7020 -3.120823 -3.199729 1 2
613
+ 605 69.5868 -2.742458 -1.876877 2 1
614
+ 606 69.6370 -3.123466 -2.684542 2 2
615
+ 607 69.5222 -2.980958 -2.580410 2 1
616
+ 608 69.5724 -3.049273 -3.155106 1 1
617
+ 609 69.4581 -7.725116 -7.488410 2 1
618
+ 610 69.5082 -2.930653 -3.180752 1 1
619
+ 611 69.3944 -4.495770 -4.425876 2 1
620
+ 612 69.4444 -5.028381 -6.557387 1 1
621
+ 613 69.4943 -3.281708 -2.236650 2 2
622
+ 614 69.3811 -5.549118 -5.630838 1 2
623
+ 615 69.4309 -2.793148 -2.453968 2 2
624
+ 616 69.3182 -5.876601 -6.027724 1 2
625
+ 617 69.3679 -12.970029 -5.626956 2 2
626
+ 618 69.2557 -3.829893 -4.113884 1 2
627
+ 619 69.3053 -3.931231 -3.343196 2 2
628
+ 620 69.1935 -4.820822 -4.540911 2 1
629
+ 621 69.2432 -3.283887 -1.889058 2 2
630
+ 622 69.2926 -3.404828 -3.863433 1 1
631
+ 623 69.3419 -3.217326 -3.385291 1 1
632
+ 624 69.3910 -4.771121 -3.961095 2 2
633
+ 625 69.4400 -4.032954 -6.531628 1 1
634
+ 626 69.3291 -1.701645 -1.034608 2 1
635
+ 627 69.3780 -3.984829 -4.550459 1 1
636
+ 628 69.4268 -3.079484 -2.618582 2 2
637
+ 629 69.4754 -2.093051 -2.534345 1 1
638
+ 630 69.5238 -1.077196 -1.674068 1 1
639
+ 631 69.5721 -2.458644 -3.879257 1 1
640
+ 632 69.6203 -1.856769 -1.253507 2 2
641
+ 633 69.6682 -2.176142 -2.148938 2 2
642
+ 634 69.7161 -2.620533 -3.612919 1 1
643
+ 635 69.6063 -4.610668 -5.216125 1 2
644
+ 636 69.6541 -1.256181 -3.071381 1 1
645
+ 637 69.7017 -3.466657 -5.721221 1 1
646
+ 638 69.7492 -5.850894 -6.660316 1 1
647
+ 639 69.7966 -1.779579 -3.165230 1 1
648
+ 640 69.8438 -2.252471 -2.153774 2 2
649
+ 641 69.7348 -6.948161 -5.673342 2 1
650
+ 642 69.7819 -4.675073 -3.278738 2 2
651
+ 643 69.8289 -1.909254 -1.383476 2 2
652
+ 644 69.8758 -3.333532 -4.314962 1 1
653
+ 645 69.9225 -0.940821 -3.082800 1 1
654
+ 646 69.8142 -4.029971 -3.542520 2 1
655
+ 647 69.7063 -3.460475 -2.177708 2 1
656
+ 648 69.7531 -3.429734 -2.929801 2 2
657
+ 649 69.7997 -3.684370 -3.972038 1 1
658
+ 650 69.8462 -3.910078 -4.430865 1 1
659
+ 651 69.7389 -6.079949 -6.467785 1 2
660
+ 652 69.7853 -5.973388 -4.921860 2 2
661
+ 653 69.8315 -5.197224 -3.933271 2 2
662
+ 654 69.7248 -3.507307 -3.824344 1 2
663
+ 655 69.6183 -2.688394 -2.896633 1 2
664
+ 656 69.6646 -3.931013 -2.470839 2 2
665
+ 657 69.5586 -5.588529 -5.950012 1 2
666
+ 658 69.6049 -4.206418 -2.065855 2 2
667
+ 659 69.6510 -2.389763 -1.882085 2 2
668
+ 660 69.5455 -3.182160 -3.700266 1 2
669
+ 661 69.5915 -3.200798 -3.555077 1 1
670
+ 662 69.6375 -3.342399 -4.323652 1 1
671
+ 663 69.6833 -6.349363 -6.304680 2 2
672
+ 664 69.5783 -4.113813 -3.790952 2 1
673
+ 665 69.6241 -3.614285 -3.465667 2 2
674
+ 666 69.6697 -3.076039 -4.680181 1 1
675
+ 667 69.7151 -7.997390 -6.407446 2 2
676
+ 668 69.7605 -2.148895 -2.297364 1 1
677
+ 669 69.8057 -2.417657 -3.076185 1 1
678
+ 670 69.7015 -3.566892 -3.237128 2 1
679
+ 671 69.7466 -2.994892 -4.607224 1 1
680
+ 672 69.7917 -3.117777 -2.968127 2 2
681
+ 673 69.8366 -1.891711 -1.598826 2 2
682
+ 674 69.7329 -5.159665 -5.060377 2 1
683
+ 675 69.7778 -3.610134 -3.768632 1 1
684
+ 676 69.6746 -2.136919 -2.715897 1 2
685
+ 677 69.7194 -3.870634 -4.576659 1 1
686
+ 678 69.6165 -5.672290 -6.129815 1 2
687
+ 679 69.6613 -5.986113 -4.965946 2 2
688
+ 680 69.7059 -3.180578 -4.107046 1 1
689
+ 681 69.7504 -2.904713 -3.286500 1 1
690
+ 682 69.7947 -2.301820 -2.242694 2 2
691
+ 683 69.8389 -1.998415 -3.546243 1 1
692
+ 684 69.8830 -4.809189 -7.490988 1 1
693
+ 685 69.9270 -4.572864 -2.290890 2 2
694
+ 686 69.9708 -5.940693 -5.627978 2 2
695
+ 687 70.0146 -4.441198 -3.365161 2 2
696
+ 688 69.9128 -8.446533 -8.440574 2 1
697
+ 689 69.9565 -2.617133 -2.279104 2 2
698
+ 690 69.8551 -2.861693 -2.980692 1 2
699
+ 691 69.7540 -2.922286 -2.490875 2 1
700
+ 692 69.7977 -1.811694 -2.606497 1 1
701
+ 693 69.8413 -4.800060 -5.341886 1 1
702
+ 694 69.8847 -3.415308 -3.371275 2 2
703
+ 695 69.9281 -3.989062 -4.924637 1 1
704
+ 696 69.9713 -4.879761 -5.661791 1 1
705
+ 697 70.0143 -2.177780 -2.657067 1 1
706
+ 698 70.0573 -4.083361 -3.861495 2 2
707
+ 699 70.1001 -3.493206 -4.809817 1 1
708
+ 700 70.1429 -4.471453 -4.102763 2 2
709
+ 701 70.0428 -2.462921 -2.239803 2 1
710
+ 702 69.9430 -4.065217 -4.995225 1 2
711
+ 703 69.9858 -3.572646 -3.927087 1 1
712
+ 704 70.0284 -1.140627 -1.479052 1 1
713
+ 705 69.9291 -3.415455 -3.583421 1 2
714
+ 706 69.9717 -3.349732 -3.042390 2 2
715
+ 707 69.8727 -4.807504 -4.545870 2 1
716
+ 708 69.9153 -3.609910 -2.979237 2 2
717
+ 709 69.9577 -2.954799 -2.435830 2 2
718
+ 710 70.0000 -4.023971 -5.102896 1 1
719
+ 711 69.9015 -1.693752 -2.262618 1 2
720
+ 712 69.8034 -2.315007 -3.483427 1 2
721
+ 713 69.7055 -3.287542 -3.291941 1 2
722
+ 714 69.6078 -3.034885 -3.105074 1 2
723
+ 715 69.6503 -4.199194 -3.654145 2 2
724
+ 716 69.6927 -4.555048 -4.486360 2 2
725
+ 717 69.7350 -2.327834 -0.254024 2 2
726
+ 718 69.6379 -5.205238 -5.127546 2 1
727
+ 719 69.5410 -5.287944 -4.900148 2 1
728
+ 720 69.4444 -2.406416 -5.038033 1 2
729
+ 721 69.3481 -4.884575 -5.918016 1 2
730
+ 722 69.2521 -2.191459 -1.742924 2 1
731
+ 723 69.2946 -2.462234 -2.751147 1 1
732
+ 724 69.3370 -4.512762 -4.290971 2 2
733
+ 725 69.2414 -2.454534 -3.365672 1 2
734
+ 726 69.2837 -4.375572 -4.035356 2 2
735
+ 727 69.3260 -3.360254 -2.582024 2 2
736
+ 728 69.2308 -3.447665 -3.380948 2 1
737
+ 729 69.2730 -0.986296 -2.429476 1 1
738
+ 730 69.3151 -4.040970 -4.480018 1 1
739
+ 731 69.2202 -5.527979 -4.848762 2 1
740
+ 732 69.2623 -2.980435 -2.135050 2 2
741
+ 733 69.3042 -6.624880 -6.606243 2 2
742
+ 734 69.3460 -0.870027 -11.429209 1 1
743
+ 735 69.2517 -3.565687 -2.984898 2 1
744
+ 736 69.2935 -2.385855 -3.121599 1 1
745
+ 737 69.3351 -4.816315 -5.428697 1 1
746
+ 738 69.2412 -4.270665 -4.525383 1 2
747
+ 739 69.2828 -0.793353 -7.025385 1 1
748
+ 740 69.3243 -4.339230 -3.132373 2 2
749
+ 741 69.2308 -4.847161 -4.304366 2 1
750
+ 742 69.2722 -2.395802 -2.306322 2 2
751
+ 743 69.1790 -3.905902 -4.008987 1 2
752
+ 744 69.2204 -4.876571 -4.127134 2 2
753
+ 745 69.2617 -2.547682 -3.406745 1 1
754
+ 746 69.1689 -7.685602 -7.519474 2 1
755
+ 747 69.2102 -5.643511 -6.705571 1 1
756
+ 748 69.2513 -4.879387 -3.762357 2 2
757
+ 749 69.2924 -4.172882 -5.408639 1 1
758
+ 750 69.3333 -3.976584 -7.562452 1 1
759
+
760
+ Final Winogrande score(750 tasks): 69.3333 +/- 1.6849
scores/Hammer2.1-7b-q3_k_m.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 66.66666667
20
+ 10 70.00000000
21
+ 11 63.63636364
22
+ 12 66.66666667
23
+ 13 69.23076923
24
+ 14 71.42857143
25
+ 15 73.33333333
26
+ 16 68.75000000
27
+ 17 70.58823529
28
+ 18 66.66666667
29
+ 19 63.15789474
30
+ 20 60.00000000
31
+ 21 57.14285714
32
+ 22 59.09090909
33
+ 23 60.86956522
34
+ 24 62.50000000
35
+ 25 60.00000000
36
+ 26 57.69230769
37
+ 27 59.25925926
38
+ 28 57.14285714
39
+ 29 58.62068966
40
+ 30 60.00000000
41
+ 31 58.06451613
42
+ 32 56.25000000
43
+ 33 57.57575758
44
+ 34 55.88235294
45
+ 35 57.14285714
46
+ 36 55.55555556
47
+ 37 56.75675676
48
+ 38 55.26315789
49
+ 39 56.41025641
50
+ 40 55.00000000
51
+ 41 56.09756098
52
+ 42 57.14285714
53
+ 43 58.13953488
54
+ 44 56.81818182
55
+ 45 55.55555556
56
+ 46 56.52173913
57
+ 47 57.44680851
58
+ 48 56.25000000
59
+ 49 55.10204082
60
+ 50 54.00000000
61
+ 51 52.94117647
62
+ 52 53.84615385
63
+ 53 54.71698113
64
+ 54 55.55555556
65
+ 55 54.54545455
66
+ 56 55.35714286
67
+ 57 56.14035088
68
+ 58 56.89655172
69
+ 59 57.62711864
70
+ 60 58.33333333
71
+ 61 57.37704918
72
+ 62 58.06451613
73
+ 63 58.73015873
74
+ 64 59.37500000
75
+ 65 58.46153846
76
+ 66 59.09090909
77
+ 67 59.70149254
78
+ 68 60.29411765
79
+ 69 59.42028986
80
+ 70 58.57142857
81
+ 71 57.74647887
82
+ 72 56.94444444
83
+ 73 57.53424658
84
+ 74 58.10810811
85
+ 75 58.66666667
86
+ 76 57.89473684
87
+ 77 58.44155844
88
+ 78 58.97435897
89
+ 79 58.22784810
90
+ 80 58.75000000
91
+ 81 58.02469136
92
+ 82 58.53658537
93
+ 83 57.83132530
94
+ 84 57.14285714
95
+ 85 57.64705882
96
+ 86 58.13953488
97
+ 87 58.62068966
98
+ 88 59.09090909
99
+ 89 59.55056180
100
+ 90 58.88888889
101
+ 91 59.34065934
102
+ 92 59.78260870
103
+ 93 59.13978495
104
+ 94 58.51063830
105
+ 95 58.94736842
106
+ 96 59.37500000
107
+ 97 58.76288660
108
+ 98 58.16326531
109
+ 99 58.58585859
110
+ 100 59.00000000
111
+ 101 58.41584158
112
+ 102 58.82352941
113
+ 103 58.25242718
114
+ 104 57.69230769
115
+ 105 57.14285714
116
+ 106 56.60377358
117
+ 107 57.00934579
118
+ 108 56.48148148
119
+ 109 55.96330275
120
+ 110 56.36363636
121
+ 111 56.75675676
122
+ 112 57.14285714
123
+ 113 56.63716814
124
+ 114 56.14035088
125
+ 115 56.52173913
126
+ 116 56.03448276
127
+ 117 55.55555556
128
+ 118 55.93220339
129
+ 119 55.46218487
130
+ 120 55.83333333
131
+ 121 55.37190083
132
+ 122 55.73770492
133
+ 123 56.09756098
134
+ 124 55.64516129
135
+ 125 56.00000000
136
+ 126 56.34920635
137
+ 127 55.90551181
138
+ 128 56.25000000
139
+ 129 55.81395349
140
+ 130 55.38461538
141
+ 131 55.72519084
142
+ 132 56.06060606
143
+ 133 56.39097744
144
+ 134 55.97014925
145
+ 135 56.29629630
146
+ 136 55.88235294
147
+ 137 56.20437956
148
+ 138 56.52173913
149
+ 139 56.83453237
150
+ 140 57.14285714
151
+ 141 57.44680851
152
+ 142 57.04225352
153
+ 143 57.34265734
154
+ 144 56.94444444
155
+ 145 57.24137931
156
+ 146 56.84931507
157
+ 147 57.14285714
158
+ 148 56.75675676
159
+ 149 57.04697987
160
+ 150 57.33333333
161
+ 151 57.61589404
162
+ 152 57.89473684
163
+ 153 58.16993464
164
+ 154 57.79220779
165
+ 155 57.41935484
166
+ 156 57.69230769
167
+ 157 57.96178344
168
+ 158 58.22784810
169
+ 159 58.49056604
170
+ 160 58.12500000
171
+ 161 57.76397516
172
+ 162 58.02469136
173
+ 163 57.66871166
174
+ 164 57.31707317
175
+ 165 57.57575758
176
+ 166 57.83132530
177
+ 167 57.48502994
178
+ 168 57.73809524
179
+ 169 57.98816568
180
+ 170 58.23529412
181
+ 171 57.89473684
182
+ 172 58.13953488
183
+ 173 58.38150289
184
+ 174 58.62068966
185
+ 175 58.28571429
186
+ 176 57.95454545
187
+ 177 58.19209040
188
+ 178 57.86516854
189
+ 179 57.54189944
190
+ 180 57.22222222
191
+ 181 57.45856354
192
+ 182 57.69230769
193
+ 183 57.37704918
194
+ 184 57.06521739
195
+ 185 57.29729730
196
+ 186 57.52688172
197
+ 187 57.75401070
198
+ 188 57.44680851
199
+ 189 57.14285714
200
+ 190 56.84210526
201
+ 191 57.06806283
202
+ 192 56.77083333
203
+ 193 56.47668394
204
+ 194 56.70103093
205
+ 195 56.92307692
206
+ 196 57.14285714
207
+ 197 57.36040609
208
+ 198 57.07070707
209
+ 199 56.78391960
210
+ 200 57.00000000
211
+ 201 57.21393035
212
+ 202 56.93069307
213
+ 203 57.14285714
214
+ 204 57.35294118
215
+ 205 57.07317073
216
+ 206 56.79611650
217
+ 207 57.00483092
218
+ 208 57.21153846
219
+ 209 57.41626794
220
+ 210 57.61904762
221
+ 211 57.81990521
222
+ 212 57.54716981
223
+ 213 57.27699531
224
+ 214 57.47663551
225
+ 215 57.67441860
226
+ 216 57.40740741
227
+ 217 57.14285714
228
+ 218 56.88073394
229
+ 219 57.07762557
230
+ 220 57.27272727
231
+ 221 57.01357466
232
+ 222 57.20720721
233
+ 223 56.95067265
234
+ 224 57.14285714
235
+ 225 56.88888889
236
+ 226 56.63716814
237
+ 227 56.82819383
238
+ 228 57.01754386
239
+ 229 57.20524017
240
+ 230 57.39130435
241
+ 231 57.14285714
242
+ 232 56.89655172
243
+ 233 57.08154506
244
+ 234 57.26495726
245
+ 235 57.02127660
246
+ 236 57.20338983
247
+ 237 57.38396624
248
+ 238 57.14285714
249
+ 239 57.32217573
250
+ 240 57.50000000
251
+ 241 57.67634855
252
+ 242 57.43801653
253
+ 243 57.20164609
254
+ 244 56.96721311
255
+ 245 57.14285714
256
+ 246 57.31707317
257
+ 247 57.48987854
258
+ 248 57.66129032
259
+ 249 57.42971888
260
+ 250 57.60000000
261
+ 251 57.76892430
262
+ 252 57.93650794
263
+ 253 58.10276680
264
+ 254 57.87401575
265
+ 255 58.03921569
266
+ 256 58.20312500
267
+ 257 58.36575875
268
+ 258 58.52713178
269
+ 259 58.30115830
270
+ 260 58.46153846
271
+ 261 58.62068966
272
+ 262 58.77862595
273
+ 263 58.93536122
274
+ 264 59.09090909
275
+ 265 58.86792453
276
+ 266 58.64661654
277
+ 267 58.80149813
278
+ 268 58.58208955
279
+ 269 58.73605948
280
+ 270 58.88888889
281
+ 271 59.04059041
282
+ 272 58.82352941
283
+ 273 58.60805861
284
+ 274 58.39416058
285
+ 275 58.18181818
286
+ 276 58.33333333
287
+ 277 58.48375451
288
+ 278 58.27338129
289
+ 279 58.42293907
290
+ 280 58.57142857
291
+ 281 58.71886121
292
+ 282 58.86524823
293
+ 283 58.65724382
294
+ 284 58.80281690
295
+ 285 58.94736842
296
+ 286 59.09090909
297
+ 287 58.88501742
298
+ 288 59.02777778
299
+ 289 58.82352941
300
+ 290 58.96551724
301
+ 291 59.10652921
302
+ 292 58.90410959
303
+ 293 59.04436860
304
+ 294 58.84353741
305
+ 295 58.98305085
306
+ 296 59.12162162
307
+ 297 59.25925926
308
+ 298 59.39597315
309
+ 299 59.53177258
310
+ 300 59.33333333
311
+ 301 59.13621262
312
+ 302 59.27152318
313
+ 303 59.40594059
314
+ 304 59.53947368
315
+ 305 59.67213115
316
+ 306 59.47712418
317
+ 307 59.28338762
318
+ 308 59.41558442
319
+ 309 59.54692557
320
+ 310 59.35483871
321
+ 311 59.48553055
322
+ 312 59.29487179
323
+ 313 59.42492013
324
+ 314 59.23566879
325
+ 315 59.36507937
326
+ 316 59.49367089
327
+ 317 59.62145110
328
+ 318 59.43396226
329
+ 319 59.24764890
330
+ 320 59.06250000
331
+ 321 58.87850467
332
+ 322 59.00621118
333
+ 323 59.13312693
334
+ 324 58.95061728
335
+ 325 58.76923077
336
+ 326 58.89570552
337
+ 327 58.71559633
338
+ 328 58.84146341
339
+ 329 58.96656535
340
+ 330 59.09090909
341
+ 331 59.21450151
342
+ 332 59.33734940
343
+ 333 59.45945946
344
+ 334 59.58083832
345
+ 335 59.40298507
346
+ 336 59.52380952
347
+ 337 59.34718101
348
+ 338 59.17159763
349
+ 339 58.99705015
350
+ 340 59.11764706
351
+ 341 59.23753666
352
+ 342 59.35672515
353
+ 343 59.47521866
354
+ 344 59.59302326
355
+ 345 59.42028986
356
+ 346 59.53757225
357
+ 347 59.36599424
358
+ 348 59.19540230
359
+ 349 59.31232092
360
+ 350 59.14285714
361
+ 351 59.25925926
362
+ 352 59.37500000
363
+ 353 59.49008499
364
+ 354 59.32203390
365
+ 355 59.15492958
366
+ 356 59.26966292
367
+ 357 59.10364146
368
+ 358 58.93854749
369
+ 359 58.77437326
370
+ 360 58.88888889
371
+ 361 59.00277008
372
+ 362 59.11602210
373
+ 363 59.22865014
374
+ 364 59.06593407
375
+ 365 58.90410959
376
+ 366 58.74316940
377
+ 367 58.85558583
378
+ 368 58.96739130
379
+ 369 59.07859079
380
+ 370 59.18918919
381
+ 371 59.02964960
382
+ 372 58.87096774
383
+ 373 58.71313673
384
+ 374 58.82352941
385
+ 375 58.93333333
386
+ 376 58.77659574
387
+ 377 58.62068966
388
+ 378 58.73015873
389
+ 379 58.83905013
390
+ 380 58.94736842
391
+ 381 58.79265092
392
+ 382 58.63874346
393
+ 383 58.48563969
394
+ 384 58.59375000
395
+ 385 58.70129870
396
+ 386 58.80829016
397
+ 387 58.65633075
398
+ 388 58.50515464
399
+ 389 58.35475578
400
+ 390 58.20512821
401
+ 391 58.05626598
402
+ 392 58.16326531
403
+ 393 58.01526718
404
+ 394 58.12182741
405
+ 395 58.22784810
406
+ 396 58.08080808
407
+ 397 58.18639798
408
+ 398 58.04020101
409
+ 399 58.14536341
410
+ 400 58.25000000
411
+ 401 58.35411471
412
+ 402 58.45771144
413
+ 403 58.56079404
414
+ 404 58.66336634
415
+ 405 58.76543210
416
+ 406 58.86699507
417
+ 407 58.96805897
418
+ 408 59.06862745
419
+ 409 58.92420538
420
+ 410 59.02439024
421
+ 411 59.12408759
422
+ 412 59.22330097
423
+ 413 59.07990315
424
+ 414 59.17874396
425
+ 415 59.03614458
426
+ 416 58.89423077
427
+ 417 58.75299760
428
+ 418 58.85167464
429
+ 419 58.71121718
430
+ 420 58.57142857
431
+ 421 58.43230404
432
+ 422 58.53080569
433
+ 423 58.62884161
434
+ 424 58.49056604
435
+ 425 58.58823529
436
+ 426 58.45070423
437
+ 427 58.54800937
438
+ 428 58.41121495
439
+ 429 58.50815851
440
+ 430 58.37209302
441
+ 431 58.46867749
442
+ 432 58.56481481
443
+ 433 58.42956120
444
+ 434 58.52534562
445
+ 435 58.39080460
446
+ 436 58.48623853
447
+ 437 58.35240275
448
+ 438 58.21917808
449
+ 439 58.31435080
450
+ 440 58.18181818
451
+ 441 58.27664399
452
+ 442 58.14479638
453
+ 443 58.01354402
454
+ 444 58.10810811
455
+ 445 58.20224719
456
+ 446 58.29596413
457
+ 447 58.38926174
458
+ 448 58.25892857
459
+ 449 58.35189310
460
+ 450 58.44444444
461
+ 451 58.53658537
462
+ 452 58.62831858
463
+ 453 58.49889625
464
+ 454 58.37004405
465
+ 455 58.46153846
466
+ 456 58.33333333
467
+ 457 58.42450766
468
+ 458 58.51528384
469
+ 459 58.60566449
470
+ 460 58.47826087
471
+ 461 58.35140998
472
+ 462 58.44155844
473
+ 463 58.31533477
474
+ 464 58.18965517
475
+ 465 58.06451613
476
+ 466 58.15450644
477
+ 467 58.24411135
478
+ 468 58.33333333
479
+ 469 58.20895522
480
+ 470 58.29787234
481
+ 471 58.38641189
482
+ 472 58.26271186
483
+ 473 58.13953488
484
+ 474 58.22784810
485
+ 475 58.10526316
486
+ 476 57.98319328
487
+ 477 58.07127883
488
+ 478 58.15899582
489
+ 479 58.24634656
490
+ 480 58.33333333
491
+ 481 58.21205821
492
+ 482 58.29875519
493
+ 483 58.38509317
494
+ 484 58.26446281
495
+ 485 58.35051546
496
+ 486 58.43621399
497
+ 487 58.31622177
498
+ 488 58.40163934
499
+ 489 58.28220859
500
+ 490 58.16326531
501
+ 491 58.04480652
502
+ 492 58.13008130
503
+ 493 58.01217039
504
+ 494 58.09716599
505
+ 495 57.97979798
506
+ 496 58.06451613
507
+ 497 58.14889336
508
+ 498 58.23293173
509
+ 499 58.31663327
510
+ 500 58.40000000
511
+ 501 58.28343313
512
+ 502 58.36653386
513
+ 503 58.44930417
514
+ 504 58.33333333
515
+ 505 58.41584158
516
+ 506 58.30039526
517
+ 507 58.18540434
518
+ 508 58.07086614
519
+ 509 57.95677800
520
+ 510 58.03921569
521
+ 511 57.92563601
522
+ 512 58.00781250
523
+ 513 57.89473684
524
+ 514 57.97665370
525
+ 515 58.05825243
526
+ 516 57.94573643
527
+ 517 58.02707930
528
+ 518 58.10810811
529
+ 519 57.99614644
530
+ 520 57.88461538
531
+ 521 57.96545106
532
+ 522 57.85440613
533
+ 523 57.74378585
534
+ 524 57.63358779
535
+ 525 57.52380952
536
+ 526 57.41444867
537
+ 527 57.30550285
538
+ 528 57.19696970
539
+ 529 57.08884688
540
+ 530 57.16981132
541
+ 531 57.06214689
542
+ 532 57.14285714
543
+ 533 57.22326454
544
+ 534 57.11610487
545
+ 535 57.00934579
546
+ 536 57.08955224
547
+ 537 57.16945996
548
+ 538 57.06319703
549
+ 539 57.14285714
550
+ 540 57.03703704
551
+ 541 57.11645102
552
+ 542 57.01107011
553
+ 543 57.09023941
554
+ 544 56.98529412
555
+ 545 57.06422018
556
+ 546 56.95970696
557
+ 547 56.85557587
558
+ 548 56.93430657
559
+ 549 57.01275046
560
+ 550 57.09090909
561
+ 551 56.98729583
562
+ 552 56.88405797
563
+ 553 56.96202532
564
+ 554 57.03971119
565
+ 555 57.11711712
566
+ 556 57.01438849
567
+ 557 57.09156194
568
+ 558 56.98924731
569
+ 559 57.06618962
570
+ 560 57.14285714
571
+ 561 57.21925134
572
+ 562 57.29537367
573
+ 563 57.37122558
574
+ 564 57.26950355
575
+ 565 57.16814159
576
+ 566 57.24381625
577
+ 567 57.14285714
578
+ 568 57.21830986
579
+ 569 57.11775044
580
+ 570 57.01754386
581
+ 571 57.09281961
582
+ 572 56.99300699
583
+ 573 57.06806283
584
+ 574 57.14285714
585
+ 575 57.21739130
586
+ 576 57.29166667
587
+ 577 57.36568458
588
+ 578 57.43944637
589
+ 579 57.51295337
590
+ 580 57.41379310
591
+ 581 57.31497418
592
+ 582 57.21649485
593
+ 583 57.28987993
594
+ 584 57.36301370
595
+ 585 57.43589744
596
+ 586 57.33788396
597
+ 587 57.24020443
598
+ 588 57.14285714
599
+ 589 57.21561969
600
+ 590 57.28813559
601
+ 591 57.36040609
602
+ 592 57.26351351
603
+ 593 57.33558179
604
+ 594 57.40740741
605
+ 595 57.31092437
606
+ 596 57.21476510
607
+ 597 57.11892797
608
+ 598 57.02341137
609
+ 599 56.92821369
610
+ 600 57.00000000
611
+ 601 57.07154742
612
+ 602 56.97674419
613
+ 603 56.88225539
614
+ 604 56.78807947
615
+ 605 56.69421488
616
+ 606 56.76567657
617
+ 607 56.83690280
618
+ 608 56.90789474
619
+ 609 56.81444992
620
+ 610 56.88524590
621
+ 611 56.79214403
622
+ 612 56.69934641
623
+ 613 56.60685155
624
+ 614 56.51465798
625
+ 615 56.42276423
626
+ 616 56.33116883
627
+ 617 56.40194489
628
+ 618 56.31067961
629
+ 619 56.21970921
630
+ 620 56.29032258
631
+ 621 56.36070853
632
+ 622 56.27009646
633
+ 623 56.34028892
634
+ 624 56.25000000
635
+ 625 56.16000000
636
+ 626 56.23003195
637
+ 627 56.29984051
638
+ 628 56.21019108
639
+ 629 56.27980922
640
+ 630 56.34920635
641
+ 631 56.25990491
642
+ 632 56.17088608
643
+ 633 56.08214850
644
+ 634 56.15141956
645
+ 635 56.22047244
646
+ 636 56.28930818
647
+ 637 56.20094192
648
+ 638 56.26959248
649
+ 639 56.33802817
650
+ 640 56.25000000
651
+ 641 56.16224649
652
+ 642 56.07476636
653
+ 643 55.98755832
654
+ 644 56.05590062
655
+ 645 55.96899225
656
+ 646 56.03715170
657
+ 647 56.10510046
658
+ 648 56.01851852
659
+ 649 55.93220339
660
+ 650 55.84615385
661
+ 651 55.91397849
662
+ 652 55.82822086
663
+ 653 55.89586524
664
+ 654 55.96330275
665
+ 655 55.87786260
666
+ 656 55.94512195
667
+ 657 55.85996956
668
+ 658 55.77507599
669
+ 659 55.84218513
670
+ 660 55.90909091
671
+ 661 55.82450832
672
+ 662 55.89123867
673
+ 663 55.80693816
674
+ 664 55.87349398
675
+ 665 55.93984962
676
+ 666 56.00600601
677
+ 667 56.07196402
678
+ 668 55.98802395
679
+ 669 55.90433483
680
+ 670 55.97014925
681
+ 671 56.03576751
682
+ 672 55.95238095
683
+ 673 55.86924220
684
+ 674 55.78635015
685
+ 675 55.85185185
686
+ 676 55.91715976
687
+ 677 55.98227474
688
+ 678 56.04719764
689
+ 679 56.11192931
690
+ 680 56.17647059
691
+ 681 56.09397944
692
+ 682 56.01173021
693
+ 683 55.92972182
694
+ 684 55.99415205
695
+ 685 56.05839416
696
+ 686 55.97667638
697
+ 687 55.89519651
698
+ 688 55.95930233
699
+ 689 56.02322206
700
+ 690 56.08695652
701
+ 691 56.15050651
702
+ 692 56.21387283
703
+ 693 56.27705628
704
+ 694 56.34005764
705
+ 695 56.25899281
706
+ 696 56.17816092
707
+ 697 56.24103300
708
+ 698 56.16045845
709
+ 699 56.08011445
710
+ 700 56.14285714
711
+ 701 56.20542083
712
+ 702 56.26780627
713
+ 703 56.18776671
714
+ 704 56.25000000
715
+ 705 56.31205674
716
+ 706 56.37393768
717
+ 707 56.43564356
718
+ 708 56.49717514
719
+ 709 56.55853315
720
+ 710 56.47887324
721
+ 711 56.54008439
722
+ 712 56.60112360
723
+ 713 56.66199158
724
+ 714 56.58263305
725
+ 715 56.50349650
726
+ 716 56.42458101
727
+ 717 56.34588563
728
+ 718 56.26740947
729
+ 719 56.18915160
730
+ 720 56.25000000
731
+ 721 56.17198336
732
+ 722 56.23268698
733
+ 723 56.29322268
734
+ 724 56.35359116
735
+ 725 56.41379310
736
+ 726 56.33608815
737
+ 727 56.25859697
738
+ 728 56.31868132
739
+ 729 56.37860082
740
+ 730 56.30136986
741
+ 731 56.36114911
742
+ 732 56.42076503
743
+ 733 56.48021828
744
+ 734 56.53950954
745
+ 735 56.46258503
746
+ 736 56.52173913
747
+ 737 56.44504749
748
+ 738 56.36856369
749
+ 739 56.29228687
750
+ 740 56.21621622
751
+ 741 56.27530364
752
+ 742 56.33423181
753
+ 743 56.39300135
754
+ 744 56.31720430
755
+ 745 56.37583893
756
+ 746 56.30026810
757
+ 747 56.22489960
758
+ 748 56.14973262
759
+ 749 56.07476636
760
+ 750 56.13333333
761
+
762
+ Final result: 56.1333 +/- 1.8132
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-q3_k_m.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 75.00000000
15
+ 5 80.00000000
16
+ 6 83.33333333
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 77.77777778
20
+ 10 80.00000000
21
+ 11 81.81818182
22
+ 12 75.00000000
23
+ 13 76.92307692
24
+ 14 78.57142857
25
+ 15 80.00000000
26
+ 16 81.25000000
27
+ 17 76.47058824
28
+ 18 72.22222222
29
+ 19 68.42105263
30
+ 20 70.00000000
31
+ 21 71.42857143
32
+ 22 72.72727273
33
+ 23 73.91304348
34
+ 24 70.83333333
35
+ 25 68.00000000
36
+ 26 65.38461538
37
+ 27 62.96296296
38
+ 28 64.28571429
39
+ 29 65.51724138
40
+ 30 63.33333333
41
+ 31 64.51612903
42
+ 32 62.50000000
43
+ 33 63.63636364
44
+ 34 64.70588235
45
+ 35 65.71428571
46
+ 36 63.88888889
47
+ 37 62.16216216
48
+ 38 63.15789474
49
+ 39 64.10256410
50
+ 40 65.00000000
51
+ 41 65.85365854
52
+ 42 64.28571429
53
+ 43 62.79069767
54
+ 44 63.63636364
55
+ 45 62.22222222
56
+ 46 60.86956522
57
+ 47 59.57446809
58
+ 48 58.33333333
59
+ 49 59.18367347
60
+ 50 60.00000000
61
+ 51 60.78431373
62
+ 52 59.61538462
63
+ 53 58.49056604
64
+ 54 57.40740741
65
+ 55 56.36363636
66
+ 56 55.35714286
67
+ 57 56.14035088
68
+ 58 56.89655172
69
+ 59 57.62711864
70
+ 60 58.33333333
71
+ 61 57.37704918
72
+ 62 56.45161290
73
+ 63 57.14285714
74
+ 64 57.81250000
75
+ 65 56.92307692
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 57.97101449
80
+ 70 58.57142857
81
+ 71 57.74647887
82
+ 72 58.33333333
83
+ 73 58.90410959
84
+ 74 59.45945946
85
+ 75 58.66666667
86
+ 76 59.21052632
87
+ 77 59.74025974
88
+ 78 60.25641026
89
+ 79 60.75949367
90
+ 80 61.25000000
91
+ 81 61.72839506
92
+ 82 62.19512195
93
+ 83 62.65060241
94
+ 84 63.09523810
95
+ 85 62.35294118
96
+ 86 62.79069767
97
+ 87 63.21839080
98
+ 88 63.63636364
99
+ 89 62.92134831
100
+ 90 63.33333333
101
+ 91 63.73626374
102
+ 92 63.04347826
103
+ 93 63.44086022
104
+ 94 63.82978723
105
+ 95 64.21052632
106
+ 96 64.58333333
107
+ 97 64.94845361
108
+ 98 65.30612245
109
+ 99 65.65656566
110
+ 100 65.00000000
111
+ 101 65.34653465
112
+ 102 64.70588235
113
+ 103 64.07766990
114
+ 104 64.42307692
115
+ 105 64.76190476
116
+ 106 65.09433962
117
+ 107 65.42056075
118
+ 108 65.74074074
119
+ 109 65.13761468
120
+ 110 65.45454545
121
+ 111 65.76576577
122
+ 112 66.07142857
123
+ 113 66.37168142
124
+ 114 66.66666667
125
+ 115 66.95652174
126
+ 116 67.24137931
127
+ 117 66.66666667
128
+ 118 66.94915254
129
+ 119 67.22689076
130
+ 120 67.50000000
131
+ 121 66.94214876
132
+ 122 67.21311475
133
+ 123 67.47967480
134
+ 124 66.93548387
135
+ 125 67.20000000
136
+ 126 67.46031746
137
+ 127 67.71653543
138
+ 128 67.18750000
139
+ 129 67.44186047
140
+ 130 66.92307692
141
+ 131 67.17557252
142
+ 132 66.66666667
143
+ 133 66.91729323
144
+ 134 67.16417910
145
+ 135 67.40740741
146
+ 136 67.64705882
147
+ 137 67.15328467
148
+ 138 67.39130435
149
+ 139 67.62589928
150
+ 140 67.14285714
151
+ 141 67.37588652
152
+ 142 67.60563380
153
+ 143 67.83216783
154
+ 144 68.05555556
155
+ 145 68.27586207
156
+ 146 68.49315068
157
+ 147 68.70748299
158
+ 148 68.91891892
159
+ 149 69.12751678
160
+ 150 69.33333333
161
+ 151 68.87417219
162
+ 152 68.42105263
163
+ 153 68.62745098
164
+ 154 68.83116883
165
+ 155 69.03225806
166
+ 156 68.58974359
167
+ 157 68.15286624
168
+ 158 68.35443038
169
+ 159 67.92452830
170
+ 160 68.12500000
171
+ 161 68.32298137
172
+ 162 68.51851852
173
+ 163 68.71165644
174
+ 164 68.90243902
175
+ 165 69.09090909
176
+ 166 69.27710843
177
+ 167 69.46107784
178
+ 168 69.64285714
179
+ 169 69.23076923
180
+ 170 69.41176471
181
+ 171 69.59064327
182
+ 172 69.18604651
183
+ 173 68.78612717
184
+ 174 68.39080460
185
+ 175 68.57142857
186
+ 176 68.75000000
187
+ 177 68.92655367
188
+ 178 69.10112360
189
+ 179 68.71508380
190
+ 180 68.88888889
191
+ 181 68.50828729
192
+ 182 68.68131868
193
+ 183 68.30601093
194
+ 184 67.93478261
195
+ 185 68.10810811
196
+ 186 67.74193548
197
+ 187 67.37967914
198
+ 188 67.02127660
199
+ 189 66.66666667
200
+ 190 66.84210526
201
+ 191 67.01570681
202
+ 192 67.18750000
203
+ 193 66.83937824
204
+ 194 67.01030928
205
+ 195 66.66666667
206
+ 196 66.83673469
207
+ 197 66.49746193
208
+ 198 66.16161616
209
+ 199 66.33165829
210
+ 200 66.50000000
211
+ 201 66.66666667
212
+ 202 66.83168317
213
+ 203 66.50246305
214
+ 204 66.66666667
215
+ 205 66.34146341
216
+ 206 66.01941748
217
+ 207 66.18357488
218
+ 208 65.86538462
219
+ 209 66.02870813
220
+ 210 66.19047619
221
+ 211 66.35071090
222
+ 212 66.50943396
223
+ 213 66.66666667
224
+ 214 66.82242991
225
+ 215 66.51162791
226
+ 216 66.66666667
227
+ 217 66.35944700
228
+ 218 66.05504587
229
+ 219 66.21004566
230
+ 220 66.36363636
231
+ 221 66.06334842
232
+ 222 66.21621622
233
+ 223 66.36771300
234
+ 224 66.51785714
235
+ 225 66.66666667
236
+ 226 66.81415929
237
+ 227 66.51982379
238
+ 228 66.66666667
239
+ 229 66.81222707
240
+ 230 66.52173913
241
+ 231 66.23376623
242
+ 232 65.94827586
243
+ 233 65.66523605
244
+ 234 65.81196581
245
+ 235 65.53191489
246
+ 236 65.25423729
247
+ 237 65.40084388
248
+ 238 65.54621849
249
+ 239 65.69037657
250
+ 240 65.83333333
251
+ 241 65.97510373
252
+ 242 66.11570248
253
+ 243 66.25514403
254
+ 244 65.98360656
255
+ 245 65.71428571
256
+ 246 65.85365854
257
+ 247 65.99190283
258
+ 248 66.12903226
259
+ 249 66.26506024
260
+ 250 66.40000000
261
+ 251 66.53386454
262
+ 252 66.66666667
263
+ 253 66.40316206
264
+ 254 66.53543307
265
+ 255 66.66666667
266
+ 256 66.79687500
267
+ 257 66.53696498
268
+ 258 66.66666667
269
+ 259 66.79536680
270
+ 260 66.92307692
271
+ 261 67.04980843
272
+ 262 66.79389313
273
+ 263 66.92015209
274
+ 264 67.04545455
275
+ 265 67.16981132
276
+ 266 66.91729323
277
+ 267 67.04119850
278
+ 268 67.16417910
279
+ 269 67.28624535
280
+ 270 67.40740741
281
+ 271 67.15867159
282
+ 272 67.27941176
283
+ 273 67.03296703
284
+ 274 67.15328467
285
+ 275 66.90909091
286
+ 276 67.02898551
287
+ 277 67.14801444
288
+ 278 67.26618705
289
+ 279 67.38351254
290
+ 280 67.50000000
291
+ 281 67.61565836
292
+ 282 67.73049645
293
+ 283 67.49116608
294
+ 284 67.60563380
295
+ 285 67.36842105
296
+ 286 67.48251748
297
+ 287 67.59581882
298
+ 288 67.70833333
299
+ 289 67.82006920
300
+ 290 67.93103448
301
+ 291 68.04123711
302
+ 292 67.80821918
303
+ 293 67.91808874
304
+ 294 68.02721088
305
+ 295 68.13559322
306
+ 296 68.24324324
307
+ 297 68.35016835
308
+ 298 68.45637584
309
+ 299 68.22742475
310
+ 300 68.00000000
311
+ 301 67.77408638
312
+ 302 67.88079470
313
+ 303 67.65676568
314
+ 304 67.43421053
315
+ 305 67.54098361
316
+ 306 67.64705882
317
+ 307 67.42671010
318
+ 308 67.53246753
319
+ 309 67.63754045
320
+ 310 67.74193548
321
+ 311 67.84565916
322
+ 312 67.94871795
323
+ 313 68.05111821
324
+ 314 67.83439490
325
+ 315 67.93650794
326
+ 316 68.03797468
327
+ 317 68.13880126
328
+ 318 67.92452830
329
+ 319 67.71159875
330
+ 320 67.81250000
331
+ 321 67.60124611
332
+ 322 67.39130435
333
+ 323 67.49226006
334
+ 324 67.59259259
335
+ 325 67.69230769
336
+ 326 67.48466258
337
+ 327 67.27828746
338
+ 328 67.37804878
339
+ 329 67.47720365
340
+ 330 67.57575758
341
+ 331 67.67371601
342
+ 332 67.46987952
343
+ 333 67.56756757
344
+ 334 67.36526946
345
+ 335 67.46268657
346
+ 336 67.55952381
347
+ 337 67.35905045
348
+ 338 67.15976331
349
+ 339 67.25663717
350
+ 340 67.35294118
351
+ 341 67.44868035
352
+ 342 67.54385965
353
+ 343 67.63848397
354
+ 344 67.44186047
355
+ 345 67.53623188
356
+ 346 67.34104046
357
+ 347 67.43515850
358
+ 348 67.24137931
359
+ 349 67.04871060
360
+ 350 67.14285714
361
+ 351 67.23646724
362
+ 352 67.32954545
363
+ 353 67.13881020
364
+ 354 67.23163842
365
+ 355 67.32394366
366
+ 356 67.41573034
367
+ 357 67.50700280
368
+ 358 67.59776536
369
+ 359 67.68802228
370
+ 360 67.50000000
371
+ 361 67.59002770
372
+ 362 67.67955801
373
+ 363 67.76859504
374
+ 364 67.85714286
375
+ 365 67.94520548
376
+ 366 68.03278689
377
+ 367 68.11989101
378
+ 368 68.20652174
379
+ 369 68.29268293
380
+ 370 68.10810811
381
+ 371 68.19407008
382
+ 372 68.27956989
383
+ 373 68.09651475
384
+ 374 68.18181818
385
+ 375 68.26666667
386
+ 376 68.35106383
387
+ 377 68.16976127
388
+ 378 68.25396825
389
+ 379 68.33773087
390
+ 380 68.42105263
391
+ 381 68.50393701
392
+ 382 68.32460733
393
+ 383 68.40731070
394
+ 384 68.48958333
395
+ 385 68.57142857
396
+ 386 68.65284974
397
+ 387 68.47545220
398
+ 388 68.29896907
399
+ 389 68.38046272
400
+ 390 68.46153846
401
+ 391 68.28644501
402
+ 392 68.36734694
403
+ 393 68.44783715
404
+ 394 68.52791878
405
+ 395 68.60759494
406
+ 396 68.68686869
407
+ 397 68.76574307
408
+ 398 68.84422111
409
+ 399 68.92230576
410
+ 400 68.75000000
411
+ 401 68.82793017
412
+ 402 68.65671642
413
+ 403 68.48635236
414
+ 404 68.56435644
415
+ 405 68.39506173
416
+ 406 68.47290640
417
+ 407 68.55036855
418
+ 408 68.62745098
419
+ 409 68.70415648
420
+ 410 68.53658537
421
+ 411 68.36982968
422
+ 412 68.20388350
423
+ 413 68.28087167
424
+ 414 68.35748792
425
+ 415 68.19277108
426
+ 416 68.26923077
427
+ 417 68.34532374
428
+ 418 68.42105263
429
+ 419 68.49642005
430
+ 420 68.57142857
431
+ 421 68.40855107
432
+ 422 68.48341232
433
+ 423 68.55791962
434
+ 424 68.63207547
435
+ 425 68.70588235
436
+ 426 68.77934272
437
+ 427 68.61826698
438
+ 428 68.69158879
439
+ 429 68.53146853
440
+ 430 68.60465116
441
+ 431 68.67749420
442
+ 432 68.75000000
443
+ 433 68.82217090
444
+ 434 68.89400922
445
+ 435 68.73563218
446
+ 436 68.80733945
447
+ 437 68.64988558
448
+ 438 68.49315068
449
+ 439 68.56492027
450
+ 440 68.63636364
451
+ 441 68.70748299
452
+ 442 68.77828054
453
+ 443 68.84875847
454
+ 444 68.91891892
455
+ 445 68.98876404
456
+ 446 69.05829596
457
+ 447 69.12751678
458
+ 448 69.19642857
459
+ 449 69.04231626
460
+ 450 69.11111111
461
+ 451 68.95787140
462
+ 452 69.02654867
463
+ 453 68.87417219
464
+ 454 68.94273128
465
+ 455 69.01098901
466
+ 456 69.07894737
467
+ 457 69.14660832
468
+ 458 69.21397380
469
+ 459 69.28104575
470
+ 460 69.34782609
471
+ 461 69.41431670
472
+ 462 69.48051948
473
+ 463 69.54643629
474
+ 464 69.61206897
475
+ 465 69.46236559
476
+ 466 69.52789700
477
+ 467 69.59314775
478
+ 468 69.44444444
479
+ 469 69.29637527
480
+ 470 69.14893617
481
+ 471 69.21443737
482
+ 472 69.06779661
483
+ 473 69.13319239
484
+ 474 69.19831224
485
+ 475 69.26315789
486
+ 476 69.11764706
487
+ 477 68.97274633
488
+ 478 69.03765690
489
+ 479 69.10229645
490
+ 480 69.16666667
491
+ 481 69.02286902
492
+ 482 68.87966805
493
+ 483 68.94409938
494
+ 484 69.00826446
495
+ 485 68.86597938
496
+ 486 68.93004115
497
+ 487 68.99383984
498
+ 488 68.85245902
499
+ 489 68.91615542
500
+ 490 68.97959184
501
+ 491 69.04276986
502
+ 492 69.10569106
503
+ 493 69.16835700
504
+ 494 69.23076923
505
+ 495 69.29292929
506
+ 496 69.35483871
507
+ 497 69.41649899
508
+ 498 69.27710843
509
+ 499 69.13827655
510
+ 500 69.20000000
511
+ 501 69.26147705
512
+ 502 69.32270916
513
+ 503 69.38369781
514
+ 504 69.44444444
515
+ 505 69.50495050
516
+ 506 69.56521739
517
+ 507 69.42800789
518
+ 508 69.29133858
519
+ 509 69.35166994
520
+ 510 69.41176471
521
+ 511 69.47162427
522
+ 512 69.33593750
523
+ 513 69.20077973
524
+ 514 69.26070039
525
+ 515 69.32038835
526
+ 516 69.18604651
527
+ 517 69.05222437
528
+ 518 69.11196911
529
+ 519 68.97880539
530
+ 520 69.03846154
531
+ 521 68.90595010
532
+ 522 68.77394636
533
+ 523 68.83365201
534
+ 524 68.89312977
535
+ 525 68.95238095
536
+ 526 68.82129278
537
+ 527 68.69070209
538
+ 528 68.75000000
539
+ 529 68.62003781
540
+ 530 68.67924528
541
+ 531 68.73822976
542
+ 532 68.79699248
543
+ 533 68.66791745
544
+ 534 68.72659176
545
+ 535 68.78504673
546
+ 536 68.84328358
547
+ 537 68.90130354
548
+ 538 68.95910781
549
+ 539 68.83116883
550
+ 540 68.88888889
551
+ 541 68.94639556
552
+ 542 69.00369004
553
+ 543 69.06077348
554
+ 544 69.11764706
555
+ 545 69.17431193
556
+ 546 69.23076923
557
+ 547 69.28702011
558
+ 548 69.34306569
559
+ 549 69.39890710
560
+ 550 69.27272727
561
+ 551 69.32849365
562
+ 552 69.20289855
563
+ 553 69.25858951
564
+ 554 69.31407942
565
+ 555 69.18918919
566
+ 556 69.24460432
567
+ 557 69.12028725
568
+ 558 68.99641577
569
+ 559 68.87298748
570
+ 560 68.92857143
571
+ 561 68.98395722
572
+ 562 68.86120996
573
+ 563 68.91651865
574
+ 564 68.79432624
575
+ 565 68.67256637
576
+ 566 68.55123675
577
+ 567 68.43033510
578
+ 568 68.48591549
579
+ 569 68.36555360
580
+ 570 68.42105263
581
+ 571 68.47635727
582
+ 572 68.53146853
583
+ 573 68.58638743
584
+ 574 68.64111498
585
+ 575 68.69565217
586
+ 576 68.75000000
587
+ 577 68.63084922
588
+ 578 68.68512111
589
+ 579 68.73920553
590
+ 580 68.62068966
591
+ 581 68.67469880
592
+ 582 68.72852234
593
+ 583 68.78216123
594
+ 584 68.83561644
595
+ 585 68.88888889
596
+ 586 68.77133106
597
+ 587 68.82453152
598
+ 588 68.70748299
599
+ 589 68.59083192
600
+ 590 68.47457627
601
+ 591 68.52791878
602
+ 592 68.58108108
603
+ 593 68.46543002
604
+ 594 68.35016835
605
+ 595 68.40336134
606
+ 596 68.45637584
607
+ 597 68.50921273
608
+ 598 68.56187291
609
+ 599 68.44741235
610
+ 600 68.50000000
611
+ 601 68.55241265
612
+ 602 68.60465116
613
+ 603 68.49087894
614
+ 604 68.54304636
615
+ 605 68.59504132
616
+ 606 68.48184818
617
+ 607 68.53377265
618
+ 608 68.42105263
619
+ 609 68.47290640
620
+ 610 68.52459016
621
+ 611 68.57610475
622
+ 612 68.46405229
623
+ 613 68.51549755
624
+ 614 68.40390879
625
+ 615 68.45528455
626
+ 616 68.34415584
627
+ 617 68.39546191
628
+ 618 68.44660194
629
+ 619 68.33602585
630
+ 620 68.38709677
631
+ 621 68.27697262
632
+ 622 68.32797428
633
+ 623 68.37881220
634
+ 624 68.42948718
635
+ 625 68.48000000
636
+ 626 68.37060703
637
+ 627 68.42105263
638
+ 628 68.47133758
639
+ 629 68.36248013
640
+ 630 68.41269841
641
+ 631 68.46275753
642
+ 632 68.35443038
643
+ 633 68.40442338
644
+ 634 68.45425868
645
+ 635 68.50393701
646
+ 636 68.55345912
647
+ 637 68.60282575
648
+ 638 68.65203762
649
+ 639 68.70109546
650
+ 640 68.59375000
651
+ 641 68.64274571
652
+ 642 68.53582555
653
+ 643 68.58475894
654
+ 644 68.47826087
655
+ 645 68.37209302
656
+ 646 68.42105263
657
+ 647 68.46986090
658
+ 648 68.51851852
659
+ 649 68.41294299
660
+ 650 68.46153846
661
+ 651 68.50998464
662
+ 652 68.55828221
663
+ 653 68.60643185
664
+ 654 68.65443425
665
+ 655 68.70229008
666
+ 656 68.75000000
667
+ 657 68.79756469
668
+ 658 68.84498480
669
+ 659 68.74051593
670
+ 660 68.63636364
671
+ 661 68.68381241
672
+ 662 68.73111782
673
+ 663 68.77828054
674
+ 664 68.82530120
675
+ 665 68.87218045
676
+ 666 68.91891892
677
+ 667 68.96551724
678
+ 668 69.01197605
679
+ 669 69.05829596
680
+ 670 69.10447761
681
+ 671 69.15052161
682
+ 672 69.19642857
683
+ 673 69.24219911
684
+ 674 69.13946588
685
+ 675 69.18518519
686
+ 676 69.08284024
687
+ 677 68.98079764
688
+ 678 69.02654867
689
+ 679 69.07216495
690
+ 680 69.11764706
691
+ 681 69.01615272
692
+ 682 69.06158358
693
+ 683 69.10688141
694
+ 684 69.15204678
695
+ 685 69.05109489
696
+ 686 69.09620991
697
+ 687 69.14119360
698
+ 688 69.18604651
699
+ 689 69.23076923
700
+ 690 69.27536232
701
+ 691 69.31982634
702
+ 692 69.36416185
703
+ 693 69.40836941
704
+ 694 69.45244957
705
+ 695 69.49640288
706
+ 696 69.54022989
707
+ 697 69.44045911
708
+ 698 69.48424069
709
+ 699 69.38483548
710
+ 700 69.42857143
711
+ 701 69.47218260
712
+ 702 69.51566952
713
+ 703 69.55903272
714
+ 704 69.60227273
715
+ 705 69.64539007
716
+ 706 69.68838527
717
+ 707 69.73125884
718
+ 708 69.77401130
719
+ 709 69.81664316
720
+ 710 69.85915493
721
+ 711 69.76090014
722
+ 712 69.80337079
723
+ 713 69.84572230
724
+ 714 69.88795518
725
+ 715 69.93006993
726
+ 716 69.97206704
727
+ 717 70.01394700
728
+ 718 70.05571031
729
+ 719 69.95827538
730
+ 720 70.00000000
731
+ 721 69.90291262
732
+ 722 69.94459834
733
+ 723 69.98616874
734
+ 724 70.02762431
735
+ 725 70.06896552
736
+ 726 70.11019284
737
+ 727 70.15130674
738
+ 728 70.19230769
739
+ 729 70.23319616
740
+ 730 70.13698630
741
+ 731 70.17783858
742
+ 732 70.08196721
743
+ 733 70.12278308
744
+ 734 70.16348774
745
+ 735 70.20408163
746
+ 736 70.24456522
747
+ 737 70.14925373
748
+ 738 70.05420054
749
+ 739 69.95940460
750
+ 740 70.00000000
751
+ 741 70.04048583
752
+ 742 70.08086253
753
+ 743 70.12113055
754
+ 744 70.16129032
755
+ 745 70.06711409
756
+ 746 70.10723861
757
+ 747 70.01338688
758
+ 748 70.05347594
759
+ 749 70.09345794
760
+ 750 70.13333333
scores/Hammer2.1-7b-q3_k_m.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 0.00000000
13
+ 3 0.00000000
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 28.57142857
18
+ 8 37.50000000
19
+ 9 33.33333333
20
+ 10 30.00000000
21
+ 11 27.27272727
22
+ 12 33.33333333
23
+ 13 38.46153846
24
+ 14 35.71428571
25
+ 15 40.00000000
26
+ 16 43.75000000
27
+ 17 47.05882353
28
+ 18 44.44444444
29
+ 19 47.36842105
30
+ 20 45.00000000
31
+ 21 42.85714286
32
+ 22 40.90909091
33
+ 23 39.13043478
34
+ 24 41.66666667
35
+ 25 44.00000000
36
+ 26 42.30769231
37
+ 27 44.44444444
38
+ 28 42.85714286
39
+ 29 44.82758621
40
+ 30 46.66666667
41
+ 31 48.38709677
42
+ 32 50.00000000
43
+ 33 48.48484848
44
+ 34 50.00000000
45
+ 35 51.42857143
46
+ 36 50.00000000
47
+ 37 48.64864865
48
+ 38 47.36842105
49
+ 39 46.15384615
50
+ 40 45.00000000
51
+ 41 43.90243902
52
+ 42 42.85714286
53
+ 43 44.18604651
54
+ 44 43.18181818
55
+ 45 42.22222222
56
+ 46 41.30434783
57
+ 47 40.42553191
58
+ 48 41.66666667
59
+ 49 40.81632653
60
+ 50 40.00000000
61
+ 51 39.21568627
62
+ 52 38.46153846
63
+ 53 37.73584906
64
+ 54 37.03703704
65
+ 55 38.18181818
66
+ 56 37.50000000
67
+ 57 36.84210526
68
+ 58 37.93103448
69
+ 59 38.98305085
70
+ 60 38.33333333
71
+ 61 37.70491803
72
+ 62 37.09677419
73
+ 63 36.50793651
74
+ 64 35.93750000
75
+ 65 35.38461538
76
+ 66 36.36363636
77
+ 67 35.82089552
78
+ 68 35.29411765
79
+ 69 34.78260870
80
+ 70 35.71428571
81
+ 71 36.61971831
82
+ 72 36.11111111
83
+ 73 35.61643836
84
+ 74 36.48648649
85
+ 75 37.33333333
86
+ 76 38.15789474
87
+ 77 38.96103896
88
+ 78 39.74358974
89
+ 79 39.24050633
90
+ 80 40.00000000
91
+ 81 40.74074074
92
+ 82 40.24390244
93
+ 83 40.96385542
94
+ 84 41.66666667
95
+ 85 41.17647059
96
+ 86 40.69767442
97
+ 87 41.37931034
98
+ 88 42.04545455
99
+ 89 41.57303371
100
+ 90 41.11111111
101
+ 91 40.65934066
102
+ 92 40.21739130
103
+ 93 39.78494624
104
+ 94 40.42553191
105
+ 95 40.00000000
106
+ 96 40.62500000
107
+ 97 40.20618557
108
+ 98 39.79591837
109
+ 99 39.39393939
110
+ 100 39.00000000
111
+ 101 39.60396040
112
+ 102 39.21568627
113
+ 103 38.83495146
114
+ 104 39.42307692
115
+ 105 40.00000000
116
+ 106 40.56603774
117
+ 107 40.18691589
118
+ 108 39.81481481
119
+ 109 39.44954128
120
+ 110 39.09090909
121
+ 111 38.73873874
122
+ 112 39.28571429
123
+ 113 38.93805310
124
+ 114 39.47368421
125
+ 115 40.00000000
126
+ 116 39.65517241
127
+ 117 39.31623932
128
+ 118 38.98305085
129
+ 119 39.49579832
130
+ 120 40.00000000
131
+ 121 40.49586777
132
+ 122 40.16393443
133
+ 123 39.83739837
134
+ 124 39.51612903
135
+ 125 40.00000000
136
+ 126 39.68253968
137
+ 127 39.37007874
138
+ 128 39.06250000
139
+ 129 38.75968992
140
+ 130 38.46153846
141
+ 131 38.93129771
142
+ 132 38.63636364
143
+ 133 38.34586466
144
+ 134 38.05970149
145
+ 135 37.77777778
146
+ 136 37.50000000
147
+ 137 37.95620438
148
+ 138 37.68115942
149
+ 139 38.12949640
150
+ 140 38.57142857
151
+ 141 38.29787234
152
+ 142 38.73239437
153
+ 143 39.16083916
154
+ 144 38.88888889
155
+ 145 38.62068966
156
+ 146 39.04109589
157
+ 147 38.77551020
158
+ 148 39.18918919
159
+ 149 39.59731544
160
+ 150 39.33333333
161
+ 151 39.07284768
162
+ 152 39.47368421
163
+ 153 39.21568627
164
+ 154 38.96103896
165
+ 155 38.70967742
166
+ 156 38.46153846
167
+ 157 38.85350318
168
+ 158 39.24050633
169
+ 159 38.99371069
170
+ 160 39.37500000
171
+ 161 39.13043478
172
+ 162 38.88888889
173
+ 163 38.65030675
174
+ 164 38.41463415
175
+ 165 38.78787879
176
+ 166 38.55421687
177
+ 167 38.92215569
178
+ 168 38.69047619
179
+ 169 39.05325444
180
+ 170 38.82352941
181
+ 171 38.59649123
182
+ 172 38.37209302
183
+ 173 38.15028902
184
+ 174 38.50574713
185
+ 175 38.28571429
186
+ 176 38.06818182
187
+ 177 37.85310734
188
+ 178 38.20224719
189
+ 179 37.98882682
190
+ 180 37.77777778
191
+ 181 37.56906077
192
+ 182 37.36263736
193
+ 183 37.70491803
194
+ 184 37.50000000
195
+ 185 37.83783784
196
+ 186 38.17204301
197
+ 187 37.96791444
198
+ 188 37.76595745
199
+ 189 38.09523810
200
+ 190 37.89473684
201
+ 191 38.21989529
202
+ 192 38.02083333
203
+ 193 37.82383420
204
+ 194 37.62886598
205
+ 195 37.94871795
206
+ 196 37.75510204
207
+ 197 38.07106599
208
+ 198 37.87878788
209
+ 199 37.68844221
210
+ 200 37.50000000
211
+ 201 37.31343284
212
+ 202 37.12871287
213
+ 203 36.94581281
214
+ 204 36.76470588
215
+ 205 37.07317073
216
+ 206 36.89320388
217
+ 207 37.19806763
218
+ 208 37.01923077
219
+ 209 36.84210526
220
+ 210 36.66666667
221
+ 211 36.96682464
222
+ 212 36.79245283
223
+ 213 36.61971831
224
+ 214 36.44859813
225
+ 215 36.74418605
226
+ 216 37.03703704
227
+ 217 36.86635945
228
+ 218 37.15596330
229
+ 219 37.44292237
230
+ 220 37.72727273
231
+ 221 38.00904977
232
+ 222 38.28828829
233
+ 223 38.56502242
234
+ 224 38.83928571
235
+ 225 39.11111111
236
+ 226 39.38053097
237
+ 227 39.20704846
238
+ 228 39.47368421
239
+ 229 39.30131004
240
+ 230 39.13043478
241
+ 231 38.96103896
242
+ 232 39.22413793
243
+ 233 39.05579399
244
+ 234 39.31623932
245
+ 235 39.14893617
246
+ 236 38.98305085
247
+ 237 38.81856540
248
+ 238 38.65546218
249
+ 239 38.49372385
250
+ 240 38.33333333
251
+ 241 38.58921162
252
+ 242 38.84297521
253
+ 243 39.09465021
254
+ 244 38.93442623
255
+ 245 39.18367347
256
+ 246 39.02439024
257
+ 247 39.27125506
258
+ 248 39.51612903
259
+ 249 39.35742972
260
+ 250 39.20000000
261
+ 251 39.04382470
262
+ 252 38.88888889
263
+ 253 39.13043478
264
+ 254 38.97637795
265
+ 255 38.82352941
266
+ 256 39.06250000
267
+ 257 39.29961089
268
+ 258 39.14728682
269
+ 259 38.99613900
270
+ 260 38.84615385
271
+ 261 38.69731801
272
+ 262 38.54961832
273
+ 263 38.40304183
274
+ 264 38.63636364
275
+ 265 38.49056604
276
+ 266 38.72180451
277
+ 267 38.57677903
278
+ 268 38.43283582
279
+ 269 38.66171004
280
+ 270 38.88888889
281
+ 271 38.74538745
282
+ 272 38.60294118
283
+ 273 38.46153846
284
+ 274 38.32116788
285
+ 275 38.18181818
286
+ 276 38.40579710
287
+ 277 38.62815884
288
+ 278 38.84892086
289
+ 279 38.70967742
290
+ 280 38.92857143
291
+ 281 38.79003559
292
+ 282 38.65248227
293
+ 283 38.51590106
294
+ 284 38.38028169
295
+ 285 38.24561404
296
+ 286 38.11188811
297
+ 287 38.32752613
298
+ 288 38.19444444
299
+ 289 38.06228374
300
+ 290 38.27586207
301
+ 291 38.48797251
302
+ 292 38.69863014
303
+ 293 38.90784983
304
+ 294 38.77551020
305
+ 295 38.64406780
306
+ 296 38.51351351
307
+ 297 38.38383838
308
+ 298 38.59060403
309
+ 299 38.79598662
310
+ 300 39.00000000
311
+ 301 38.87043189
312
+ 302 38.74172185
313
+ 303 38.61386139
314
+ 304 38.48684211
315
+ 305 38.68852459
316
+ 306 38.56209150
317
+ 307 38.76221498
318
+ 308 38.63636364
319
+ 309 38.51132686
320
+ 310 38.70967742
321
+ 311 38.90675241
322
+ 312 39.10256410
323
+ 313 39.29712460
324
+ 314 39.49044586
325
+ 315 39.68253968
326
+ 316 39.55696203
327
+ 317 39.43217666
328
+ 318 39.30817610
329
+ 319 39.49843260
330
+ 320 39.37500000
331
+ 321 39.25233645
332
+ 322 39.13043478
333
+ 323 39.00928793
334
+ 324 38.88888889
335
+ 325 39.07692308
336
+ 326 38.95705521
337
+ 327 39.14373089
338
+ 328 39.32926829
339
+ 329 39.20972644
340
+ 330 39.09090909
341
+ 331 38.97280967
342
+ 332 38.85542169
343
+ 333 38.73873874
344
+ 334 38.62275449
345
+ 335 38.80597015
346
+ 336 38.98809524
347
+ 337 38.87240356
348
+ 338 38.75739645
349
+ 339 38.64306785
350
+ 340 38.82352941
351
+ 341 39.00293255
352
+ 342 39.18128655
353
+ 343 39.06705539
354
+ 344 38.95348837
355
+ 345 38.84057971
356
+ 346 39.01734104
357
+ 347 39.19308357
358
+ 348 39.08045977
359
+ 349 39.25501433
360
+ 350 39.42857143
361
+ 351 39.31623932
362
+ 352 39.20454545
363
+ 353 39.37677054
364
+ 354 39.26553672
365
+ 355 39.15492958
366
+ 356 39.32584270
367
+ 357 39.21568627
368
+ 358 39.10614525
369
+ 359 38.99721448
370
+ 360 39.16666667
371
+ 361 39.05817175
372
+ 362 39.22651934
373
+ 363 39.39393939
374
+ 364 39.28571429
375
+ 365 39.17808219
376
+ 366 39.34426230
377
+ 367 39.23705722
378
+ 368 39.13043478
379
+ 369 39.02439024
380
+ 370 39.18918919
381
+ 371 39.08355795
382
+ 372 38.97849462
383
+ 373 38.87399464
384
+ 374 39.03743316
385
+ 375 39.20000000
386
+ 376 39.09574468
387
+ 377 38.99204244
388
+ 378 39.15343915
389
+ 379 39.05013193
390
+ 380 38.94736842
391
+ 381 39.10761155
392
+ 382 39.26701571
393
+ 383 39.42558747
394
+ 384 39.32291667
395
+ 385 39.48051948
396
+ 386 39.37823834
397
+ 387 39.27648579
398
+ 388 39.17525773
399
+ 389 39.33161954
400
+ 390 39.48717949
401
+ 391 39.64194373
402
+ 392 39.79591837
403
+ 393 39.69465649
404
+ 394 39.59390863
405
+ 395 39.74683544
406
+ 396 39.89898990
407
+ 397 39.79848866
408
+ 398 39.69849246
409
+ 399 39.84962406
410
+ 400 39.75000000
411
+ 401 39.90024938
412
+ 402 39.80099502
413
+ 403 39.70223325
414
+ 404 39.60396040
415
+ 405 39.50617284
416
+ 406 39.40886700
417
+ 407 39.31203931
418
+ 408 39.21568627
419
+ 409 39.36430318
420
+ 410 39.26829268
421
+ 411 39.17274939
422
+ 412 39.07766990
423
+ 413 38.98305085
424
+ 414 38.88888889
425
+ 415 39.03614458
426
+ 416 38.94230769
427
+ 417 38.84892086
428
+ 418 38.75598086
429
+ 419 38.90214797
430
+ 420 39.04761905
431
+ 421 38.95486936
432
+ 422 38.86255924
433
+ 423 38.77068558
434
+ 424 38.91509434
435
+ 425 39.05882353
436
+ 426 39.20187793
437
+ 427 39.11007026
438
+ 428 39.01869159
439
+ 429 39.16083916
440
+ 430 39.06976744
441
+ 431 39.21113689
442
+ 432 39.35185185
443
+ 433 39.26096998
444
+ 434 39.17050691
445
+ 435 39.08045977
446
+ 436 38.99082569
447
+ 437 39.13043478
448
+ 438 39.26940639
449
+ 439 39.40774487
450
+ 440 39.31818182
451
+ 441 39.22902494
452
+ 442 39.14027149
453
+ 443 39.05191874
454
+ 444 38.96396396
455
+ 445 38.87640449
456
+ 446 38.78923767
457
+ 447 38.92617450
458
+ 448 38.83928571
459
+ 449 38.75278396
460
+ 450 38.66666667
461
+ 451 38.58093126
462
+ 452 38.71681416
463
+ 453 38.63134658
464
+ 454 38.54625551
465
+ 455 38.46153846
466
+ 456 38.59649123
467
+ 457 38.73085339
468
+ 458 38.64628821
469
+ 459 38.56209150
470
+ 460 38.69565217
471
+ 461 38.61171367
472
+ 462 38.52813853
473
+ 463 38.44492441
474
+ 464 38.57758621
475
+ 465 38.49462366
476
+ 466 38.41201717
477
+ 467 38.54389722
478
+ 468 38.46153846
479
+ 469 38.59275053
480
+ 470 38.51063830
481
+ 471 38.42887473
482
+ 472 38.34745763
483
+ 473 38.26638478
484
+ 474 38.18565401
485
+ 475 38.10526316
486
+ 476 38.23529412
487
+ 477 38.15513627
488
+ 478 38.28451883
489
+ 479 38.41336117
490
+ 480 38.54166667
491
+ 481 38.46153846
492
+ 482 38.38174274
493
+ 483 38.30227743
494
+ 484 38.22314050
495
+ 485 38.14432990
496
+ 486 38.06584362
497
+ 487 38.19301848
498
+ 488 38.11475410
499
+ 489 38.03680982
500
+ 490 38.16326531
501
+ 491 38.08553971
502
+ 492 38.00813008
503
+ 493 37.93103448
504
+ 494 38.05668016
505
+ 495 38.18181818
506
+ 496 38.30645161
507
+ 497 38.22937626
508
+ 498 38.15261044
509
+ 499 38.07615230
510
+ 500 38.00000000
511
+ 501 37.92415170
512
+ 502 37.84860558
513
+ 503 37.77335984
514
+ 504 37.89682540
515
+ 505 37.82178218
516
+ 506 37.94466403
517
+ 507 37.86982249
518
+ 508 37.99212598
519
+ 509 37.91748527
520
+ 510 38.03921569
521
+ 511 37.96477495
522
+ 512 37.89062500
523
+ 513 37.81676413
524
+ 514 37.74319066
525
+ 515 37.66990291
526
+ 516 37.79069767
527
+ 517 37.91102515
528
+ 518 38.03088803
529
+ 519 37.95761079
530
+ 520 38.07692308
531
+ 521 38.00383877
532
+ 522 37.93103448
533
+ 523 37.85850860
534
+ 524 37.97709924
535
+ 525 37.90476190
536
+ 526 37.83269962
537
+ 527 37.95066414
538
+ 528 38.06818182
539
+ 529 37.99621928
540
+ 530 38.11320755
541
+ 531 38.04143126
542
+ 532 37.96992481
543
+ 533 37.89868668
544
+ 534 37.82771536
545
+ 535 37.94392523
546
+ 536 37.87313433
547
+ 537 37.98882682
548
+ 538 38.10408922
549
+ 539 38.03339518
550
+ 540 37.96296296
551
+ 541 37.89279113
552
+ 542 38.00738007
553
+ 543 37.93738490
554
+ 544 37.86764706
555
+ 545 37.79816514
556
+ 546 37.72893773
557
+ 547 37.65996344
558
+ 548 37.77372263
559
+ 549 37.70491803
560
+ 550 37.63636364
561
+ 551 37.74954628
562
+ 552 37.68115942
563
+ 553 37.61301989
564
+ 554 37.54512635
565
+ 555 37.47747748
566
+ 556 37.58992806
567
+ 557 37.52244165
568
+ 558 37.63440860
569
+ 559 37.56708408
570
+ 560 37.50000000
571
+ 561 37.43315508
572
+ 562 37.54448399
573
+ 563 37.65541741
574
+ 564 37.58865248
575
+ 565 37.69911504
576
+ 566 37.80918728
577
+ 567 37.74250441
578
+ 568 37.67605634
579
+ 569 37.60984183
580
+ 570 37.71929825
581
+ 571 37.65323993
582
+ 572 37.76223776
583
+ 573 37.69633508
584
+ 574 37.80487805
585
+ 575 37.73913043
586
+ 576 37.84722222
587
+ 577 37.95493934
588
+ 578 37.88927336
589
+ 579 37.82383420
590
+ 580 37.93103448
591
+ 581 37.86574871
592
+ 582 37.80068729
593
+ 583 37.90737564
594
+ 584 38.01369863
595
+ 585 37.94871795
596
+ 586 38.05460751
597
+ 587 37.98977853
598
+ 588 37.92517007
599
+ 589 37.86078098
600
+ 590 37.79661017
601
+ 591 37.73265651
602
+ 592 37.66891892
603
+ 593 37.60539629
604
+ 594 37.71043771
605
+ 595 37.64705882
606
+ 596 37.75167785
607
+ 597 37.68844221
608
+ 598 37.62541806
609
+ 599 37.72954925
610
+ 600 37.66666667
611
+ 601 37.77038270
612
+ 602 37.70764120
613
+ 603 37.81094527
614
+ 604 37.74834437
615
+ 605 37.68595041
616
+ 606 37.62376238
617
+ 607 37.72652389
618
+ 608 37.66447368
619
+ 609 37.60262726
620
+ 610 37.54098361
621
+ 611 37.64320786
622
+ 612 37.58169935
623
+ 613 37.52039152
624
+ 614 37.45928339
625
+ 615 37.39837398
626
+ 616 37.50000000
627
+ 617 37.43922204
628
+ 618 37.37864078
629
+ 619 37.31825525
630
+ 620 37.25806452
631
+ 621 37.19806763
632
+ 622 37.13826367
633
+ 623 37.07865169
634
+ 624 37.01923077
635
+ 625 36.96000000
636
+ 626 37.06070288
637
+ 627 37.00159490
638
+ 628 36.94267516
639
+ 629 37.04292528
640
+ 630 36.98412698
641
+ 631 37.08399366
642
+ 632 37.18354430
643
+ 633 37.12480253
644
+ 634 37.06624606
645
+ 635 37.00787402
646
+ 636 36.94968553
647
+ 637 37.04866562
648
+ 638 36.99059561
649
+ 639 37.08920188
650
+ 640 37.18750000
651
+ 641 37.12948518
652
+ 642 37.22741433
653
+ 643 37.16951788
654
+ 644 37.26708075
655
+ 645 37.20930233
656
+ 646 37.15170279
657
+ 647 37.09428130
658
+ 648 37.03703704
659
+ 649 36.97996918
660
+ 650 36.92307692
661
+ 651 36.86635945
662
+ 652 36.80981595
663
+ 653 36.75344564
664
+ 654 36.85015291
665
+ 655 36.79389313
666
+ 656 36.73780488
667
+ 657 36.68188737
668
+ 658 36.77811550
669
+ 659 36.72230653
670
+ 660 36.66666667
671
+ 661 36.61119516
672
+ 662 36.55589124
673
+ 663 36.50075415
674
+ 664 36.59638554
675
+ 665 36.69172932
676
+ 666 36.63663664
677
+ 667 36.73163418
678
+ 668 36.82634731
679
+ 669 36.92077728
680
+ 670 36.86567164
681
+ 671 36.81073025
682
+ 672 36.75595238
683
+ 673 36.84992571
684
+ 674 36.79525223
685
+ 675 36.74074074
686
+ 676 36.68639053
687
+ 677 36.63220089
688
+ 678 36.57817109
689
+ 679 36.52430044
690
+ 680 36.61764706
691
+ 681 36.71071953
692
+ 682 36.65689150
693
+ 683 36.60322108
694
+ 684 36.54970760
695
+ 685 36.49635036
696
+ 686 36.44314869
697
+ 687 36.39010189
698
+ 688 36.33720930
699
+ 689 36.42960813
700
+ 690 36.37681159
701
+ 691 36.46888567
702
+ 692 36.41618497
703
+ 693 36.36363636
704
+ 694 36.31123919
705
+ 695 36.25899281
706
+ 696 36.35057471
707
+ 697 36.44189383
708
+ 698 36.38968481
709
+ 699 36.48068670
710
+ 700 36.42857143
711
+ 701 36.37660485
712
+ 702 36.46723647
713
+ 703 36.55761024
714
+ 704 36.50568182
715
+ 705 36.45390071
716
+ 706 36.40226629
717
+ 707 36.35077793
718
+ 708 36.29943503
719
+ 709 36.38928068
720
+ 710 36.47887324
721
+ 711 36.56821378
722
+ 712 36.51685393
723
+ 713 36.60589060
724
+ 714 36.55462185
725
+ 715 36.50349650
726
+ 716 36.59217877
727
+ 717 36.54114365
728
+ 718 36.49025070
729
+ 719 36.57858136
730
+ 720 36.52777778
731
+ 721 36.47711512
732
+ 722 36.56509695
733
+ 723 36.65283541
734
+ 724 36.60220994
735
+ 725 36.68965517
736
+ 726 36.63911846
737
+ 727 36.58872077
738
+ 728 36.53846154
739
+ 729 36.48834019
740
+ 730 36.43835616
741
+ 731 36.38850889
742
+ 732 36.47540984
743
+ 733 36.56207367
744
+ 734 36.64850136
745
+ 735 36.59863946
746
+ 736 36.54891304
747
+ 737 36.63500678
748
+ 738 36.58536585
749
+ 739 36.67117727
750
+ 740 36.62162162
751
+ 741 36.57219973
752
+ 742 36.65768194
753
+ 743 36.74293405
754
+ 744 36.82795699
755
+ 745 36.77852349
756
+ 746 36.72922252
757
+ 747 36.68005355
758
+ 748 36.63101604
759
+ 749 36.58210948
760
+ 750 36.66666667
761
+
762
+ Final result: 36.6667 +/- 1.7608
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-q3_k_m.ppx ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ====== Perplexity statistics ======
2
+ Mean PPL(Q) : 10.562569 ± 0.076862
3
+ Mean PPL(base) : 9.366577 ± 0.066397
4
+ Cor(ln(PPL(Q)), ln(PPL(base))): 97.91%
5
+ Mean ln(PPL(Q)/PPL(base)) : 0.120169 ± 0.001479
6
+ Mean PPL(Q)/PPL(base) : 1.127687 ± 0.001668
7
+ Mean PPL(Q)-PPL(base) : 1.195992 ± 0.017958
8
+
9
+ ====== KL divergence statistics ======
10
+ Mean KLD: 0.129660 ± 0.000389
11
+ Maximum KLD: 5.317839
12
+ 99.9% KLD: 1.371907
13
+ 99.0% KLD: 0.679472
14
+ 99.0% KLD: 0.679472
15
+ Median KLD: 0.094181
16
+ 10.0% KLD: 0.001932
17
+ 5.0% KLD: 0.000411
18
+ 1.0% KLD: 0.000032
19
+ Minimum KLD: -0.000008
20
+
21
+ ====== Token probability statistics ======
22
+ Mean Δp: -1.763 ± 0.024 %
23
+ Maximum Δp: 85.997%
24
+ 99.9% Δp: 39.654%
25
+ 99.0% Δp: 22.976%
26
+ 95.0% Δp: 10.740%
27
+ 90.0% Δp: 5.578%
28
+ 75.0% Δp: 0.515%
29
+ Median Δp: -0.109%
30
+ 25.0% Δp: -3.393%
31
+ 10.0% Δp: -11.616%
32
+ 5.0% Δp: -18.101%
33
+ 1.0% Δp: -33.871%
34
+ 0.1% Δp: -59.099%
35
+ Minimum Δp: -94.810%
36
+ RMS Δp : 9.338 ± 0.040 %
37
+ Same top p: 80.895 ± 0.102 %
scores/Hammer2.1-7b-q3_k_m.tqa ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 817 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 817 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 14.28571429
18
+ 8 12.50000000
19
+ 9 11.11111111
20
+ 10 10.00000000
21
+ 11 9.09090909
22
+ 12 16.66666667
23
+ 13 23.07692308
24
+ 14 21.42857143
25
+ 15 20.00000000
26
+ 16 18.75000000
27
+ 17 17.64705882
28
+ 18 16.66666667
29
+ 19 21.05263158
30
+ 20 20.00000000
31
+ 21 19.04761905
32
+ 22 18.18181818
33
+ 23 17.39130435
34
+ 24 20.83333333
35
+ 25 20.00000000
36
+ 26 19.23076923
37
+ 27 18.51851852
38
+ 28 17.85714286
39
+ 29 20.68965517
40
+ 30 20.00000000
41
+ 31 19.35483871
42
+ 32 21.87500000
43
+ 33 24.24242424
44
+ 34 23.52941176
45
+ 35 22.85714286
46
+ 36 22.22222222
47
+ 37 24.32432432
48
+ 38 23.68421053
49
+ 39 23.07692308
50
+ 40 22.50000000
51
+ 41 21.95121951
52
+ 42 21.42857143
53
+ 43 23.25581395
54
+ 44 22.72727273
55
+ 45 22.22222222
56
+ 46 23.91304348
57
+ 47 25.53191489
58
+ 48 27.08333333
59
+ 49 26.53061224
60
+ 50 28.00000000
61
+ 51 29.41176471
62
+ 52 28.84615385
63
+ 53 28.30188679
64
+ 54 29.62962963
65
+ 55 30.90909091
66
+ 56 30.35714286
67
+ 57 29.82456140
68
+ 58 29.31034483
69
+ 59 28.81355932
70
+ 60 28.33333333
71
+ 61 27.86885246
72
+ 62 29.03225806
73
+ 63 28.57142857
74
+ 64 29.68750000
75
+ 65 29.23076923
76
+ 66 28.78787879
77
+ 67 28.35820896
78
+ 68 27.94117647
79
+ 69 28.98550725
80
+ 70 30.00000000
81
+ 71 29.57746479
82
+ 72 30.55555556
83
+ 73 30.13698630
84
+ 74 29.72972973
85
+ 75 29.33333333
86
+ 76 28.94736842
87
+ 77 28.57142857
88
+ 78 28.20512821
89
+ 79 27.84810127
90
+ 80 28.75000000
91
+ 81 28.39506173
92
+ 82 29.26829268
93
+ 83 28.91566265
94
+ 84 28.57142857
95
+ 85 28.23529412
96
+ 86 27.90697674
97
+ 87 27.58620690
98
+ 88 27.27272727
99
+ 89 28.08988764
100
+ 90 27.77777778
101
+ 91 28.57142857
102
+ 92 28.26086957
103
+ 93 27.95698925
104
+ 94 27.65957447
105
+ 95 28.42105263
106
+ 96 29.16666667
107
+ 97 29.89690722
108
+ 98 29.59183673
109
+ 99 30.30303030
110
+ 100 31.00000000
111
+ 101 31.68316832
112
+ 102 31.37254902
113
+ 103 32.03883495
114
+ 104 31.73076923
115
+ 105 31.42857143
116
+ 106 31.13207547
117
+ 107 31.77570093
118
+ 108 32.40740741
119
+ 109 32.11009174
120
+ 110 32.72727273
121
+ 111 32.43243243
122
+ 112 32.14285714
123
+ 113 31.85840708
124
+ 114 32.45614035
125
+ 115 32.17391304
126
+ 116 31.89655172
127
+ 117 31.62393162
128
+ 118 31.35593220
129
+ 119 31.09243697
130
+ 120 31.66666667
131
+ 121 32.23140496
132
+ 122 31.96721311
133
+ 123 31.70731707
134
+ 124 32.25806452
135
+ 125 32.80000000
136
+ 126 32.53968254
137
+ 127 32.28346457
138
+ 128 32.03125000
139
+ 129 31.78294574
140
+ 130 31.53846154
141
+ 131 31.29770992
142
+ 132 31.06060606
143
+ 133 30.82706767
144
+ 134 31.34328358
145
+ 135 31.85185185
146
+ 136 31.61764706
147
+ 137 32.11678832
148
+ 138 32.60869565
149
+ 139 33.09352518
150
+ 140 32.85714286
151
+ 141 32.62411348
152
+ 142 32.39436620
153
+ 143 32.16783217
154
+ 144 32.63888889
155
+ 145 32.41379310
156
+ 146 32.19178082
157
+ 147 31.97278912
158
+ 148 31.75675676
159
+ 149 31.54362416
160
+ 150 31.33333333
161
+ 151 31.78807947
162
+ 152 31.57894737
163
+ 153 31.37254902
164
+ 154 31.16883117
165
+ 155 31.61290323
166
+ 156 32.05128205
167
+ 157 31.84713376
168
+ 158 32.27848101
169
+ 159 32.07547170
170
+ 160 31.87500000
171
+ 161 31.67701863
172
+ 162 31.48148148
173
+ 163 31.28834356
174
+ 164 31.09756098
175
+ 165 31.51515152
176
+ 166 31.92771084
177
+ 167 31.73652695
178
+ 168 31.54761905
179
+ 169 31.36094675
180
+ 170 31.17647059
181
+ 171 30.99415205
182
+ 172 30.81395349
183
+ 173 30.63583815
184
+ 174 31.03448276
185
+ 175 30.85714286
186
+ 176 30.68181818
187
+ 177 30.50847458
188
+ 178 30.33707865
189
+ 179 30.72625698
190
+ 180 30.55555556
191
+ 181 30.38674033
192
+ 182 30.21978022
193
+ 183 30.05464481
194
+ 184 29.89130435
195
+ 185 29.72972973
196
+ 186 29.56989247
197
+ 187 29.41176471
198
+ 188 29.78723404
199
+ 189 30.15873016
200
+ 190 30.00000000
201
+ 191 29.84293194
202
+ 192 30.20833333
203
+ 193 30.05181347
204
+ 194 30.41237113
205
+ 195 30.25641026
206
+ 196 30.10204082
207
+ 197 30.45685279
208
+ 198 30.80808081
209
+ 199 30.65326633
210
+ 200 30.50000000
211
+ 201 30.34825871
212
+ 202 30.69306931
213
+ 203 30.54187192
214
+ 204 30.39215686
215
+ 205 30.73170732
216
+ 206 30.58252427
217
+ 207 30.91787440
218
+ 208 30.76923077
219
+ 209 30.62200957
220
+ 210 30.47619048
221
+ 211 30.33175355
222
+ 212 30.18867925
223
+ 213 30.04694836
224
+ 214 29.90654206
225
+ 215 29.76744186
226
+ 216 30.09259259
227
+ 217 29.95391705
228
+ 218 30.27522936
229
+ 219 30.13698630
230
+ 220 30.00000000
231
+ 221 29.86425339
232
+ 222 29.72972973
233
+ 223 29.59641256
234
+ 224 29.91071429
235
+ 225 29.77777778
236
+ 226 29.64601770
237
+ 227 29.95594714
238
+ 228 29.82456140
239
+ 229 29.69432314
240
+ 230 29.56521739
241
+ 231 29.43722944
242
+ 232 29.74137931
243
+ 233 30.04291845
244
+ 234 29.91452991
245
+ 235 29.78723404
246
+ 236 30.08474576
247
+ 237 30.37974684
248
+ 238 30.25210084
249
+ 239 30.12552301
250
+ 240 30.00000000
251
+ 241 29.87551867
252
+ 242 29.75206612
253
+ 243 29.62962963
254
+ 244 29.50819672
255
+ 245 29.38775510
256
+ 246 29.26829268
257
+ 247 29.14979757
258
+ 248 29.03225806
259
+ 249 28.91566265
260
+ 250 28.80000000
261
+ 251 29.08366534
262
+ 252 28.96825397
263
+ 253 28.85375494
264
+ 254 28.74015748
265
+ 255 29.01960784
266
+ 256 28.90625000
267
+ 257 28.79377432
268
+ 258 28.68217054
269
+ 259 28.57142857
270
+ 260 28.46153846
271
+ 261 28.73563218
272
+ 262 28.62595420
273
+ 263 28.51711027
274
+ 264 28.78787879
275
+ 265 29.05660377
276
+ 266 28.94736842
277
+ 267 29.21348315
278
+ 268 29.10447761
279
+ 269 28.99628253
280
+ 270 28.88888889
281
+ 271 28.78228782
282
+ 272 28.67647059
283
+ 273 28.57142857
284
+ 274 28.83211679
285
+ 275 28.72727273
286
+ 276 28.62318841
287
+ 277 28.51985560
288
+ 278 28.41726619
289
+ 279 28.31541219
290
+ 280 28.57142857
291
+ 281 28.82562278
292
+ 282 28.72340426
293
+ 283 28.62190813
294
+ 284 28.87323944
295
+ 285 28.77192982
296
+ 286 29.02097902
297
+ 287 29.26829268
298
+ 288 29.16666667
299
+ 289 29.06574394
300
+ 290 28.96551724
301
+ 291 29.20962199
302
+ 292 29.10958904
303
+ 293 29.35153584
304
+ 294 29.25170068
305
+ 295 29.49152542
306
+ 296 29.72972973
307
+ 297 29.96632997
308
+ 298 29.86577181
309
+ 299 29.76588629
310
+ 300 29.66666667
311
+ 301 29.56810631
312
+ 302 29.47019868
313
+ 303 29.37293729
314
+ 304 29.27631579
315
+ 305 29.18032787
316
+ 306 29.08496732
317
+ 307 28.99022801
318
+ 308 29.22077922
319
+ 309 29.12621359
320
+ 310 29.03225806
321
+ 311 28.93890675
322
+ 312 29.16666667
323
+ 313 29.07348243
324
+ 314 29.29936306
325
+ 315 29.20634921
326
+ 316 29.11392405
327
+ 317 29.02208202
328
+ 318 28.93081761
329
+ 319 28.84012539
330
+ 320 28.75000000
331
+ 321 28.66043614
332
+ 322 28.88198758
333
+ 323 28.79256966
334
+ 324 28.70370370
335
+ 325 28.61538462
336
+ 326 28.52760736
337
+ 327 28.74617737
338
+ 328 28.96341463
339
+ 329 28.87537994
340
+ 330 29.09090909
341
+ 331 29.00302115
342
+ 332 29.21686747
343
+ 333 29.12912913
344
+ 334 29.04191617
345
+ 335 29.25373134
346
+ 336 29.46428571
347
+ 337 29.37685460
348
+ 338 29.28994083
349
+ 339 29.20353982
350
+ 340 29.41176471
351
+ 341 29.61876833
352
+ 342 29.53216374
353
+ 343 29.44606414
354
+ 344 29.65116279
355
+ 345 29.85507246
356
+ 346 30.05780347
357
+ 347 30.25936599
358
+ 348 30.17241379
359
+ 349 30.08595989
360
+ 350 30.00000000
361
+ 351 29.91452991
362
+ 352 29.82954545
363
+ 353 29.74504249
364
+ 354 29.66101695
365
+ 355 29.85915493
366
+ 356 29.77528090
367
+ 357 29.69187675
368
+ 358 29.60893855
369
+ 359 29.52646240
370
+ 360 29.44444444
371
+ 361 29.63988920
372
+ 362 29.55801105
373
+ 363 29.75206612
374
+ 364 29.67032967
375
+ 365 29.86301370
376
+ 366 29.78142077
377
+ 367 29.97275204
378
+ 368 29.89130435
379
+ 369 29.81029810
380
+ 370 29.72972973
381
+ 371 29.64959569
382
+ 372 29.83870968
383
+ 373 29.75871314
384
+ 374 29.94652406
385
+ 375 29.86666667
386
+ 376 29.78723404
387
+ 377 29.70822281
388
+ 378 29.62962963
389
+ 379 29.55145119
390
+ 380 29.47368421
391
+ 381 29.39632546
392
+ 382 29.58115183
393
+ 383 29.50391645
394
+ 384 29.42708333
395
+ 385 29.35064935
396
+ 386 29.27461140
397
+ 387 29.19896641
398
+ 388 29.38144330
399
+ 389 29.30591260
400
+ 390 29.23076923
401
+ 391 29.15601023
402
+ 392 29.08163265
403
+ 393 29.00763359
404
+ 394 29.18781726
405
+ 395 29.11392405
406
+ 396 29.04040404
407
+ 397 29.21914358
408
+ 398 29.14572864
409
+ 399 29.07268170
410
+ 400 29.00000000
411
+ 401 28.92768080
412
+ 402 29.10447761
413
+ 403 29.28039702
414
+ 404 29.20792079
415
+ 405 29.13580247
416
+ 406 29.06403941
417
+ 407 28.99262899
418
+ 408 28.92156863
419
+ 409 28.85085575
420
+ 410 28.78048780
421
+ 411 28.71046229
422
+ 412 28.88349515
423
+ 413 28.81355932
424
+ 414 28.98550725
425
+ 415 29.15662651
426
+ 416 29.08653846
427
+ 417 29.01678657
428
+ 418 28.94736842
429
+ 419 28.87828162
430
+ 420 29.04761905
431
+ 421 29.21615202
432
+ 422 29.14691943
433
+ 423 29.07801418
434
+ 424 29.00943396
435
+ 425 28.94117647
436
+ 426 28.87323944
437
+ 427 28.80562061
438
+ 428 28.73831776
439
+ 429 28.90442890
440
+ 430 28.83720930
441
+ 431 28.77030162
442
+ 432 28.93518519
443
+ 433 29.09930716
444
+ 434 29.26267281
445
+ 435 29.19540230
446
+ 436 29.12844037
447
+ 437 29.06178490
448
+ 438 28.99543379
449
+ 439 28.92938497
450
+ 440 28.86363636
451
+ 441 28.79818594
452
+ 442 28.73303167
453
+ 443 28.66817156
454
+ 444 28.82882883
455
+ 445 28.76404494
456
+ 446 28.69955157
457
+ 447 28.63534676
458
+ 448 28.57142857
459
+ 449 28.50779510
460
+ 450 28.66666667
461
+ 451 28.60310421
462
+ 452 28.53982301
463
+ 453 28.69757174
464
+ 454 28.85462555
465
+ 455 28.79120879
466
+ 456 28.72807018
467
+ 457 28.66520788
468
+ 458 28.60262009
469
+ 459 28.54030501
470
+ 460 28.47826087
471
+ 461 28.41648590
472
+ 462 28.35497835
473
+ 463 28.50971922
474
+ 464 28.66379310
475
+ 465 28.81720430
476
+ 466 28.75536481
477
+ 467 28.69379015
478
+ 468 28.63247863
479
+ 469 28.57142857
480
+ 470 28.51063830
481
+ 471 28.66242038
482
+ 472 28.60169492
483
+ 473 28.54122622
484
+ 474 28.48101266
485
+ 475 28.63157895
486
+ 476 28.57142857
487
+ 477 28.72117400
488
+ 478 28.66108787
489
+ 479 28.60125261
490
+ 480 28.54166667
491
+ 481 28.48232848
492
+ 482 28.63070539
493
+ 483 28.57142857
494
+ 484 28.71900826
495
+ 485 28.86597938
496
+ 486 28.80658436
497
+ 487 28.74743326
498
+ 488 28.68852459
499
+ 489 28.83435583
500
+ 490 28.97959184
501
+ 491 29.12423625
502
+ 492 29.26829268
503
+ 493 29.20892495
504
+ 494 29.14979757
505
+ 495 29.29292929
506
+ 496 29.23387097
507
+ 497 29.17505030
508
+ 498 29.11646586
509
+ 499 29.25851703
510
+ 500 29.20000000
511
+ 501 29.34131737
512
+ 502 29.28286853
513
+ 503 29.22465209
514
+ 504 29.36507937
515
+ 505 29.30693069
516
+ 506 29.24901186
517
+ 507 29.38856016
518
+ 508 29.33070866
519
+ 509 29.46954813
520
+ 510 29.60784314
521
+ 511 29.54990215
522
+ 512 29.49218750
523
+ 513 29.43469786
524
+ 514 29.37743191
525
+ 515 29.51456311
526
+ 516 29.45736434
527
+ 517 29.59381044
528
+ 518 29.53667954
529
+ 519 29.47976879
530
+ 520 29.42307692
531
+ 521 29.55854127
532
+ 522 29.50191571
533
+ 523 29.44550669
534
+ 524 29.38931298
535
+ 525 29.33333333
536
+ 526 29.27756654
537
+ 527 29.22201139
538
+ 528 29.16666667
539
+ 529 29.11153119
540
+ 530 29.05660377
541
+ 531 29.00188324
542
+ 532 29.13533835
543
+ 533 29.26829268
544
+ 534 29.21348315
545
+ 535 29.15887850
546
+ 536 29.10447761
547
+ 537 29.23649907
548
+ 538 29.18215613
549
+ 539 29.12801484
550
+ 540 29.07407407
551
+ 541 29.02033272
552
+ 542 28.96678967
553
+ 543 28.91344383
554
+ 544 28.86029412
555
+ 545 28.80733945
556
+ 546 28.93772894
557
+ 547 29.06764168
558
+ 548 29.19708029
559
+ 549 29.14389800
560
+ 550 29.09090909
561
+ 551 29.03811252
562
+ 552 29.16666667
563
+ 553 29.29475588
564
+ 554 29.24187726
565
+ 555 29.18918919
566
+ 556 29.13669065
567
+ 557 29.08438061
568
+ 558 29.03225806
569
+ 559 28.98032200
570
+ 560 29.10714286
571
+ 561 29.23351159
572
+ 562 29.18149466
573
+ 563 29.12966252
574
+ 564 29.25531915
575
+ 565 29.38053097
576
+ 566 29.50530035
577
+ 567 29.45326279
578
+ 568 29.40140845
579
+ 569 29.52548330
580
+ 570 29.47368421
581
+ 571 29.42206655
582
+ 572 29.37062937
583
+ 573 29.31937173
584
+ 574 29.44250871
585
+ 575 29.56521739
586
+ 576 29.51388889
587
+ 577 29.46273830
588
+ 578 29.58477509
589
+ 579 29.70639033
590
+ 580 29.82758621
591
+ 581 29.77624785
592
+ 582 29.89690722
593
+ 583 29.84562607
594
+ 584 29.96575342
595
+ 585 29.91452991
596
+ 586 30.03412969
597
+ 587 29.98296422
598
+ 588 29.93197279
599
+ 589 29.88115450
600
+ 590 30.00000000
601
+ 591 30.11844332
602
+ 592 30.06756757
603
+ 593 30.01686341
604
+ 594 30.13468013
605
+ 595 30.08403361
606
+ 596 30.03355705
607
+ 597 30.15075377
608
+ 598 30.10033445
609
+ 599 30.05008347
610
+ 600 30.00000000
611
+ 601 30.11647255
612
+ 602 30.06644518
613
+ 603 30.18242123
614
+ 604 30.13245033
615
+ 605 30.08264463
616
+ 606 30.19801980
617
+ 607 30.14827018
618
+ 608 30.09868421
619
+ 609 30.21346470
620
+ 610 30.16393443
621
+ 611 30.11456628
622
+ 612 30.22875817
623
+ 613 30.17944535
624
+ 614 30.13029316
625
+ 615 30.08130081
626
+ 616 30.19480519
627
+ 617 30.30794165
628
+ 618 30.25889968
629
+ 619 30.37156704
630
+ 620 30.32258065
631
+ 621 30.27375201
632
+ 622 30.38585209
633
+ 623 30.33707865
634
+ 624 30.28846154
635
+ 625 30.24000000
636
+ 626 30.19169329
637
+ 627 30.30303030
638
+ 628 30.25477707
639
+ 629 30.20667727
640
+ 630 30.15873016
641
+ 631 30.26941363
642
+ 632 30.22151899
643
+ 633 30.17377567
644
+ 634 30.28391167
645
+ 635 30.23622047
646
+ 636 30.18867925
647
+ 637 30.14128728
648
+ 638 30.09404389
649
+ 639 30.04694836
650
+ 640 30.00000000
651
+ 641 29.95319813
652
+ 642 30.06230530
653
+ 643 30.01555210
654
+ 644 29.96894410
655
+ 645 30.07751938
656
+ 646 30.03095975
657
+ 647 29.98454405
658
+ 648 30.09259259
659
+ 649 30.04622496
660
+ 650 30.15384615
661
+ 651 30.26113671
662
+ 652 30.36809816
663
+ 653 30.47473201
664
+ 654 30.42813456
665
+ 655 30.38167939
666
+ 656 30.33536585
667
+ 657 30.44140030
668
+ 658 30.39513678
669
+ 659 30.34901366
670
+ 660 30.30303030
671
+ 661 30.25718608
672
+ 662 30.21148036
673
+ 663 30.16591252
674
+ 664 30.12048193
675
+ 665 30.07518797
676
+ 666 30.18018018
677
+ 667 30.13493253
678
+ 668 30.08982036
679
+ 669 30.04484305
680
+ 670 30.14925373
681
+ 671 30.10432191
682
+ 672 30.05952381
683
+ 673 30.01485884
684
+ 674 29.97032641
685
+ 675 30.07407407
686
+ 676 30.17751479
687
+ 677 30.13293944
688
+ 678 30.08849558
689
+ 679 30.04418262
690
+ 680 30.00000000
691
+ 681 30.10279001
692
+ 682 30.05865103
693
+ 683 30.01464129
694
+ 684 30.11695906
695
+ 685 30.07299270
696
+ 686 30.02915452
697
+ 687 29.98544396
698
+ 688 29.94186047
699
+ 689 30.04354136
700
+ 690 30.00000000
701
+ 691 30.10130246
702
+ 692 30.20231214
703
+ 693 30.30303030
704
+ 694 30.25936599
705
+ 695 30.21582734
706
+ 696 30.31609195
707
+ 697 30.27259684
708
+ 698 30.22922636
709
+ 699 30.18597997
710
+ 700 30.28571429
711
+ 701 30.24251070
712
+ 702 30.19943020
713
+ 703 30.15647226
714
+ 704 30.11363636
715
+ 705 30.07092199
716
+ 706 30.02832861
717
+ 707 30.12729844
718
+ 708 30.08474576
719
+ 709 30.04231312
720
+ 710 30.00000000
721
+ 711 29.95780591
722
+ 712 29.91573034
723
+ 713 29.87377279
724
+ 714 29.97198880
725
+ 715 30.06993007
726
+ 716 30.02793296
727
+ 717 29.98605300
728
+ 718 29.94428969
729
+ 719 30.04172462
730
+ 720 30.00000000
731
+ 721 29.95839112
732
+ 722 29.91689751
733
+ 723 29.87551867
734
+ 724 29.97237569
735
+ 725 29.93103448
736
+ 726 30.02754821
737
+ 727 29.98624484
738
+ 728 29.94505495
739
+ 729 30.04115226
740
+ 730 30.00000000
741
+ 731 29.95896033
742
+ 732 30.05464481
743
+ 733 30.01364256
744
+ 734 29.97275204
745
+ 735 29.93197279
746
+ 736 29.89130435
747
+ 737 29.98643148
748
+ 738 29.94579946
749
+ 739 30.04059540
750
+ 740 30.00000000
751
+ 741 30.09446694
752
+ 742 30.05390836
753
+ 743 30.01345895
754
+ 744 30.10752688
755
+ 745 30.06711409
756
+ 746 30.16085791
757
+ 747 30.25435074
758
+ 748 30.21390374
759
+ 749 30.30707610
760
+ 750 30.40000000
761
+
762
+ Final result: 30.4000 +/- 1.6807
763
+ Random chance: 19.8992 +/- 1.4588
scores/Hammer2.1-7b-q3_k_m.wng ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ winogrande_score : loaded 1267 tasks from prompt.
6
+ winogrande_score : selecting 750 random tasks
7
+ winogrande_score : tokenizing selected tasks
8
+ winogrande_score : calculating winogrande score over selected tasks.
9
+ 1 100.0000 -4.280200 -4.938750 1 1
10
+ 2 100.0000 -2.223864 -2.713899 1 1
11
+ 3 100.0000 -2.836886 -2.335751 2 2
12
+ 4 75.0000 -4.842021 -4.381081 2 1
13
+ 5 60.0000 -2.328923 -3.314353 1 2
14
+ 6 50.0000 -4.876793 -4.864092 2 1
15
+ 7 57.1429 -4.198294 -3.049478 2 2
16
+ 8 50.0000 -1.868659 -1.944426 1 2
17
+ 9 55.5556 -3.429965 -5.653793 1 1
18
+ 10 60.0000 -1.775458 -1.540926 2 2
19
+ 11 63.6364 -11.215016 -5.319075 2 2
20
+ 12 66.6667 -4.308177 -3.782213 2 2
21
+ 13 69.2308 -3.120992 -6.520279 1 1
22
+ 14 64.2857 -3.895017 -3.803847 2 1
23
+ 15 60.0000 -3.890220 -5.722140 1 2
24
+ 16 62.5000 -4.484763 -4.039827 2 2
25
+ 17 58.8235 -6.241928 -6.817817 1 2
26
+ 18 61.1111 -1.467996 -2.847542 1 1
27
+ 19 63.1579 -4.463470 -5.340464 1 1
28
+ 20 65.0000 -3.224109 -4.257095 1 1
29
+ 21 66.6667 -2.823231 -3.344740 1 1
30
+ 22 68.1818 -2.954458 -2.302758 2 2
31
+ 23 69.5652 -5.229430 -5.294956 1 1
32
+ 24 66.6667 -2.815536 -2.509757 2 1
33
+ 25 68.0000 -2.643208 -2.428190 2 2
34
+ 26 69.2308 -3.273677 -4.186934 1 1
35
+ 27 66.6667 -5.346833 -5.351336 1 2
36
+ 28 67.8571 -4.391919 -5.136487 1 1
37
+ 29 68.9655 -4.037320 -4.287533 1 1
38
+ 30 66.6667 -1.460111 -1.773133 1 2
39
+ 31 67.7419 -5.760154 -5.795126 1 1
40
+ 32 68.7500 -4.355121 -5.646965 1 1
41
+ 33 69.6970 -3.690576 -6.256041 1 1
42
+ 34 67.6471 -2.394412 -2.349900 2 1
43
+ 35 68.5714 -4.785401 -4.709689 2 2
44
+ 36 69.4444 -5.095976 -4.213293 2 2
45
+ 37 67.5676 -3.476686 -3.253574 2 1
46
+ 38 68.4211 -3.430626 -3.625711 1 1
47
+ 39 69.2308 -4.210289 -4.021990 2 2
48
+ 40 67.5000 -4.463763 -5.668484 1 2
49
+ 41 68.2927 -2.534106 -2.921771 1 1
50
+ 42 69.0476 -1.844051 -2.593866 1 1
51
+ 43 69.7674 -4.160378 -4.105949 2 2
52
+ 44 68.1818 -2.069877 -2.101482 1 2
53
+ 45 68.8889 -0.076923 -14.314107 1 1
54
+ 46 69.5652 -2.582086 -2.812560 1 1
55
+ 47 68.0851 -4.182916 -4.763305 1 2
56
+ 48 68.7500 -5.064484 -4.903445 2 2
57
+ 49 67.3469 -2.907546 -0.853312 2 1
58
+ 50 66.0000 -2.547427 -2.573295 1 2
59
+ 51 66.6667 -2.447172 -2.856760 1 1
60
+ 52 67.3077 -3.258227 -2.422616 2 2
61
+ 53 67.9245 -3.799854 -4.850302 1 1
62
+ 54 68.5185 -4.118059 -3.665833 2 2
63
+ 55 69.0909 -3.347865 -3.128961 2 2
64
+ 56 67.8571 -4.943979 -5.354531 1 2
65
+ 57 66.6667 -2.035163 -3.341469 1 2
66
+ 58 67.2414 -3.286887 -4.268116 1 1
67
+ 59 67.7966 -3.063550 -2.826411 2 2
68
+ 60 68.3333 -4.831946 -4.432106 2 2
69
+ 61 68.8525 -4.099392 -4.639569 1 1
70
+ 62 67.7419 -7.460231 -7.373551 2 1
71
+ 63 66.6667 -3.812858 -4.754472 1 2
72
+ 64 67.1875 -4.257965 -4.486609 1 1
73
+ 65 67.6923 -4.677990 -4.399535 2 2
74
+ 66 68.1818 -2.459864 -3.733743 1 1
75
+ 67 68.6567 -2.398082 -2.136019 2 2
76
+ 68 69.1176 -4.828958 -4.584779 2 2
77
+ 69 69.5652 -4.337438 -8.151694 1 1
78
+ 70 70.0000 -3.360768 -2.864784 2 2
79
+ 71 69.0141 -3.405254 -3.026601 2 1
80
+ 72 68.0556 -2.809325 -3.164026 1 2
81
+ 73 68.4932 -3.109483 -3.547434 1 1
82
+ 74 67.5676 -2.556977 -2.492932 2 1
83
+ 75 68.0000 -1.501786 -2.236329 1 1
84
+ 76 68.4211 -1.980864 -4.200487 1 1
85
+ 77 67.5325 -2.797045 -3.186024 1 2
86
+ 78 67.9487 -1.355658 -1.611338 1 1
87
+ 79 67.0886 -2.162954 -2.507403 1 2
88
+ 80 67.5000 -2.425688 -3.654714 1 1
89
+ 81 67.9012 -3.597034 -3.449363 2 2
90
+ 82 67.0732 -2.607773 -2.706487 1 2
91
+ 83 66.2651 -6.378112 -6.619146 1 2
92
+ 84 66.6667 -2.514495 -2.430089 2 2
93
+ 85 67.0588 -5.011103 -5.537503 1 1
94
+ 86 66.2791 -6.077499 -6.237943 1 2
95
+ 87 65.5172 -1.708448 -1.440719 2 1
96
+ 88 64.7727 -4.499374 -4.214777 2 1
97
+ 89 65.1685 -3.265859 -3.542133 1 1
98
+ 90 65.5556 -6.049582 -4.844272 2 2
99
+ 91 64.8352 -4.706752 -4.247361 2 1
100
+ 92 65.2174 -5.427320 -4.084415 2 2
101
+ 93 65.5914 -2.815779 -2.424251 2 2
102
+ 94 65.9574 -3.854462 -3.017101 2 2
103
+ 95 66.3158 -5.334395 -6.056692 1 1
104
+ 96 66.6667 -2.027571 -3.153738 1 1
105
+ 97 67.0103 -5.370554 -5.508006 1 1
106
+ 98 67.3469 -5.269902 -3.699687 2 2
107
+ 99 66.6667 -3.024630 -2.353267 2 1
108
+ 100 67.0000 -7.508992 -6.792748 2 2
109
+ 101 66.3366 -4.656434 -4.677257 1 2
110
+ 102 66.6667 -3.921800 -5.287197 1 1
111
+ 103 66.0194 -2.951660 -3.331884 1 2
112
+ 104 66.3462 -8.598382 -8.434212 2 2
113
+ 105 66.6667 -1.292790 -2.317306 1 1
114
+ 106 66.9811 -3.274728 -2.837611 2 2
115
+ 107 67.2897 -3.378821 -2.696817 2 2
116
+ 108 67.5926 -3.883997 -3.038775 2 2
117
+ 109 66.9725 -5.658813 -5.896756 1 2
118
+ 110 67.2727 -4.124294 -3.672935 2 2
119
+ 111 67.5676 -4.052268 -3.616653 2 2
120
+ 112 66.9643 -3.386460 -3.222726 2 1
121
+ 113 66.3717 -4.582460 -4.604137 1 2
122
+ 114 66.6667 -3.844115 -4.519928 1 1
123
+ 115 66.9565 -2.206511 -3.055743 1 1
124
+ 116 67.2414 -3.479800 -2.573179 2 2
125
+ 117 66.6667 -4.513166 -5.172890 1 2
126
+ 118 66.9492 -1.016741 -2.905342 1 1
127
+ 119 67.2269 -2.714496 -3.036875 1 1
128
+ 120 66.6667 -2.684234 -2.423896 2 1
129
+ 121 66.9421 -1.344429 -0.847242 2 2
130
+ 122 67.2131 -6.194398 -4.877008 2 2
131
+ 123 66.6667 -3.333183 -3.113467 2 1
132
+ 124 66.9355 -2.689431 -2.364930 2 2
133
+ 125 67.2000 -3.860793 -3.120114 2 2
134
+ 126 66.6667 -6.809641 -7.177415 1 2
135
+ 127 66.9291 -2.258566 -1.202132 2 2
136
+ 128 67.1875 -2.174435 -2.589417 1 1
137
+ 129 67.4419 -2.207008 -1.433527 2 2
138
+ 130 67.6923 -5.612330 -5.240479 2 2
139
+ 131 67.1756 -3.720991 -3.939342 1 2
140
+ 132 66.6667 -3.051108 -3.191732 1 2
141
+ 133 66.9173 -1.511601 -3.873252 1 1
142
+ 134 67.1642 -5.235392 -4.470531 2 2
143
+ 135 67.4074 -4.519744 -5.192277 1 1
144
+ 136 67.6471 -5.178538 -4.369496 2 2
145
+ 137 67.1533 -6.636692 -6.552429 2 1
146
+ 138 66.6667 -4.182358 -4.544240 1 2
147
+ 139 66.1871 -4.631855 -4.097419 2 1
148
+ 140 66.4286 -1.295784 -1.175479 2 2
149
+ 141 66.6667 -4.540184 -3.963823 2 2
150
+ 142 66.9014 -2.893055 -2.801962 2 2
151
+ 143 67.1329 -1.579338 -5.153162 1 1
152
+ 144 66.6667 -2.053747 -2.053220 2 1
153
+ 145 66.2069 -1.353948 -13.988531 1 2
154
+ 146 66.4384 -4.407381 -4.015147 2 2
155
+ 147 65.9864 -4.844709 -5.343283 1 2
156
+ 148 66.2162 -2.155434 -2.507784 1 1
157
+ 149 66.4430 -3.807034 -3.485063 2 2
158
+ 150 66.6667 -4.349822 -3.410800 2 2
159
+ 151 66.2252 -2.453552 -2.355528 2 1
160
+ 152 66.4474 -4.041846 -3.929892 2 2
161
+ 153 66.0131 -4.221467 -3.882427 2 1
162
+ 154 66.2338 -3.544898 -3.766903 1 1
163
+ 155 66.4516 -2.091151 -1.952021 2 2
164
+ 156 66.0256 -2.792517 -2.486880 2 1
165
+ 157 66.2420 -3.733007 -4.949964 1 1
166
+ 158 66.4557 -4.427187 -4.119103 2 2
167
+ 159 66.6667 -3.706394 -5.328347 1 1
168
+ 160 66.8750 -4.007047 -3.173951 2 2
169
+ 161 67.0807 -2.656434 -2.800208 1 1
170
+ 162 66.6667 -0.519076 -15.810663 1 2
171
+ 163 66.2577 -1.870085 -1.858327 2 1
172
+ 164 66.4634 -5.203072 -4.389721 2 2
173
+ 165 66.0606 -4.652649 -4.711136 1 2
174
+ 166 66.2651 -4.160146 -3.668982 2 2
175
+ 167 65.8683 -4.716683 -4.320720 2 1
176
+ 168 65.4762 -4.015834 -4.512036 1 2
177
+ 169 65.0888 -3.605609 -3.945662 1 2
178
+ 170 64.7059 -4.554327 -4.154665 2 1
179
+ 171 64.9123 -3.789217 -4.781222 1 1
180
+ 172 65.1163 -4.630020 -5.742409 1 1
181
+ 173 65.3179 -3.465518 -2.953719 2 2
182
+ 174 65.5172 -6.898776 -6.352669 2 2
183
+ 175 65.1429 -4.901087 -4.624314 2 1
184
+ 176 65.3409 -1.849811 -3.234115 1 1
185
+ 177 65.5367 -5.493014 -5.194139 2 2
186
+ 178 65.1685 -2.597558 -2.295174 2 1
187
+ 179 65.3631 -2.914682 -3.056597 1 1
188
+ 180 65.5556 -5.983708 -5.393334 2 2
189
+ 181 65.7459 -4.010581 -2.834095 2 2
190
+ 182 65.9341 -4.517448 -3.894486 2 2
191
+ 183 66.1202 -6.495389 -6.168350 2 2
192
+ 184 66.3043 -2.672099 -3.798900 1 1
193
+ 185 66.4865 -6.063508 -6.060800 2 2
194
+ 186 66.6667 -3.464194 -3.770002 1 1
195
+ 187 66.8449 -5.122416 -4.863115 2 2
196
+ 188 66.4894 -3.540798 -3.332755 2 1
197
+ 189 66.1376 -5.436298 -5.693525 1 2
198
+ 190 66.3158 -4.891019 -4.533747 2 2
199
+ 191 66.4921 -3.858989 -4.229456 1 1
200
+ 192 66.6667 -6.605809 -4.103882 2 2
201
+ 193 66.8394 -4.258616 -4.191416 2 2
202
+ 194 67.0103 -2.170288 -1.373966 2 2
203
+ 195 66.6667 -4.034788 -3.917907 2 1
204
+ 196 66.8367 -4.212144 -2.924054 2 2
205
+ 197 67.0051 -0.757313 -2.902493 1 1
206
+ 198 67.1717 -5.678018 -6.985734 1 1
207
+ 199 67.3367 -5.131108 -3.890488 2 2
208
+ 200 67.5000 -3.308764 -3.062225 2 2
209
+ 201 67.6617 -2.921183 -3.603622 1 1
210
+ 202 67.8218 -7.611313 -7.461401 2 2
211
+ 203 67.4877 -3.290627 -3.372761 1 2
212
+ 204 67.6471 -3.253304 -3.519306 1 1
213
+ 205 67.8049 -3.830851 -3.162289 2 2
214
+ 206 67.4757 -6.010978 -6.071541 1 2
215
+ 207 67.6329 -2.762127 -2.970915 1 1
216
+ 208 67.7885 -4.547163 -3.069915 2 2
217
+ 209 67.9426 -4.479416 -4.026765 2 2
218
+ 210 67.6190 -8.219235 -9.194888 1 2
219
+ 211 67.2986 -6.153570 -5.775264 2 1
220
+ 212 67.4528 -4.608042 -3.515116 2 2
221
+ 213 67.1362 -5.164749 -6.031498 1 2
222
+ 214 67.2897 -3.212677 -3.493358 1 1
223
+ 215 67.4419 -5.630838 -5.218313 2 2
224
+ 216 67.1296 -3.717083 -3.460700 2 1
225
+ 217 67.2811 -1.327152 -1.921944 1 1
226
+ 218 67.4312 -2.386339 -2.332480 2 2
227
+ 219 67.5799 -2.815891 -3.090737 1 1
228
+ 220 67.7273 -3.417163 -3.396993 2 2
229
+ 221 67.8733 -2.599195 -7.946582 1 1
230
+ 222 67.5676 -0.278413 -6.601529 1 2
231
+ 223 67.7130 -4.603728 -4.346261 2 2
232
+ 224 67.4107 -6.302454 -6.386499 1 2
233
+ 225 67.5556 -3.975754 -3.103049 2 2
234
+ 226 67.6991 -3.306345 -3.646454 1 1
235
+ 227 67.4009 -4.305426 -3.626448 2 1
236
+ 228 67.5439 -1.882404 -3.609988 1 1
237
+ 229 67.2489 -4.816438 -4.961259 1 2
238
+ 230 67.3913 -3.206578 -3.448730 1 1
239
+ 231 67.5325 -2.173718 -2.514609 1 1
240
+ 232 67.6724 -2.357982 -1.456412 2 2
241
+ 233 67.8112 -2.025875 -2.464128 1 1
242
+ 234 67.9487 -3.974901 -3.260987 2 2
243
+ 235 68.0851 -3.976332 -3.538529 2 2
244
+ 236 67.7966 -2.510144 -2.351704 2 1
245
+ 237 67.9325 -1.130746 -1.396901 1 1
246
+ 238 68.0672 -3.524879 -2.971834 2 2
247
+ 239 68.2008 -2.898981 -2.730687 2 2
248
+ 240 67.9167 -7.945528 -8.245802 1 2
249
+ 241 68.0498 -4.600017 -3.575257 2 2
250
+ 242 68.1818 -6.192393 -5.806869 2 2
251
+ 243 68.3128 -4.212337 -4.306649 1 1
252
+ 244 68.4426 -1.995902 -3.643345 1 1
253
+ 245 68.5714 -4.696620 -5.674230 1 1
254
+ 246 68.6992 -2.407572 -3.083261 1 1
255
+ 247 68.4211 -4.963299 -5.603644 1 2
256
+ 248 68.5484 -3.375971 -3.100820 2 2
257
+ 249 68.6747 -3.016305 -2.026303 2 2
258
+ 250 68.8000 -3.312809 -3.939655 1 1
259
+ 251 68.9243 -0.838371 -0.897315 1 1
260
+ 252 69.0476 -6.538906 -6.221210 2 2
261
+ 253 68.7747 -5.496850 -5.342104 2 1
262
+ 254 68.8976 -4.371301 -3.386937 2 2
263
+ 255 69.0196 -3.342354 -4.695533 1 1
264
+ 256 69.1406 -3.658998 -4.602803 1 1
265
+ 257 68.8716 -2.733688 -2.218835 2 1
266
+ 258 68.6047 -3.680315 -4.163735 1 2
267
+ 259 68.3398 -4.452356 -4.330983 2 1
268
+ 260 68.4615 -4.610126 -5.295274 1 1
269
+ 261 68.5824 -3.825164 -3.744284 2 2
270
+ 262 68.7023 -5.139393 -4.997712 2 2
271
+ 263 68.8213 -3.912225 -4.709412 1 1
272
+ 264 68.9394 -4.401178 -4.164123 2 2
273
+ 265 69.0566 -1.439023 -1.966523 1 1
274
+ 266 69.1729 -6.464272 -8.220514 1 1
275
+ 267 69.2884 -3.824045 -3.945654 1 1
276
+ 268 69.0299 -2.949816 -2.689632 2 1
277
+ 269 69.1450 -3.561639 -3.142668 2 2
278
+ 270 69.2593 -2.346801 -4.794458 1 1
279
+ 271 69.3727 -3.179304 -3.105627 2 2
280
+ 272 69.4853 -4.296397 -3.666438 2 2
281
+ 273 69.5971 -4.024181 -3.705677 2 2
282
+ 274 69.7080 -2.969125 -2.172290 2 2
283
+ 275 69.8182 -1.369733 -1.994340 1 1
284
+ 276 69.5652 -2.536829 -2.362844 2 1
285
+ 277 69.6751 -2.893939 -3.155651 1 1
286
+ 278 69.7842 -2.857046 -2.917820 1 1
287
+ 279 69.8925 -4.774305 -3.070180 2 2
288
+ 280 70.0000 -4.130198 -3.803076 2 2
289
+ 281 70.1068 -2.155140 -1.613620 2 2
290
+ 282 70.2128 -4.107583 -3.686445 2 2
291
+ 283 70.3180 -3.260878 -3.250870 2 2
292
+ 284 70.0704 -2.772082 -2.754028 2 1
293
+ 285 70.1754 -5.705332 -6.280913 1 1
294
+ 286 70.2797 -3.572887 -3.652509 1 1
295
+ 287 70.3833 -2.188175 -4.231860 1 1
296
+ 288 70.4861 -3.940098 -2.718934 2 2
297
+ 289 70.5882 -2.261160 -2.711464 1 1
298
+ 290 70.6897 -5.197462 -5.726307 1 1
299
+ 291 70.7904 -1.772690 -3.173702 1 1
300
+ 292 70.5479 -2.057144 -2.351336 1 2
301
+ 293 70.6485 -3.682290 -2.991176 2 2
302
+ 294 70.4082 -5.080773 -3.799649 2 1
303
+ 295 70.5085 -1.872089 -2.239158 1 1
304
+ 296 70.6081 -4.951310 -4.798253 2 2
305
+ 297 70.3704 -0.994005 -1.234718 1 2
306
+ 298 70.1342 -6.019843 -7.679120 1 2
307
+ 299 70.2341 -2.471275 -1.990013 2 2
308
+ 300 70.3333 -3.064888 -4.108580 1 1
309
+ 301 70.4319 -4.515052 -5.596288 1 1
310
+ 302 70.5298 -4.464383 -4.275475 2 2
311
+ 303 70.2970 -9.433143 -8.075669 2 1
312
+ 304 70.3947 -3.388210 -3.370951 2 2
313
+ 305 70.1639 -2.941397 -3.141092 1 2
314
+ 306 70.2614 -2.590833 -2.227275 2 2
315
+ 307 70.3583 -3.138125 -3.299800 1 1
316
+ 308 70.4545 -4.549550 -4.471664 2 2
317
+ 309 70.5502 -3.048597 -3.701558 1 1
318
+ 310 70.6452 -5.112962 -5.043553 2 2
319
+ 311 70.7395 -1.403330 -4.973595 1 1
320
+ 312 70.5128 -5.823156 -4.751065 2 1
321
+ 313 70.2875 -4.433831 -5.450379 1 2
322
+ 314 70.3822 -5.768642 -5.003527 2 2
323
+ 315 70.4762 -2.461389 -3.767734 1 1
324
+ 316 70.5696 -6.642685 -5.172297 2 2
325
+ 317 70.3470 -4.698512 -4.437281 2 1
326
+ 318 70.4403 -5.177583 -4.930469 2 2
327
+ 319 70.5329 -2.146365 -3.247338 1 1
328
+ 320 70.6250 -3.989161 -5.221470 1 1
329
+ 321 70.4050 -2.919903 -3.010036 1 2
330
+ 322 70.1863 -2.396773 -2.457613 1 2
331
+ 323 70.2786 -2.612600 -3.250714 1 1
332
+ 324 70.3704 -4.099564 -4.347834 1 1
333
+ 325 70.4615 -4.917180 -4.180085 2 2
334
+ 326 70.5521 -2.883487 -1.946986 2 2
335
+ 327 70.6422 -2.754976 -3.695615 1 1
336
+ 328 70.7317 -4.103072 -4.458674 1 1
337
+ 329 70.8207 -3.700763 -9.799190 1 1
338
+ 330 70.9091 -1.277137 -1.964091 1 1
339
+ 331 70.6949 -5.635013 -4.036103 2 1
340
+ 332 70.7831 -2.033170 -3.146781 1 1
341
+ 333 70.8709 -2.888536 -1.860881 2 2
342
+ 334 70.9581 -1.882427 -5.792214 1 1
343
+ 335 71.0448 -4.392949 -4.396135 1 1
344
+ 336 70.8333 -3.206734 -4.050812 1 2
345
+ 337 70.9199 -2.102931 -1.927133 2 2
346
+ 338 70.7101 -4.013740 -3.377185 2 1
347
+ 339 70.7965 -6.144493 -5.083758 2 2
348
+ 340 70.8824 -5.497935 -5.777782 1 1
349
+ 341 70.9677 -2.944571 -4.857083 1 1
350
+ 342 71.0526 -6.075398 -5.912587 2 2
351
+ 343 71.1370 -3.906004 -4.290212 1 1
352
+ 344 71.2209 -2.558982 -2.605488 1 1
353
+ 345 71.3043 -1.850296 -3.462931 1 1
354
+ 346 71.0983 -4.221220 -6.769714 1 2
355
+ 347 71.1816 -4.351704 -3.837929 2 2
356
+ 348 71.2644 -1.027800 -2.641546 1 1
357
+ 349 71.0602 -3.443504 -5.973479 1 2
358
+ 350 70.8571 -1.199152 -4.064590 1 2
359
+ 351 70.9402 -2.341972 -3.333789 1 1
360
+ 352 71.0227 -4.085605 -3.021524 2 2
361
+ 353 70.8215 -2.466453 -1.955920 2 1
362
+ 354 70.9040 -5.474221 -4.025865 2 2
363
+ 355 70.9859 -3.347014 -2.600322 2 2
364
+ 356 70.7865 -3.388856 -3.504270 1 2
365
+ 357 70.8683 -4.206091 -4.061966 2 2
366
+ 358 70.9497 -2.444744 -2.396273 2 2
367
+ 359 71.0306 -5.992445 -3.760437 2 2
368
+ 360 70.8333 -2.520254 -3.047544 1 2
369
+ 361 70.9141 -3.152054 -1.792511 2 2
370
+ 362 70.9945 -5.066024 -4.733976 2 2
371
+ 363 70.7989 -2.563706 -2.337230 2 1
372
+ 364 70.8791 -6.372628 -6.891501 1 1
373
+ 365 70.9589 -5.765427 -6.360561 1 1
374
+ 366 70.7650 -4.219020 -9.182902 1 2
375
+ 367 70.8447 -5.396886 -5.968128 1 1
376
+ 368 70.9239 -3.232887 -1.276152 2 2
377
+ 369 70.7317 -3.922394 -4.002391 1 2
378
+ 370 70.8108 -4.898875 -4.974946 1 1
379
+ 371 70.8895 -4.291617 -4.754527 1 1
380
+ 372 70.6989 -2.880523 -2.582238 2 1
381
+ 373 70.7775 -5.426719 -5.731504 1 1
382
+ 374 70.8556 -1.913290 -3.351445 1 1
383
+ 375 70.9333 -3.231920 -2.333503 2 2
384
+ 376 71.0106 -5.689308 -4.260668 2 2
385
+ 377 70.8223 -3.903446 -4.145327 1 2
386
+ 378 70.6349 -3.378970 -3.631984 1 2
387
+ 379 70.4485 -1.531905 -2.366236 1 2
388
+ 380 70.2632 -7.850513 -7.267235 2 1
389
+ 381 70.3412 -11.710489 -8.973927 2 2
390
+ 382 70.1571 -2.180809 -2.244128 1 2
391
+ 383 70.2350 -1.719019 -0.728431 2 2
392
+ 384 70.0521 -3.952284 -2.892804 2 1
393
+ 385 69.8701 -1.324019 -1.138520 2 1
394
+ 386 69.9482 -1.626065 -9.408861 1 1
395
+ 387 70.0258 -1.490253 -1.854356 1 1
396
+ 388 69.8454 -5.121956 -6.176535 1 2
397
+ 389 69.9229 -2.461058 -1.901264 2 2
398
+ 390 70.0000 -2.635324 -3.724351 1 1
399
+ 391 69.8210 -3.690833 -5.352731 1 2
400
+ 392 69.6429 -6.105227 -4.428777 2 1
401
+ 393 69.7201 -6.778244 -6.124967 2 2
402
+ 394 69.7970 -1.972893 -4.491798 1 1
403
+ 395 69.6203 -4.398692 -4.543170 1 2
404
+ 396 69.4444 -7.062549 -6.248410 2 1
405
+ 397 69.2695 -7.398591 -6.968115 2 1
406
+ 398 69.3467 -3.984722 -7.112214 1 1
407
+ 399 69.1729 -5.011809 -4.980955 2 1
408
+ 400 69.2500 -1.954567 -4.650184 1 1
409
+ 401 69.3267 -2.695418 -3.109483 1 1
410
+ 402 69.1542 -4.130033 -4.282753 1 2
411
+ 403 69.2308 -7.225667 -7.172878 2 2
412
+ 404 69.3069 -4.209410 -2.678038 2 2
413
+ 405 69.3827 -3.229034 -4.164284 1 1
414
+ 406 69.4581 -8.243574 -9.164969 1 1
415
+ 407 69.5332 -5.371981 -5.165489 2 2
416
+ 408 69.6078 -3.339207 -3.150958 2 2
417
+ 409 69.6822 -3.855037 -4.139284 1 1
418
+ 410 69.7561 -5.562648 -6.276645 1 1
419
+ 411 69.8297 -5.248012 -5.885139 1 1
420
+ 412 69.9029 -4.307793 -2.917091 2 2
421
+ 413 69.9758 -4.144701 -3.087673 2 2
422
+ 414 69.8068 -4.808239 -4.951344 1 2
423
+ 415 69.8795 -2.913055 -2.885088 2 2
424
+ 416 69.9519 -5.541188 -3.134670 2 2
425
+ 417 70.0240 -5.130403 -5.837963 1 1
426
+ 418 70.0957 -5.757628 -7.026382 1 1
427
+ 419 69.9284 -4.444838 -5.158317 1 2
428
+ 420 69.7619 -3.336051 -3.410633 1 2
429
+ 421 69.8337 -4.276133 -3.439562 2 2
430
+ 422 69.6682 -2.584708 -2.053797 2 1
431
+ 423 69.5035 -3.580045 -3.373130 2 1
432
+ 424 69.5755 -5.078812 -5.644776 1 1
433
+ 425 69.6471 -3.873866 -2.586675 2 2
434
+ 426 69.4836 -1.972991 -2.556873 1 2
435
+ 427 69.5550 -3.952948 -5.266049 1 1
436
+ 428 69.3925 -3.433964 -2.753516 2 1
437
+ 429 69.4639 -4.080954 -4.069551 2 2
438
+ 430 69.3023 -6.730424 -5.667634 2 1
439
+ 431 69.3735 -5.289485 -5.924118 1 1
440
+ 432 69.2130 -4.291545 -4.678481 1 2
441
+ 433 69.2841 -1.741498 -1.612025 2 2
442
+ 434 69.3548 -7.375439 -8.023372 1 1
443
+ 435 69.4253 -3.170892 -3.786865 1 1
444
+ 436 69.4954 -0.565934 -2.357858 1 1
445
+ 437 69.5652 -3.863678 -3.760226 2 2
446
+ 438 69.6347 -3.698835 -3.565849 2 2
447
+ 439 69.4761 -3.690658 -3.096106 2 1
448
+ 440 69.5455 -4.233709 -4.978428 1 1
449
+ 441 69.6145 -3.901012 -5.258943 1 1
450
+ 442 69.6833 -5.117783 -5.507871 1 1
451
+ 443 69.7517 -1.638414 -3.385224 1 1
452
+ 444 69.5946 -5.671926 -7.252038 1 2
453
+ 445 69.6629 -1.762344 -2.982723 1 1
454
+ 446 69.7309 -5.791960 -4.848780 2 2
455
+ 447 69.5749 -1.838997 -1.838151 2 1
456
+ 448 69.6429 -4.034778 -3.422896 2 2
457
+ 449 69.7105 -3.074008 -2.924282 2 2
458
+ 450 69.7778 -4.215415 -3.370661 2 2
459
+ 451 69.8448 -4.162704 -3.343832 2 2
460
+ 452 69.9115 -4.751702 -5.557837 1 1
461
+ 453 69.9779 -5.445050 -4.690263 2 2
462
+ 454 70.0441 -3.090643 -3.446739 1 1
463
+ 455 69.8901 -1.396506 -8.869473 1 2
464
+ 456 69.9561 -4.648709 -4.597373 2 2
465
+ 457 69.8031 -11.142669 -10.616446 2 1
466
+ 458 69.8690 -4.271450 -3.687119 2 2
467
+ 459 69.9346 -4.596141 -4.323482 2 2
468
+ 460 70.0000 -0.048001 -7.286666 1 1
469
+ 461 70.0651 -2.587376 -3.748636 1 1
470
+ 462 69.9134 -5.441957 -5.262061 2 1
471
+ 463 69.9784 -1.031371 -1.278809 1 1
472
+ 464 69.8276 -2.753017 -2.602764 2 1
473
+ 465 69.8925 -3.150552 -3.117472 2 2
474
+ 466 69.9571 -5.428410 -5.066604 2 2
475
+ 467 70.0214 -4.903269 -5.417830 1 1
476
+ 468 70.0855 -5.576814 -4.170667 2 2
477
+ 469 69.9360 -1.969762 -1.711309 2 1
478
+ 470 70.0000 -1.909906 -1.687729 2 2
479
+ 471 69.8514 -2.433513 -2.004988 2 1
480
+ 472 69.9153 -4.418573 -3.651079 2 2
481
+ 473 69.9789 -4.535279 -3.157188 2 2
482
+ 474 70.0422 -5.566380 -6.205374 1 1
483
+ 475 70.1053 -1.475825 -1.174528 2 2
484
+ 476 70.1681 -4.814318 -4.494823 2 2
485
+ 477 70.0210 -4.860154 -5.521813 1 2
486
+ 478 69.8745 -4.664805 -4.275145 2 1
487
+ 479 69.9374 -3.570051 -3.713366 1 1
488
+ 480 70.0000 -6.007993 -5.595348 2 2
489
+ 481 70.0624 -2.248334 -2.785947 1 1
490
+ 482 69.9170 -4.421207 -5.324065 1 2
491
+ 483 69.7723 -5.625041 -4.727635 2 1
492
+ 484 69.6281 -2.638904 -1.583613 2 1
493
+ 485 69.6907 -1.630841 -2.697541 1 1
494
+ 486 69.7531 -2.754014 -3.410098 1 1
495
+ 487 69.8152 -2.504989 -1.777390 2 2
496
+ 488 69.8770 -3.386166 -2.236151 2 2
497
+ 489 69.9387 -2.256036 -2.754419 1 1
498
+ 490 70.0000 -3.399930 -3.810551 1 1
499
+ 491 70.0611 -5.658032 -4.815548 2 2
500
+ 492 70.1220 -5.690098 -5.349053 2 2
501
+ 493 70.1826 -1.471227 -2.891692 1 1
502
+ 494 70.0405 -4.601736 -4.713576 1 2
503
+ 495 70.1010 -4.112256 -4.324358 1 1
504
+ 496 70.1613 -3.642080 -3.500962 2 2
505
+ 497 70.0201 -5.233197 -4.946602 2 1
506
+ 498 69.8795 -1.417887 -1.708691 1 2
507
+ 499 69.9399 -2.771445 -3.234040 1 1
508
+ 500 70.0000 -5.248368 -4.318505 2 2
509
+ 501 70.0599 -1.127621 -1.980365 1 1
510
+ 502 70.1195 -1.702671 -1.838225 1 1
511
+ 503 70.1789 -3.086535 -2.919570 2 2
512
+ 504 70.2381 -2.904469 -2.973240 1 1
513
+ 505 70.0990 -7.468460 -8.288746 1 2
514
+ 506 69.9605 -6.116572 -5.960472 2 1
515
+ 507 70.0197 -4.777333 -4.818705 1 1
516
+ 508 70.0787 -6.469469 -2.595325 2 2
517
+ 509 70.1375 -3.418619 -8.422461 1 1
518
+ 510 70.0000 -4.296793 -3.677398 2 1
519
+ 511 70.0587 -5.125052 -3.927873 2 2
520
+ 512 70.1172 -3.738578 -4.327974 1 1
521
+ 513 70.1754 -3.850675 -3.793802 2 2
522
+ 514 70.0389 -5.933384 -5.656672 2 1
523
+ 515 69.9029 -7.406465 -7.017375 2 1
524
+ 516 69.7674 -4.277382 -4.166453 2 1
525
+ 517 69.8259 -1.496753 -1.548515 1 1
526
+ 518 69.8842 -3.075224 -2.060291 2 2
527
+ 519 69.9422 -9.763030 -9.119306 2 2
528
+ 520 70.0000 -2.831803 -2.274945 2 2
529
+ 521 70.0576 -1.306701 -1.455363 1 1
530
+ 522 70.1149 -3.680949 -3.015634 2 2
531
+ 523 69.9809 -3.669348 -3.840536 1 2
532
+ 524 70.0382 -6.923017 -7.093206 1 1
533
+ 525 70.0952 -5.003000 -1.807381 2 2
534
+ 526 70.1521 -0.262336 -17.793411 1 1
535
+ 527 70.2087 -6.724911 -5.168465 2 2
536
+ 528 70.2652 -6.456259 -5.905441 2 2
537
+ 529 70.3214 -5.188770 -6.161730 1 1
538
+ 530 70.3774 -5.631208 -4.101879 2 2
539
+ 531 70.2448 -3.907815 -3.524150 2 1
540
+ 532 70.3008 -2.293680 -2.994776 1 1
541
+ 533 70.3565 -3.677505 -3.015044 2 2
542
+ 534 70.4120 -3.107142 -3.674581 1 1
543
+ 535 70.2804 -3.214589 -3.274925 1 2
544
+ 536 70.1493 -2.318542 -2.107107 2 1
545
+ 537 70.2048 -6.238227 -6.511444 1 1
546
+ 538 70.2602 -2.726542 -2.737637 1 1
547
+ 539 70.3154 -4.634414 -2.725788 2 2
548
+ 540 70.1852 -2.725173 -3.008456 1 2
549
+ 541 70.2403 -2.668614 -2.716535 1 1
550
+ 542 70.1107 -3.743375 -4.571726 1 2
551
+ 543 70.1657 -5.507936 -5.010995 2 2
552
+ 544 70.2206 -2.809943 -4.408360 1 1
553
+ 545 70.0917 -1.333641 -4.849306 1 2
554
+ 546 70.1465 -2.840882 -3.533504 1 1
555
+ 547 70.2011 -6.149029 -4.833288 2 2
556
+ 548 70.2555 -4.364368 -4.656668 1 1
557
+ 549 70.1275 -5.647542 -5.494740 2 1
558
+ 550 70.1818 -3.535378 -3.412114 2 2
559
+ 551 70.0544 -1.916304 -8.150373 1 2
560
+ 552 70.1087 -2.565929 -3.253664 1 1
561
+ 553 70.1627 -5.913306 -5.692492 2 2
562
+ 554 70.0361 -4.390555 -4.843592 1 2
563
+ 555 70.0901 -2.997225 -2.078758 2 2
564
+ 556 70.1439 -6.090450 -4.869322 2 2
565
+ 557 70.1975 -4.421179 -5.371248 1 1
566
+ 558 70.2509 -2.290604 -3.738022 1 1
567
+ 559 70.3041 -2.504094 -2.342979 2 2
568
+ 560 70.1786 -2.619698 -2.099572 2 1
569
+ 561 70.2317 -1.975783 -2.614818 1 1
570
+ 562 70.2847 -0.704535 -1.514128 1 1
571
+ 563 70.1599 -1.567814 -2.651136 1 2
572
+ 564 70.2128 -3.130314 -3.743594 1 1
573
+ 565 70.0885 -3.718529 -3.738443 1 2
574
+ 566 70.1413 -4.185196 -5.182401 1 1
575
+ 567 70.0176 -2.769588 -2.784697 1 2
576
+ 568 70.0704 -7.307737 -5.166417 2 2
577
+ 569 69.9473 -5.623743 -5.277799 2 1
578
+ 570 70.0000 -4.450958 -6.254492 1 1
579
+ 571 70.0525 -4.651528 -3.353267 2 2
580
+ 572 70.1049 -2.942632 -2.688879 2 2
581
+ 573 69.9825 -4.544494 -4.991291 1 2
582
+ 574 70.0348 -3.805833 -2.573040 2 2
583
+ 575 70.0870 -5.004134 -4.702531 2 2
584
+ 576 70.1389 -2.769706 -3.370371 1 1
585
+ 577 70.1906 -4.407430 -4.078175 2 2
586
+ 578 70.2422 -3.219502 -2.869369 2 2
587
+ 579 70.2936 -5.483323 -5.464789 2 2
588
+ 580 70.3448 -1.979996 -3.440853 1 1
589
+ 581 70.3959 -5.551309 -7.513388 1 1
590
+ 582 70.2749 -5.291482 -5.558209 1 2
591
+ 583 70.3259 -4.640796 -4.512021 2 2
592
+ 584 70.2055 -4.806472 -4.775786 2 1
593
+ 585 70.2564 -0.409592 -1.416006 1 1
594
+ 586 70.3072 -4.023146 -4.474933 1 1
595
+ 587 70.1874 -5.313824 -5.067295 2 1
596
+ 588 70.2381 -4.984319 -2.677775 2 2
597
+ 589 70.1188 -3.200726 -3.507306 1 2
598
+ 590 70.1695 -4.386247 -3.815193 2 2
599
+ 591 70.0508 -4.009931 -4.160497 1 2
600
+ 592 70.1014 -3.309110 -4.182189 1 1
601
+ 593 70.1518 -4.360841 -4.323915 2 2
602
+ 594 70.0337 -4.741245 -3.758525 2 1
603
+ 595 70.0840 -1.946007 -1.905343 2 2
604
+ 596 70.1342 -4.981290 -4.559746 2 2
605
+ 597 70.1843 -2.294994 -2.295590 1 1
606
+ 598 70.2341 -3.351894 -3.702831 1 1
607
+ 599 70.2838 -2.848432 -4.247764 1 1
608
+ 600 70.3333 -3.876140 -3.659794 2 2
609
+ 601 70.3827 -3.439826 -2.948650 2 2
610
+ 602 70.4319 -3.235380 -5.352420 1 1
611
+ 603 70.4809 -6.975878 -5.258179 2 2
612
+ 604 70.3642 -3.133326 -3.223966 1 2
613
+ 605 70.2479 -2.639240 -1.883118 2 1
614
+ 606 70.2970 -3.197966 -2.675207 2 2
615
+ 607 70.1812 -3.022762 -2.611722 2 1
616
+ 608 70.2303 -2.951456 -3.179042 1 1
617
+ 609 70.1149 -7.712037 -7.565568 2 1
618
+ 610 70.1639 -2.919109 -3.182302 1 1
619
+ 611 70.0491 -4.441766 -4.367417 2 1
620
+ 612 70.0980 -5.346774 -6.637982 1 1
621
+ 613 70.1468 -3.206577 -2.284918 2 2
622
+ 614 70.1954 -5.413858 -5.291433 2 2
623
+ 615 70.2439 -2.831960 -2.487804 2 2
624
+ 616 70.1299 -5.986930 -6.211999 1 2
625
+ 617 70.1783 -12.250810 -5.675500 2 2
626
+ 618 70.0647 -3.856429 -4.152160 1 2
627
+ 619 70.1131 -3.932436 -3.369243 2 2
628
+ 620 70.0000 -4.846333 -4.614872 2 1
629
+ 621 70.0483 -3.285696 -1.996834 2 2
630
+ 622 70.0965 -3.403208 -3.781863 1 1
631
+ 623 70.1445 -3.248004 -3.491855 1 1
632
+ 624 70.1923 -4.729292 -3.939061 2 2
633
+ 625 70.2400 -4.195808 -6.544857 1 1
634
+ 626 70.1278 -1.778387 -0.999567 2 1
635
+ 627 70.1754 -4.038756 -4.536799 1 1
636
+ 628 70.2229 -3.140392 -2.619173 2 2
637
+ 629 70.2703 -2.046552 -2.599115 1 1
638
+ 630 70.3175 -1.177008 -1.773550 1 1
639
+ 631 70.3645 -2.476307 -3.860799 1 1
640
+ 632 70.4114 -1.759346 -1.309474 2 2
641
+ 633 70.3002 -2.102228 -2.120774 1 2
642
+ 634 70.3470 -2.625976 -3.477599 1 1
643
+ 635 70.2362 -4.610543 -5.164552 1 2
644
+ 636 70.2830 -1.160834 -2.927926 1 1
645
+ 637 70.3297 -3.409387 -5.695817 1 1
646
+ 638 70.3762 -5.990832 -6.687023 1 1
647
+ 639 70.4225 -1.861946 -2.912908 1 1
648
+ 640 70.4688 -2.399562 -2.302016 2 2
649
+ 641 70.3588 -6.853079 -5.581668 2 1
650
+ 642 70.4050 -4.776742 -3.267476 2 2
651
+ 643 70.4510 -2.002271 -1.448071 2 2
652
+ 644 70.4969 -3.254791 -4.292400 1 1
653
+ 645 70.5426 -1.057652 -3.020738 1 1
654
+ 646 70.4334 -3.948150 -3.432905 2 1
655
+ 647 70.3246 -3.341922 -2.312466 2 1
656
+ 648 70.3704 -3.444494 -2.994037 2 2
657
+ 649 70.4160 -3.650745 -3.990887 1 1
658
+ 650 70.4615 -3.949920 -4.483450 1 1
659
+ 651 70.3533 -6.073598 -6.391645 1 2
660
+ 652 70.3988 -5.895391 -5.017771 2 2
661
+ 653 70.4441 -5.255305 -3.861006 2 2
662
+ 654 70.3364 -3.520408 -3.852512 1 2
663
+ 655 70.2290 -2.759230 -2.911669 1 2
664
+ 656 70.2744 -3.885617 -2.501316 2 2
665
+ 657 70.1674 -5.541340 -5.798390 1 2
666
+ 658 70.2128 -4.134595 -2.137526 2 2
667
+ 659 70.2580 -2.369566 -1.959589 2 2
668
+ 660 70.1515 -3.135733 -3.785056 1 2
669
+ 661 70.1967 -3.247208 -3.481769 1 1
670
+ 662 70.2417 -3.367330 -4.307842 1 1
671
+ 663 70.1357 -6.295202 -6.447533 1 2
672
+ 664 70.0301 -4.077337 -3.769963 2 1
673
+ 665 70.0752 -3.717223 -3.608134 2 2
674
+ 666 70.1201 -3.203295 -4.806832 1 1
675
+ 667 70.1649 -8.018996 -6.673742 2 2
676
+ 668 70.2096 -2.125388 -2.190125 1 1
677
+ 669 70.2541 -2.478151 -3.223055 1 1
678
+ 670 70.1493 -3.576915 -3.273778 2 1
679
+ 671 70.1937 -2.962452 -4.556664 1 1
680
+ 672 70.2381 -2.971766 -2.937562 2 2
681
+ 673 70.2823 -1.949597 -1.651224 2 2
682
+ 674 70.1780 -5.078279 -4.797681 2 1
683
+ 675 70.0741 -3.689014 -3.617026 2 1
684
+ 676 69.9704 -2.279256 -2.691937 1 2
685
+ 677 70.0148 -3.796523 -4.519240 1 1
686
+ 678 69.9115 -5.666880 -6.055246 1 2
687
+ 679 69.9558 -5.837439 -4.833749 2 2
688
+ 680 70.0000 -3.175436 -4.062710 1 1
689
+ 681 70.0441 -2.877535 -3.225712 1 1
690
+ 682 69.9413 -2.343846 -2.353436 1 2
691
+ 683 69.9854 -1.965481 -3.474722 1 1
692
+ 684 70.0292 -4.892859 -7.286825 1 1
693
+ 685 70.0730 -4.530518 -2.206820 2 2
694
+ 686 70.1166 -6.060281 -5.649464 2 2
695
+ 687 70.1601 -4.438544 -3.426948 2 2
696
+ 688 70.0581 -8.496990 -8.468616 2 1
697
+ 689 70.1016 -2.635051 -2.261382 2 2
698
+ 690 70.0000 -2.789154 -2.935484 1 2
699
+ 691 69.8987 -2.918720 -2.457160 2 1
700
+ 692 69.9422 -1.888323 -2.669247 1 1
701
+ 693 69.9856 -4.856672 -5.409632 1 1
702
+ 694 70.0288 -3.425603 -3.318269 2 2
703
+ 695 70.0719 -4.200788 -4.943260 1 1
704
+ 696 70.1149 -4.915711 -5.589746 1 1
705
+ 697 70.1578 -2.196992 -2.596272 1 1
706
+ 698 70.2006 -4.157510 -3.789952 2 2
707
+ 699 70.2432 -3.503533 -4.736110 1 1
708
+ 700 70.2857 -4.617546 -4.326663 2 2
709
+ 701 70.1854 -2.385080 -2.187102 2 1
710
+ 702 70.0855 -4.106074 -5.037796 1 2
711
+ 703 70.1280 -3.441170 -4.070502 1 1
712
+ 704 70.1705 -1.168596 -1.468433 1 1
713
+ 705 70.0709 -3.392010 -3.651516 1 2
714
+ 706 70.1133 -3.342202 -2.857856 2 2
715
+ 707 70.0141 -4.897212 -4.671188 2 1
716
+ 708 70.0565 -3.808279 -3.180806 2 2
717
+ 709 70.0987 -2.900715 -2.447774 2 2
718
+ 710 70.1408 -4.001250 -5.007661 1 1
719
+ 711 70.0422 -1.731481 -2.201700 1 2
720
+ 712 69.9438 -2.156821 -3.498678 1 2
721
+ 713 69.8457 -3.384270 -3.409154 1 2
722
+ 714 69.7479 -3.008576 -3.126923 1 2
723
+ 715 69.7902 -4.103062 -3.579433 2 2
724
+ 716 69.8324 -4.578363 -4.408474 2 2
725
+ 717 69.8745 -2.199079 -0.260915 2 2
726
+ 718 69.7772 -4.977598 -4.945681 2 1
727
+ 719 69.6801 -5.204844 -4.841586 2 1
728
+ 720 69.5833 -2.580046 -5.113744 1 2
729
+ 721 69.4868 -5.000897 -5.927897 1 2
730
+ 722 69.3906 -2.257700 -1.791322 2 1
731
+ 723 69.4329 -2.428018 -2.730106 1 1
732
+ 724 69.4751 -4.423358 -4.221862 2 2
733
+ 725 69.3793 -2.288509 -3.013081 1 2
734
+ 726 69.4215 -4.351902 -4.035414 2 2
735
+ 727 69.4635 -3.502896 -2.675055 2 2
736
+ 728 69.3681 -3.412232 -3.353740 2 1
737
+ 729 69.4102 -1.004031 -2.393043 1 1
738
+ 730 69.4521 -3.860554 -4.292231 1 1
739
+ 731 69.3570 -5.717840 -4.854691 2 1
740
+ 732 69.3989 -2.988842 -2.123508 2 2
741
+ 733 69.3042 -6.594011 -6.659606 1 2
742
+ 734 69.3460 -0.854530 -11.558423 1 1
743
+ 735 69.2517 -3.614158 -3.054090 2 1
744
+ 736 69.2935 -2.601015 -3.309520 1 1
745
+ 737 69.3351 -4.706058 -5.394544 1 1
746
+ 738 69.2412 -4.348562 -4.616922 1 2
747
+ 739 69.2828 -1.114702 -6.888380 1 1
748
+ 740 69.3243 -4.377176 -3.259498 2 2
749
+ 741 69.2308 -4.804738 -4.386001 2 1
750
+ 742 69.2722 -2.352131 -2.275234 2 2
751
+ 743 69.1790 -4.072536 -4.109853 1 2
752
+ 744 69.2204 -4.868029 -4.070650 2 2
753
+ 745 69.2617 -2.640790 -3.583345 1 1
754
+ 746 69.1689 -7.632085 -7.504710 2 1
755
+ 747 69.2102 -5.629326 -6.625494 1 1
756
+ 748 69.2513 -4.879400 -3.774464 2 2
757
+ 749 69.2924 -4.057473 -5.319310 1 1
758
+ 750 69.3333 -3.100903 -7.176168 1 1
759
+
760
+ Final Winogrande score(750 tasks): 69.3333 +/- 1.6849
scores/Hammer2.1-7b-q3_k_s.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 66.66666667
20
+ 10 70.00000000
21
+ 11 63.63636364
22
+ 12 66.66666667
23
+ 13 69.23076923
24
+ 14 71.42857143
25
+ 15 73.33333333
26
+ 16 68.75000000
27
+ 17 70.58823529
28
+ 18 66.66666667
29
+ 19 63.15789474
30
+ 20 60.00000000
31
+ 21 57.14285714
32
+ 22 59.09090909
33
+ 23 60.86956522
34
+ 24 62.50000000
35
+ 25 60.00000000
36
+ 26 57.69230769
37
+ 27 59.25925926
38
+ 28 57.14285714
39
+ 29 58.62068966
40
+ 30 60.00000000
41
+ 31 58.06451613
42
+ 32 56.25000000
43
+ 33 54.54545455
44
+ 34 52.94117647
45
+ 35 54.28571429
46
+ 36 52.77777778
47
+ 37 54.05405405
48
+ 38 52.63157895
49
+ 39 53.84615385
50
+ 40 55.00000000
51
+ 41 56.09756098
52
+ 42 54.76190476
53
+ 43 55.81395349
54
+ 44 54.54545455
55
+ 45 53.33333333
56
+ 46 54.34782609
57
+ 47 55.31914894
58
+ 48 54.16666667
59
+ 49 53.06122449
60
+ 50 52.00000000
61
+ 51 50.98039216
62
+ 52 51.92307692
63
+ 53 52.83018868
64
+ 54 53.70370370
65
+ 55 52.72727273
66
+ 56 51.78571429
67
+ 57 52.63157895
68
+ 58 53.44827586
69
+ 59 54.23728814
70
+ 60 55.00000000
71
+ 61 54.09836066
72
+ 62 54.83870968
73
+ 63 55.55555556
74
+ 64 56.25000000
75
+ 65 55.38461538
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 56.52173913
80
+ 70 55.71428571
81
+ 71 54.92957746
82
+ 72 54.16666667
83
+ 73 54.79452055
84
+ 74 55.40540541
85
+ 75 56.00000000
86
+ 76 55.26315789
87
+ 77 55.84415584
88
+ 78 55.12820513
89
+ 79 54.43037975
90
+ 80 55.00000000
91
+ 81 54.32098765
92
+ 82 54.87804878
93
+ 83 55.42168675
94
+ 84 54.76190476
95
+ 85 55.29411765
96
+ 86 55.81395349
97
+ 87 56.32183908
98
+ 88 56.81818182
99
+ 89 57.30337079
100
+ 90 56.66666667
101
+ 91 57.14285714
102
+ 92 57.60869565
103
+ 93 56.98924731
104
+ 94 56.38297872
105
+ 95 56.84210526
106
+ 96 57.29166667
107
+ 97 56.70103093
108
+ 98 56.12244898
109
+ 99 56.56565657
110
+ 100 57.00000000
111
+ 101 56.43564356
112
+ 102 56.86274510
113
+ 103 56.31067961
114
+ 104 55.76923077
115
+ 105 55.23809524
116
+ 106 54.71698113
117
+ 107 55.14018692
118
+ 108 54.62962963
119
+ 109 54.12844037
120
+ 110 54.54545455
121
+ 111 54.95495495
122
+ 112 55.35714286
123
+ 113 54.86725664
124
+ 114 54.38596491
125
+ 115 54.78260870
126
+ 116 54.31034483
127
+ 117 53.84615385
128
+ 118 54.23728814
129
+ 119 53.78151261
130
+ 120 54.16666667
131
+ 121 53.71900826
132
+ 122 54.09836066
133
+ 123 54.47154472
134
+ 124 54.03225806
135
+ 125 54.40000000
136
+ 126 54.76190476
137
+ 127 54.33070866
138
+ 128 54.68750000
139
+ 129 54.26356589
140
+ 130 53.84615385
141
+ 131 54.19847328
142
+ 132 54.54545455
143
+ 133 54.88721805
144
+ 134 54.47761194
145
+ 135 54.81481481
146
+ 136 54.41176471
147
+ 137 54.74452555
148
+ 138 55.07246377
149
+ 139 55.39568345
150
+ 140 55.71428571
151
+ 141 56.02836879
152
+ 142 55.63380282
153
+ 143 55.94405594
154
+ 144 55.55555556
155
+ 145 55.86206897
156
+ 146 55.47945205
157
+ 147 55.78231293
158
+ 148 55.40540541
159
+ 149 55.70469799
160
+ 150 56.00000000
161
+ 151 56.29139073
162
+ 152 56.57894737
163
+ 153 56.86274510
164
+ 154 56.49350649
165
+ 155 56.12903226
166
+ 156 56.41025641
167
+ 157 56.68789809
168
+ 158 56.32911392
169
+ 159 56.60377358
170
+ 160 56.25000000
171
+ 161 55.90062112
172
+ 162 56.17283951
173
+ 163 55.82822086
174
+ 164 55.48780488
175
+ 165 55.75757576
176
+ 166 56.02409639
177
+ 167 55.68862275
178
+ 168 55.95238095
179
+ 169 56.21301775
180
+ 170 56.47058824
181
+ 171 56.14035088
182
+ 172 56.39534884
183
+ 173 56.64739884
184
+ 174 56.89655172
185
+ 175 56.57142857
186
+ 176 56.25000000
187
+ 177 56.49717514
188
+ 178 56.17977528
189
+ 179 56.42458101
190
+ 180 56.11111111
191
+ 181 56.35359116
192
+ 182 56.59340659
193
+ 183 56.28415301
194
+ 184 55.97826087
195
+ 185 56.21621622
196
+ 186 56.45161290
197
+ 187 56.68449198
198
+ 188 56.38297872
199
+ 189 56.08465608
200
+ 190 55.78947368
201
+ 191 56.02094241
202
+ 192 55.72916667
203
+ 193 55.44041451
204
+ 194 55.67010309
205
+ 195 55.89743590
206
+ 196 56.12244898
207
+ 197 56.34517766
208
+ 198 56.06060606
209
+ 199 55.77889447
210
+ 200 56.00000000
211
+ 201 56.21890547
212
+ 202 56.43564356
213
+ 203 56.65024631
214
+ 204 56.86274510
215
+ 205 56.58536585
216
+ 206 56.31067961
217
+ 207 56.52173913
218
+ 208 56.73076923
219
+ 209 56.93779904
220
+ 210 57.14285714
221
+ 211 57.34597156
222
+ 212 57.07547170
223
+ 213 56.80751174
224
+ 214 57.00934579
225
+ 215 57.20930233
226
+ 216 56.94444444
227
+ 217 56.68202765
228
+ 218 56.42201835
229
+ 219 56.62100457
230
+ 220 56.81818182
231
+ 221 56.56108597
232
+ 222 56.30630631
233
+ 223 56.05381166
234
+ 224 56.25000000
235
+ 225 56.00000000
236
+ 226 55.75221239
237
+ 227 55.94713656
238
+ 228 56.14035088
239
+ 229 56.33187773
240
+ 230 56.52173913
241
+ 231 56.27705628
242
+ 232 56.03448276
243
+ 233 56.22317597
244
+ 234 56.41025641
245
+ 235 56.17021277
246
+ 236 55.93220339
247
+ 237 56.11814346
248
+ 238 55.88235294
249
+ 239 56.06694561
250
+ 240 56.25000000
251
+ 241 56.43153527
252
+ 242 56.19834711
253
+ 243 56.37860082
254
+ 244 56.14754098
255
+ 245 56.32653061
256
+ 246 56.50406504
257
+ 247 56.68016194
258
+ 248 56.85483871
259
+ 249 57.02811245
260
+ 250 57.20000000
261
+ 251 57.37051793
262
+ 252 57.53968254
263
+ 253 57.70750988
264
+ 254 57.48031496
265
+ 255 57.64705882
266
+ 256 57.81250000
267
+ 257 57.97665370
268
+ 258 57.75193798
269
+ 259 57.52895753
270
+ 260 57.69230769
271
+ 261 57.85440613
272
+ 262 57.63358779
273
+ 263 57.79467681
274
+ 264 57.95454545
275
+ 265 57.73584906
276
+ 266 57.51879699
277
+ 267 57.67790262
278
+ 268 57.46268657
279
+ 269 57.62081784
280
+ 270 57.77777778
281
+ 271 57.93357934
282
+ 272 57.72058824
283
+ 273 57.50915751
284
+ 274 57.29927007
285
+ 275 57.09090909
286
+ 276 57.24637681
287
+ 277 57.40072202
288
+ 278 57.19424460
289
+ 279 57.34767025
290
+ 280 57.50000000
291
+ 281 57.65124555
292
+ 282 57.80141844
293
+ 283 57.59717314
294
+ 284 57.74647887
295
+ 285 57.89473684
296
+ 286 58.04195804
297
+ 287 57.83972125
298
+ 288 57.98611111
299
+ 289 57.78546713
300
+ 290 57.93103448
301
+ 291 58.07560137
302
+ 292 57.87671233
303
+ 293 58.02047782
304
+ 294 57.82312925
305
+ 295 57.96610169
306
+ 296 58.10810811
307
+ 297 58.24915825
308
+ 298 58.38926174
309
+ 299 58.52842809
310
+ 300 58.33333333
311
+ 301 58.13953488
312
+ 302 58.27814570
313
+ 303 58.41584158
314
+ 304 58.55263158
315
+ 305 58.68852459
316
+ 306 58.49673203
317
+ 307 58.30618893
318
+ 308 58.44155844
319
+ 309 58.57605178
320
+ 310 58.38709677
321
+ 311 58.52090032
322
+ 312 58.33333333
323
+ 313 58.46645367
324
+ 314 58.28025478
325
+ 315 58.41269841
326
+ 316 58.54430380
327
+ 317 58.67507886
328
+ 318 58.49056604
329
+ 319 58.30721003
330
+ 320 58.12500000
331
+ 321 57.94392523
332
+ 322 58.07453416
333
+ 323 58.20433437
334
+ 324 58.02469136
335
+ 325 57.84615385
336
+ 326 57.97546012
337
+ 327 57.79816514
338
+ 328 57.92682927
339
+ 329 58.05471125
340
+ 330 58.18181818
341
+ 331 58.30815710
342
+ 332 58.43373494
343
+ 333 58.55855856
344
+ 334 58.68263473
345
+ 335 58.50746269
346
+ 336 58.63095238
347
+ 337 58.45697329
348
+ 338 58.28402367
349
+ 339 58.11209440
350
+ 340 58.23529412
351
+ 341 58.35777126
352
+ 342 58.47953216
353
+ 343 58.60058309
354
+ 344 58.72093023
355
+ 345 58.55072464
356
+ 346 58.67052023
357
+ 347 58.50144092
358
+ 348 58.33333333
359
+ 349 58.45272206
360
+ 350 58.28571429
361
+ 351 58.40455840
362
+ 352 58.52272727
363
+ 353 58.64022663
364
+ 354 58.47457627
365
+ 355 58.30985915
366
+ 356 58.14606742
367
+ 357 57.98319328
368
+ 358 57.82122905
369
+ 359 57.66016713
370
+ 360 57.77777778
371
+ 361 57.89473684
372
+ 362 58.01104972
373
+ 363 58.12672176
374
+ 364 57.96703297
375
+ 365 57.80821918
376
+ 366 57.65027322
377
+ 367 57.76566757
378
+ 368 57.88043478
379
+ 369 57.99457995
380
+ 370 58.10810811
381
+ 371 57.95148248
382
+ 372 57.79569892
383
+ 373 57.64075067
384
+ 374 57.75401070
385
+ 375 57.86666667
386
+ 376 57.71276596
387
+ 377 57.55968170
388
+ 378 57.67195767
389
+ 379 57.78364116
390
+ 380 57.89473684
391
+ 381 57.74278215
392
+ 382 57.59162304
393
+ 383 57.44125326
394
+ 384 57.55208333
395
+ 385 57.66233766
396
+ 386 57.77202073
397
+ 387 57.62273902
398
+ 388 57.47422680
399
+ 389 57.32647815
400
+ 390 57.17948718
401
+ 391 57.03324808
402
+ 392 57.14285714
403
+ 393 56.99745547
404
+ 394 57.10659898
405
+ 395 57.21518987
406
+ 396 57.07070707
407
+ 397 57.17884131
408
+ 398 57.03517588
409
+ 399 57.14285714
410
+ 400 57.25000000
411
+ 401 57.35660848
412
+ 402 57.21393035
413
+ 403 57.32009926
414
+ 404 57.42574257
415
+ 405 57.53086420
416
+ 406 57.63546798
417
+ 407 57.73955774
418
+ 408 57.84313725
419
+ 409 57.70171149
420
+ 410 57.80487805
421
+ 411 57.90754258
422
+ 412 58.00970874
423
+ 413 57.86924939
424
+ 414 57.97101449
425
+ 415 57.83132530
426
+ 416 57.69230769
427
+ 417 57.55395683
428
+ 418 57.41626794
429
+ 419 57.27923628
430
+ 420 57.14285714
431
+ 421 57.00712589
432
+ 422 57.10900474
433
+ 423 57.21040189
434
+ 424 57.07547170
435
+ 425 57.17647059
436
+ 426 57.04225352
437
+ 427 57.14285714
438
+ 428 57.00934579
439
+ 429 57.10955711
440
+ 430 56.97674419
441
+ 431 57.07656613
442
+ 432 57.17592593
443
+ 433 57.04387991
444
+ 434 57.14285714
445
+ 435 57.01149425
446
+ 436 57.11009174
447
+ 437 56.97940503
448
+ 438 56.84931507
449
+ 439 56.94760820
450
+ 440 56.81818182
451
+ 441 56.91609977
452
+ 442 56.78733032
453
+ 443 56.65914221
454
+ 444 56.75675676
455
+ 445 56.85393258
456
+ 446 56.95067265
457
+ 447 57.04697987
458
+ 448 56.91964286
459
+ 449 57.01559020
460
+ 450 56.88888889
461
+ 451 56.98447894
462
+ 452 57.07964602
463
+ 453 56.95364238
464
+ 454 56.82819383
465
+ 455 56.92307692
466
+ 456 56.79824561
467
+ 457 56.89277899
468
+ 458 56.98689956
469
+ 459 57.08061002
470
+ 460 56.95652174
471
+ 461 57.04989154
472
+ 462 57.14285714
473
+ 463 57.01943844
474
+ 464 56.89655172
475
+ 465 56.77419355
476
+ 466 56.86695279
477
+ 467 56.95931478
478
+ 468 57.05128205
479
+ 469 56.92963753
480
+ 470 57.02127660
481
+ 471 57.11252654
482
+ 472 56.99152542
483
+ 473 56.87103594
484
+ 474 56.96202532
485
+ 475 56.84210526
486
+ 476 56.72268908
487
+ 477 56.81341719
488
+ 478 56.90376569
489
+ 479 56.99373695
490
+ 480 57.08333333
491
+ 481 56.96465696
492
+ 482 57.05394191
493
+ 483 57.14285714
494
+ 484 57.23140496
495
+ 485 57.31958763
496
+ 486 57.40740741
497
+ 487 57.28952772
498
+ 488 57.37704918
499
+ 489 57.25971370
500
+ 490 57.14285714
501
+ 491 57.02647658
502
+ 492 57.11382114
503
+ 493 56.99797160
504
+ 494 57.08502024
505
+ 495 56.96969697
506
+ 496 57.05645161
507
+ 497 57.14285714
508
+ 498 57.22891566
509
+ 499 57.31462926
510
+ 500 57.40000000
511
+ 501 57.48502994
512
+ 502 57.56972112
513
+ 503 57.65407555
514
+ 504 57.73809524
515
+ 505 57.82178218
516
+ 506 57.70750988
517
+ 507 57.59368836
518
+ 508 57.48031496
519
+ 509 57.36738703
520
+ 510 57.45098039
521
+ 511 57.33855186
522
+ 512 57.42187500
523
+ 513 57.30994152
524
+ 514 57.39299611
525
+ 515 57.47572816
526
+ 516 57.36434109
527
+ 517 57.44680851
528
+ 518 57.52895753
529
+ 519 57.41811175
530
+ 520 57.30769231
531
+ 521 57.38963532
532
+ 522 57.47126437
533
+ 523 57.36137667
534
+ 524 57.25190840
535
+ 525 57.14285714
536
+ 526 57.03422053
537
+ 527 56.92599620
538
+ 528 56.81818182
539
+ 529 56.71077505
540
+ 530 56.79245283
541
+ 531 56.68549906
542
+ 532 56.76691729
543
+ 533 56.84803002
544
+ 534 56.74157303
545
+ 535 56.63551402
546
+ 536 56.71641791
547
+ 537 56.79702048
548
+ 538 56.69144981
549
+ 539 56.77179963
550
+ 540 56.66666667
551
+ 541 56.74676525
552
+ 542 56.64206642
553
+ 543 56.53775322
554
+ 544 56.43382353
555
+ 545 56.51376147
556
+ 546 56.41025641
557
+ 547 56.30712980
558
+ 548 56.38686131
559
+ 549 56.46630237
560
+ 550 56.54545455
561
+ 551 56.44283122
562
+ 552 56.52173913
563
+ 553 56.60036166
564
+ 554 56.67870036
565
+ 555 56.75675676
566
+ 556 56.65467626
567
+ 557 56.73249551
568
+ 558 56.63082437
569
+ 559 56.70840787
570
+ 560 56.78571429
571
+ 561 56.86274510
572
+ 562 56.93950178
573
+ 563 57.01598579
574
+ 564 56.91489362
575
+ 565 56.81415929
576
+ 566 56.89045936
577
+ 567 56.79012346
578
+ 568 56.86619718
579
+ 569 56.76625659
580
+ 570 56.66666667
581
+ 571 56.74255692
582
+ 572 56.64335664
583
+ 573 56.71902269
584
+ 574 56.79442509
585
+ 575 56.86956522
586
+ 576 56.94444444
587
+ 577 57.01906412
588
+ 578 57.09342561
589
+ 579 57.16753022
590
+ 580 57.06896552
591
+ 581 56.97074010
592
+ 582 56.87285223
593
+ 583 56.94682676
594
+ 584 57.02054795
595
+ 585 57.09401709
596
+ 586 56.99658703
597
+ 587 56.89948893
598
+ 588 56.80272109
599
+ 589 56.87606112
600
+ 590 56.94915254
601
+ 591 57.02199662
602
+ 592 56.92567568
603
+ 593 56.99831366
604
+ 594 57.07070707
605
+ 595 56.97478992
606
+ 596 56.87919463
607
+ 597 56.78391960
608
+ 598 56.68896321
609
+ 599 56.59432387
610
+ 600 56.50000000
611
+ 601 56.57237937
612
+ 602 56.47840532
613
+ 603 56.38474295
614
+ 604 56.29139073
615
+ 605 56.19834711
616
+ 606 56.27062706
617
+ 607 56.34266886
618
+ 608 56.41447368
619
+ 609 56.32183908
620
+ 610 56.39344262
621
+ 611 56.30114566
622
+ 612 56.20915033
623
+ 613 56.11745514
624
+ 614 56.02605863
625
+ 615 55.93495935
626
+ 616 55.84415584
627
+ 617 55.91572123
628
+ 618 55.82524272
629
+ 619 55.73505654
630
+ 620 55.80645161
631
+ 621 55.87761675
632
+ 622 55.78778135
633
+ 623 55.85874799
634
+ 624 55.76923077
635
+ 625 55.68000000
636
+ 626 55.75079872
637
+ 627 55.82137161
638
+ 628 55.73248408
639
+ 629 55.80286169
640
+ 630 55.87301587
641
+ 631 55.78446910
642
+ 632 55.69620253
643
+ 633 55.60821485
644
+ 634 55.67823344
645
+ 635 55.74803150
646
+ 636 55.81761006
647
+ 637 55.72998430
648
+ 638 55.79937304
649
+ 639 55.71205008
650
+ 640 55.62500000
651
+ 641 55.53822153
652
+ 642 55.45171340
653
+ 643 55.36547434
654
+ 644 55.43478261
655
+ 645 55.34883721
656
+ 646 55.41795666
657
+ 647 55.33230294
658
+ 648 55.24691358
659
+ 649 55.16178737
660
+ 650 55.07692308
661
+ 651 55.14592934
662
+ 652 55.06134969
663
+ 653 55.13016845
664
+ 654 55.19877676
665
+ 655 55.11450382
666
+ 656 55.18292683
667
+ 657 55.09893455
668
+ 658 55.01519757
669
+ 659 55.08345979
670
+ 660 55.15151515
671
+ 661 55.06807867
672
+ 662 55.13595166
673
+ 663 55.05279035
674
+ 664 55.12048193
675
+ 665 55.18796992
676
+ 666 55.25525526
677
+ 667 55.32233883
678
+ 668 55.23952096
679
+ 669 55.15695067
680
+ 670 55.22388060
681
+ 671 55.29061103
682
+ 672 55.20833333
683
+ 673 55.12630015
684
+ 674 55.04451039
685
+ 675 55.11111111
686
+ 676 55.17751479
687
+ 677 55.24372230
688
+ 678 55.30973451
689
+ 679 55.37555228
690
+ 680 55.44117647
691
+ 681 55.35976505
692
+ 682 55.27859238
693
+ 683 55.19765739
694
+ 684 55.26315789
695
+ 685 55.32846715
696
+ 686 55.24781341
697
+ 687 55.16739447
698
+ 688 55.23255814
699
+ 689 55.29753266
700
+ 690 55.36231884
701
+ 691 55.42691751
702
+ 692 55.49132948
703
+ 693 55.55555556
704
+ 694 55.61959654
705
+ 695 55.53956835
706
+ 696 55.45977011
707
+ 697 55.52367288
708
+ 698 55.44412607
709
+ 699 55.36480687
710
+ 700 55.42857143
711
+ 701 55.49215407
712
+ 702 55.55555556
713
+ 703 55.47652916
714
+ 704 55.53977273
715
+ 705 55.60283688
716
+ 706 55.66572238
717
+ 707 55.72842999
718
+ 708 55.79096045
719
+ 709 55.85331453
720
+ 710 55.77464789
721
+ 711 55.83684951
722
+ 712 55.89887640
723
+ 713 55.96072931
724
+ 714 55.88235294
725
+ 715 55.80419580
726
+ 716 55.86592179
727
+ 717 55.78800558
728
+ 718 55.71030641
729
+ 719 55.63282337
730
+ 720 55.69444444
731
+ 721 55.61719834
732
+ 722 55.67867036
733
+ 723 55.73997234
734
+ 724 55.80110497
735
+ 725 55.86206897
736
+ 726 55.78512397
737
+ 727 55.70839065
738
+ 728 55.76923077
739
+ 729 55.82990398
740
+ 730 55.89041096
741
+ 731 55.95075239
742
+ 732 56.01092896
743
+ 733 56.07094134
744
+ 734 56.13079019
745
+ 735 56.05442177
746
+ 736 56.11413043
747
+ 737 56.03799186
748
+ 738 55.96205962
749
+ 739 55.88633288
750
+ 740 55.81081081
751
+ 741 55.87044534
752
+ 742 55.92991914
753
+ 743 55.98923284
754
+ 744 55.91397849
755
+ 745 55.97315436
756
+ 746 55.89812332
757
+ 747 55.82329317
758
+ 748 55.74866310
759
+ 749 55.67423231
760
+ 750 55.73333333
761
+
762
+ Final result: 55.7333 +/- 1.8149
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-q3_k_s.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 75.00000000
15
+ 5 80.00000000
16
+ 6 83.33333333
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 77.77777778
20
+ 10 80.00000000
21
+ 11 81.81818182
22
+ 12 75.00000000
23
+ 13 76.92307692
24
+ 14 78.57142857
25
+ 15 80.00000000
26
+ 16 81.25000000
27
+ 17 76.47058824
28
+ 18 72.22222222
29
+ 19 68.42105263
30
+ 20 70.00000000
31
+ 21 66.66666667
32
+ 22 68.18181818
33
+ 23 69.56521739
34
+ 24 66.66666667
35
+ 25 64.00000000
36
+ 26 61.53846154
37
+ 27 59.25925926
38
+ 28 60.71428571
39
+ 29 62.06896552
40
+ 30 60.00000000
41
+ 31 61.29032258
42
+ 32 59.37500000
43
+ 33 60.60606061
44
+ 34 61.76470588
45
+ 35 62.85714286
46
+ 36 61.11111111
47
+ 37 59.45945946
48
+ 38 60.52631579
49
+ 39 61.53846154
50
+ 40 62.50000000
51
+ 41 63.41463415
52
+ 42 61.90476190
53
+ 43 60.46511628
54
+ 44 61.36363636
55
+ 45 60.00000000
56
+ 46 58.69565217
57
+ 47 57.44680851
58
+ 48 56.25000000
59
+ 49 57.14285714
60
+ 50 56.00000000
61
+ 51 56.86274510
62
+ 52 55.76923077
63
+ 53 54.71698113
64
+ 54 53.70370370
65
+ 55 52.72727273
66
+ 56 51.78571429
67
+ 57 52.63157895
68
+ 58 51.72413793
69
+ 59 52.54237288
70
+ 60 53.33333333
71
+ 61 52.45901639
72
+ 62 51.61290323
73
+ 63 52.38095238
74
+ 64 53.12500000
75
+ 65 53.84615385
76
+ 66 53.03030303
77
+ 67 53.73134328
78
+ 68 54.41176471
79
+ 69 55.07246377
80
+ 70 55.71428571
81
+ 71 54.92957746
82
+ 72 55.55555556
83
+ 73 56.16438356
84
+ 74 56.75675676
85
+ 75 56.00000000
86
+ 76 56.57894737
87
+ 77 57.14285714
88
+ 78 57.69230769
89
+ 79 58.22784810
90
+ 80 58.75000000
91
+ 81 59.25925926
92
+ 82 59.75609756
93
+ 83 60.24096386
94
+ 84 60.71428571
95
+ 85 60.00000000
96
+ 86 60.46511628
97
+ 87 60.91954023
98
+ 88 61.36363636
99
+ 89 60.67415730
100
+ 90 61.11111111
101
+ 91 61.53846154
102
+ 92 60.86956522
103
+ 93 61.29032258
104
+ 94 61.70212766
105
+ 95 62.10526316
106
+ 96 62.50000000
107
+ 97 62.88659794
108
+ 98 63.26530612
109
+ 99 63.63636364
110
+ 100 63.00000000
111
+ 101 63.36633663
112
+ 102 62.74509804
113
+ 103 62.13592233
114
+ 104 62.50000000
115
+ 105 62.85714286
116
+ 106 63.20754717
117
+ 107 63.55140187
118
+ 108 63.88888889
119
+ 109 63.30275229
120
+ 110 63.63636364
121
+ 111 63.96396396
122
+ 112 64.28571429
123
+ 113 64.60176991
124
+ 114 64.91228070
125
+ 115 65.21739130
126
+ 116 64.65517241
127
+ 117 64.10256410
128
+ 118 63.55932203
129
+ 119 63.86554622
130
+ 120 64.16666667
131
+ 121 63.63636364
132
+ 122 63.93442623
133
+ 123 64.22764228
134
+ 124 63.70967742
135
+ 125 64.00000000
136
+ 126 64.28571429
137
+ 127 64.56692913
138
+ 128 64.06250000
139
+ 129 64.34108527
140
+ 130 63.84615385
141
+ 131 64.12213740
142
+ 132 63.63636364
143
+ 133 63.90977444
144
+ 134 64.17910448
145
+ 135 64.44444444
146
+ 136 64.70588235
147
+ 137 64.23357664
148
+ 138 64.49275362
149
+ 139 64.02877698
150
+ 140 64.28571429
151
+ 141 64.53900709
152
+ 142 64.78873239
153
+ 143 65.03496503
154
+ 144 65.27777778
155
+ 145 65.51724138
156
+ 146 65.75342466
157
+ 147 65.98639456
158
+ 148 66.21621622
159
+ 149 66.44295302
160
+ 150 66.66666667
161
+ 151 66.22516556
162
+ 152 65.78947368
163
+ 153 66.01307190
164
+ 154 66.23376623
165
+ 155 66.45161290
166
+ 156 66.02564103
167
+ 157 65.60509554
168
+ 158 65.82278481
169
+ 159 65.40880503
170
+ 160 65.62500000
171
+ 161 65.83850932
172
+ 162 66.04938272
173
+ 163 66.25766871
174
+ 164 66.46341463
175
+ 165 66.66666667
176
+ 166 66.86746988
177
+ 167 67.06586826
178
+ 168 67.26190476
179
+ 169 66.86390533
180
+ 170 67.05882353
181
+ 171 67.25146199
182
+ 172 66.86046512
183
+ 173 66.47398844
184
+ 174 66.09195402
185
+ 175 66.28571429
186
+ 176 66.47727273
187
+ 177 66.66666667
188
+ 178 66.85393258
189
+ 179 66.48044693
190
+ 180 66.66666667
191
+ 181 66.29834254
192
+ 182 66.48351648
193
+ 183 66.12021858
194
+ 184 65.76086957
195
+ 185 65.94594595
196
+ 186 65.59139785
197
+ 187 65.24064171
198
+ 188 64.89361702
199
+ 189 64.55026455
200
+ 190 64.73684211
201
+ 191 64.92146597
202
+ 192 65.10416667
203
+ 193 64.76683938
204
+ 194 64.94845361
205
+ 195 64.61538462
206
+ 196 64.28571429
207
+ 197 63.95939086
208
+ 198 63.63636364
209
+ 199 63.81909548
210
+ 200 64.00000000
211
+ 201 64.17910448
212
+ 202 64.35643564
213
+ 203 64.03940887
214
+ 204 64.21568627
215
+ 205 63.90243902
216
+ 206 63.59223301
217
+ 207 63.76811594
218
+ 208 63.46153846
219
+ 209 63.63636364
220
+ 210 63.80952381
221
+ 211 63.98104265
222
+ 212 64.15094340
223
+ 213 64.31924883
224
+ 214 64.48598131
225
+ 215 64.18604651
226
+ 216 64.35185185
227
+ 217 64.05529954
228
+ 218 63.76146789
229
+ 219 63.92694064
230
+ 220 64.09090909
231
+ 221 63.80090498
232
+ 222 63.96396396
233
+ 223 64.12556054
234
+ 224 64.28571429
235
+ 225 64.44444444
236
+ 226 64.60176991
237
+ 227 64.31718062
238
+ 228 64.47368421
239
+ 229 64.62882096
240
+ 230 64.34782609
241
+ 231 64.06926407
242
+ 232 63.79310345
243
+ 233 63.51931330
244
+ 234 63.67521368
245
+ 235 63.40425532
246
+ 236 63.13559322
247
+ 237 63.29113924
248
+ 238 63.44537815
249
+ 239 63.59832636
250
+ 240 63.75000000
251
+ 241 63.90041494
252
+ 242 64.04958678
253
+ 243 64.19753086
254
+ 244 63.93442623
255
+ 245 63.67346939
256
+ 246 63.82113821
257
+ 247 63.96761134
258
+ 248 64.11290323
259
+ 249 64.25702811
260
+ 250 64.40000000
261
+ 251 64.54183267
262
+ 252 64.68253968
263
+ 253 64.42687747
264
+ 254 64.56692913
265
+ 255 64.70588235
266
+ 256 64.84375000
267
+ 257 64.59143969
268
+ 258 64.72868217
269
+ 259 64.86486486
270
+ 260 65.00000000
271
+ 261 65.13409962
272
+ 262 65.26717557
273
+ 263 65.39923954
274
+ 264 65.53030303
275
+ 265 65.66037736
276
+ 266 65.41353383
277
+ 267 65.54307116
278
+ 268 65.67164179
279
+ 269 65.79925651
280
+ 270 65.92592593
281
+ 271 65.68265683
282
+ 272 65.80882353
283
+ 273 65.56776557
284
+ 274 65.69343066
285
+ 275 65.45454545
286
+ 276 65.57971014
287
+ 277 65.70397112
288
+ 278 65.82733813
289
+ 279 65.94982079
290
+ 280 66.07142857
291
+ 281 66.19217082
292
+ 282 66.31205674
293
+ 283 66.07773852
294
+ 284 66.19718310
295
+ 285 65.96491228
296
+ 286 66.08391608
297
+ 287 66.20209059
298
+ 288 66.31944444
299
+ 289 66.43598616
300
+ 290 66.55172414
301
+ 291 66.66666667
302
+ 292 66.43835616
303
+ 293 66.55290102
304
+ 294 66.66666667
305
+ 295 66.77966102
306
+ 296 66.89189189
307
+ 297 67.00336700
308
+ 298 67.11409396
309
+ 299 66.88963211
310
+ 300 66.66666667
311
+ 301 66.44518272
312
+ 302 66.55629139
313
+ 303 66.33663366
314
+ 304 66.11842105
315
+ 305 66.22950820
316
+ 306 66.33986928
317
+ 307 66.12377850
318
+ 308 66.23376623
319
+ 309 66.34304207
320
+ 310 66.45161290
321
+ 311 66.55948553
322
+ 312 66.66666667
323
+ 313 66.77316294
324
+ 314 66.87898089
325
+ 315 66.98412698
326
+ 316 67.08860759
327
+ 317 67.19242902
328
+ 318 66.98113208
329
+ 319 66.77115987
330
+ 320 66.87500000
331
+ 321 66.66666667
332
+ 322 66.45962733
333
+ 323 66.56346749
334
+ 324 66.66666667
335
+ 325 66.76923077
336
+ 326 66.56441718
337
+ 327 66.36085627
338
+ 328 66.46341463
339
+ 329 66.56534954
340
+ 330 66.66666667
341
+ 331 66.76737160
342
+ 332 66.56626506
343
+ 333 66.66666667
344
+ 334 66.46706587
345
+ 335 66.56716418
346
+ 336 66.66666667
347
+ 337 66.46884273
348
+ 338 66.27218935
349
+ 339 66.37168142
350
+ 340 66.47058824
351
+ 341 66.56891496
352
+ 342 66.66666667
353
+ 343 66.76384840
354
+ 344 66.56976744
355
+ 345 66.66666667
356
+ 346 66.47398844
357
+ 347 66.57060519
358
+ 348 66.37931034
359
+ 349 66.18911175
360
+ 350 66.28571429
361
+ 351 66.38176638
362
+ 352 66.47727273
363
+ 353 66.28895184
364
+ 354 66.38418079
365
+ 355 66.47887324
366
+ 356 66.57303371
367
+ 357 66.66666667
368
+ 358 66.75977654
369
+ 359 66.85236769
370
+ 360 66.66666667
371
+ 361 66.75900277
372
+ 362 66.85082873
373
+ 363 66.94214876
374
+ 364 67.03296703
375
+ 365 67.12328767
376
+ 366 67.21311475
377
+ 367 67.30245232
378
+ 368 67.39130435
379
+ 369 67.47967480
380
+ 370 67.29729730
381
+ 371 67.38544474
382
+ 372 67.47311828
383
+ 373 67.29222520
384
+ 374 67.37967914
385
+ 375 67.46666667
386
+ 376 67.55319149
387
+ 377 67.37400531
388
+ 378 67.46031746
389
+ 379 67.54617414
390
+ 380 67.63157895
391
+ 381 67.71653543
392
+ 382 67.53926702
393
+ 383 67.62402089
394
+ 384 67.70833333
395
+ 385 67.79220779
396
+ 386 67.87564767
397
+ 387 67.70025840
398
+ 388 67.52577320
399
+ 389 67.60925450
400
+ 390 67.69230769
401
+ 391 67.51918159
402
+ 392 67.60204082
403
+ 393 67.68447837
404
+ 394 67.76649746
405
+ 395 67.84810127
406
+ 396 67.92929293
407
+ 397 68.01007557
408
+ 398 68.09045226
409
+ 399 68.17042607
410
+ 400 68.00000000
411
+ 401 68.07980050
412
+ 402 67.91044776
413
+ 403 67.74193548
414
+ 404 67.82178218
415
+ 405 67.65432099
416
+ 406 67.73399015
417
+ 407 67.81326781
418
+ 408 67.89215686
419
+ 409 67.97066015
420
+ 410 67.80487805
421
+ 411 67.63990268
422
+ 412 67.47572816
423
+ 413 67.55447942
424
+ 414 67.63285024
425
+ 415 67.46987952
426
+ 416 67.54807692
427
+ 417 67.62589928
428
+ 418 67.70334928
429
+ 419 67.78042959
430
+ 420 67.85714286
431
+ 421 67.69596200
432
+ 422 67.77251185
433
+ 423 67.84869976
434
+ 424 67.92452830
435
+ 425 68.00000000
436
+ 426 68.07511737
437
+ 427 67.91569087
438
+ 428 67.99065421
439
+ 429 67.83216783
440
+ 430 67.90697674
441
+ 431 67.98143852
442
+ 432 68.05555556
443
+ 433 68.12933025
444
+ 434 68.20276498
445
+ 435 68.04597701
446
+ 436 68.11926606
447
+ 437 67.96338673
448
+ 438 67.80821918
449
+ 439 67.88154897
450
+ 440 67.95454545
451
+ 441 68.02721088
452
+ 442 68.09954751
453
+ 443 68.17155756
454
+ 444 68.24324324
455
+ 445 68.31460674
456
+ 446 68.38565022
457
+ 447 68.45637584
458
+ 448 68.52678571
459
+ 449 68.37416481
460
+ 450 68.44444444
461
+ 451 68.29268293
462
+ 452 68.36283186
463
+ 453 68.21192053
464
+ 454 68.28193833
465
+ 455 68.35164835
466
+ 456 68.42105263
467
+ 457 68.49015317
468
+ 458 68.55895197
469
+ 459 68.62745098
470
+ 460 68.69565217
471
+ 461 68.76355748
472
+ 462 68.83116883
473
+ 463 68.89848812
474
+ 464 68.96551724
475
+ 465 68.81720430
476
+ 466 68.88412017
477
+ 467 68.95074946
478
+ 468 68.80341880
479
+ 469 68.65671642
480
+ 470 68.51063830
481
+ 471 68.57749469
482
+ 472 68.43220339
483
+ 473 68.49894292
484
+ 474 68.56540084
485
+ 475 68.63157895
486
+ 476 68.48739496
487
+ 477 68.34381551
488
+ 478 68.41004184
489
+ 479 68.47599165
490
+ 480 68.54166667
491
+ 481 68.39916840
492
+ 482 68.25726141
493
+ 483 68.32298137
494
+ 484 68.18181818
495
+ 485 68.04123711
496
+ 486 68.10699588
497
+ 487 68.17248460
498
+ 488 68.03278689
499
+ 489 68.09815951
500
+ 490 68.16326531
501
+ 491 68.22810591
502
+ 492 68.29268293
503
+ 493 68.35699797
504
+ 494 68.42105263
505
+ 495 68.48484848
506
+ 496 68.54838710
507
+ 497 68.61167002
508
+ 498 68.47389558
509
+ 499 68.33667335
510
+ 500 68.40000000
511
+ 501 68.46307385
512
+ 502 68.52589641
513
+ 503 68.58846918
514
+ 504 68.65079365
515
+ 505 68.71287129
516
+ 506 68.77470356
517
+ 507 68.63905325
518
+ 508 68.50393701
519
+ 509 68.56581532
520
+ 510 68.62745098
521
+ 511 68.68884540
522
+ 512 68.55468750
523
+ 513 68.42105263
524
+ 514 68.48249027
525
+ 515 68.54368932
526
+ 516 68.41085271
527
+ 517 68.27852998
528
+ 518 68.33976834
529
+ 519 68.20809249
530
+ 520 68.26923077
531
+ 521 68.13819578
532
+ 522 68.00766284
533
+ 523 68.06883365
534
+ 524 68.12977099
535
+ 525 68.19047619
536
+ 526 68.06083650
537
+ 527 67.93168880
538
+ 528 67.99242424
539
+ 529 67.86389414
540
+ 530 67.92452830
541
+ 531 67.98493409
542
+ 532 68.04511278
543
+ 533 68.10506567
544
+ 534 68.16479401
545
+ 535 68.22429907
546
+ 536 68.28358209
547
+ 537 68.34264432
548
+ 538 68.40148699
549
+ 539 68.27458256
550
+ 540 68.33333333
551
+ 541 68.39186691
552
+ 542 68.45018450
553
+ 543 68.50828729
554
+ 544 68.56617647
555
+ 545 68.62385321
556
+ 546 68.68131868
557
+ 547 68.73857404
558
+ 548 68.79562044
559
+ 549 68.85245902
560
+ 550 68.72727273
561
+ 551 68.78402904
562
+ 552 68.65942029
563
+ 553 68.71609403
564
+ 554 68.77256318
565
+ 555 68.64864865
566
+ 556 68.70503597
567
+ 557 68.58168761
568
+ 558 68.45878136
569
+ 559 68.33631485
570
+ 560 68.39285714
571
+ 561 68.44919786
572
+ 562 68.32740214
573
+ 563 68.20603908
574
+ 564 68.08510638
575
+ 565 67.96460177
576
+ 566 67.84452297
577
+ 567 67.72486772
578
+ 568 67.78169014
579
+ 569 67.66256591
580
+ 570 67.71929825
581
+ 571 67.77583187
582
+ 572 67.83216783
583
+ 573 67.88830716
584
+ 574 67.94425087
585
+ 575 68.00000000
586
+ 576 68.05555556
587
+ 577 67.93760832
588
+ 578 67.99307958
589
+ 579 68.04835924
590
+ 580 67.93103448
591
+ 581 67.98623064
592
+ 582 68.04123711
593
+ 583 68.09605489
594
+ 584 68.15068493
595
+ 585 68.20512821
596
+ 586 68.08873720
597
+ 587 68.14310051
598
+ 588 68.02721088
599
+ 589 67.91171477
600
+ 590 67.79661017
601
+ 591 67.85109983
602
+ 592 67.90540541
603
+ 593 67.79089376
604
+ 594 67.67676768
605
+ 595 67.73109244
606
+ 596 67.78523490
607
+ 597 67.83919598
608
+ 598 67.89297659
609
+ 599 67.77963272
610
+ 600 67.83333333
611
+ 601 67.88685524
612
+ 602 67.94019934
613
+ 603 67.82752902
614
+ 604 67.88079470
615
+ 605 67.93388430
616
+ 606 67.82178218
617
+ 607 67.87479407
618
+ 608 67.76315789
619
+ 609 67.81609195
620
+ 610 67.86885246
621
+ 611 67.92144026
622
+ 612 67.81045752
623
+ 613 67.86296900
624
+ 614 67.75244300
625
+ 615 67.80487805
626
+ 616 67.69480519
627
+ 617 67.58508914
628
+ 618 67.63754045
629
+ 619 67.52827141
630
+ 620 67.58064516
631
+ 621 67.47181965
632
+ 622 67.52411576
633
+ 623 67.57624398
634
+ 624 67.62820513
635
+ 625 67.68000000
636
+ 626 67.57188498
637
+ 627 67.62360447
638
+ 628 67.67515924
639
+ 629 67.56756757
640
+ 630 67.61904762
641
+ 631 67.67036450
642
+ 632 67.56329114
643
+ 633 67.61453397
644
+ 634 67.66561514
645
+ 635 67.71653543
646
+ 636 67.76729560
647
+ 637 67.81789639
648
+ 638 67.86833856
649
+ 639 67.91862285
650
+ 640 67.81250000
651
+ 641 67.86271451
652
+ 642 67.75700935
653
+ 643 67.80715397
654
+ 644 67.85714286
655
+ 645 67.75193798
656
+ 646 67.80185759
657
+ 647 67.85162287
658
+ 648 67.90123457
659
+ 649 67.79661017
660
+ 650 67.84615385
661
+ 651 67.89554531
662
+ 652 67.94478528
663
+ 653 67.99387443
664
+ 654 68.04281346
665
+ 655 68.09160305
666
+ 656 68.14024390
667
+ 657 68.18873668
668
+ 658 68.23708207
669
+ 659 68.13353566
670
+ 660 68.03030303
671
+ 661 68.07866868
672
+ 662 68.12688822
673
+ 663 68.17496229
674
+ 664 68.22289157
675
+ 665 68.27067669
676
+ 666 68.31831832
677
+ 667 68.36581709
678
+ 668 68.41317365
679
+ 669 68.46038864
680
+ 670 68.50746269
681
+ 671 68.55439642
682
+ 672 68.60119048
683
+ 673 68.64784547
684
+ 674 68.54599407
685
+ 675 68.59259259
686
+ 676 68.49112426
687
+ 677 68.53766617
688
+ 678 68.58407080
689
+ 679 68.63033873
690
+ 680 68.67647059
691
+ 681 68.57562408
692
+ 682 68.62170088
693
+ 683 68.66764275
694
+ 684 68.71345029
695
+ 685 68.61313869
696
+ 686 68.65889213
697
+ 687 68.70451237
698
+ 688 68.75000000
699
+ 689 68.79535559
700
+ 690 68.84057971
701
+ 691 68.88567294
702
+ 692 68.93063584
703
+ 693 68.97546898
704
+ 694 69.02017291
705
+ 695 69.06474820
706
+ 696 69.10919540
707
+ 697 69.01004304
708
+ 698 69.05444126
709
+ 699 68.95565093
710
+ 700 69.00000000
711
+ 701 69.04422254
712
+ 702 69.08831909
713
+ 703 69.13229018
714
+ 704 69.17613636
715
+ 705 69.21985816
716
+ 706 69.26345609
717
+ 707 69.30693069
718
+ 708 69.35028249
719
+ 709 69.39351199
720
+ 710 69.43661972
721
+ 711 69.33895921
722
+ 712 69.38202247
723
+ 713 69.42496494
724
+ 714 69.46778711
725
+ 715 69.51048951
726
+ 716 69.55307263
727
+ 717 69.59553696
728
+ 718 69.63788301
729
+ 719 69.54102921
730
+ 720 69.58333333
731
+ 721 69.48682386
732
+ 722 69.52908587
733
+ 723 69.57123098
734
+ 724 69.61325967
735
+ 725 69.65517241
736
+ 726 69.69696970
737
+ 727 69.73865199
738
+ 728 69.78021978
739
+ 729 69.82167353
740
+ 730 69.72602740
741
+ 731 69.76744186
742
+ 732 69.67213115
743
+ 733 69.71350614
744
+ 734 69.75476839
745
+ 735 69.79591837
746
+ 736 69.83695652
747
+ 737 69.74219810
748
+ 738 69.64769648
749
+ 739 69.55345061
750
+ 740 69.59459459
751
+ 741 69.63562753
752
+ 742 69.67654987
753
+ 743 69.71736205
754
+ 744 69.75806452
755
+ 745 69.66442953
756
+ 746 69.70509383
757
+ 747 69.61178046
758
+ 748 69.65240642
759
+ 749 69.69292390
760
+ 750 69.73333333
scores/Hammer2.1-7b-q3_k_s.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 0.00000000
13
+ 3 0.00000000
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 28.57142857
18
+ 8 37.50000000
19
+ 9 44.44444444
20
+ 10 40.00000000
21
+ 11 36.36363636
22
+ 12 41.66666667
23
+ 13 46.15384615
24
+ 14 42.85714286
25
+ 15 46.66666667
26
+ 16 50.00000000
27
+ 17 52.94117647
28
+ 18 50.00000000
29
+ 19 52.63157895
30
+ 20 50.00000000
31
+ 21 47.61904762
32
+ 22 45.45454545
33
+ 23 43.47826087
34
+ 24 41.66666667
35
+ 25 44.00000000
36
+ 26 42.30769231
37
+ 27 44.44444444
38
+ 28 42.85714286
39
+ 29 44.82758621
40
+ 30 46.66666667
41
+ 31 48.38709677
42
+ 32 50.00000000
43
+ 33 48.48484848
44
+ 34 50.00000000
45
+ 35 51.42857143
46
+ 36 50.00000000
47
+ 37 48.64864865
48
+ 38 47.36842105
49
+ 39 46.15384615
50
+ 40 45.00000000
51
+ 41 43.90243902
52
+ 42 42.85714286
53
+ 43 41.86046512
54
+ 44 40.90909091
55
+ 45 42.22222222
56
+ 46 41.30434783
57
+ 47 40.42553191
58
+ 48 41.66666667
59
+ 49 42.85714286
60
+ 50 42.00000000
61
+ 51 41.17647059
62
+ 52 40.38461538
63
+ 53 39.62264151
64
+ 54 38.88888889
65
+ 55 40.00000000
66
+ 56 41.07142857
67
+ 57 42.10526316
68
+ 58 43.10344828
69
+ 59 44.06779661
70
+ 60 43.33333333
71
+ 61 42.62295082
72
+ 62 41.93548387
73
+ 63 41.26984127
74
+ 64 40.62500000
75
+ 65 40.00000000
76
+ 66 40.90909091
77
+ 67 40.29850746
78
+ 68 39.70588235
79
+ 69 39.13043478
80
+ 70 40.00000000
81
+ 71 40.84507042
82
+ 72 40.27777778
83
+ 73 39.72602740
84
+ 74 40.54054054
85
+ 75 41.33333333
86
+ 76 42.10526316
87
+ 77 42.85714286
88
+ 78 43.58974359
89
+ 79 43.03797468
90
+ 80 43.75000000
91
+ 81 44.44444444
92
+ 82 43.90243902
93
+ 83 44.57831325
94
+ 84 45.23809524
95
+ 85 45.88235294
96
+ 86 45.34883721
97
+ 87 45.97701149
98
+ 88 46.59090909
99
+ 89 46.06741573
100
+ 90 45.55555556
101
+ 91 45.05494505
102
+ 92 44.56521739
103
+ 93 44.08602151
104
+ 94 44.68085106
105
+ 95 45.26315789
106
+ 96 45.83333333
107
+ 97 45.36082474
108
+ 98 44.89795918
109
+ 99 45.45454545
110
+ 100 45.00000000
111
+ 101 45.54455446
112
+ 102 45.09803922
113
+ 103 44.66019417
114
+ 104 45.19230769
115
+ 105 45.71428571
116
+ 106 46.22641509
117
+ 107 45.79439252
118
+ 108 45.37037037
119
+ 109 44.95412844
120
+ 110 44.54545455
121
+ 111 44.14414414
122
+ 112 44.64285714
123
+ 113 44.24778761
124
+ 114 44.73684211
125
+ 115 45.21739130
126
+ 116 44.82758621
127
+ 117 44.44444444
128
+ 118 44.06779661
129
+ 119 44.53781513
130
+ 120 45.00000000
131
+ 121 45.45454545
132
+ 122 45.08196721
133
+ 123 44.71544715
134
+ 124 44.35483871
135
+ 125 44.80000000
136
+ 126 44.44444444
137
+ 127 44.09448819
138
+ 128 43.75000000
139
+ 129 43.41085271
140
+ 130 43.07692308
141
+ 131 43.51145038
142
+ 132 43.18181818
143
+ 133 42.85714286
144
+ 134 42.53731343
145
+ 135 42.22222222
146
+ 136 41.91176471
147
+ 137 41.60583942
148
+ 138 41.30434783
149
+ 139 41.72661871
150
+ 140 42.14285714
151
+ 141 41.84397163
152
+ 142 42.25352113
153
+ 143 42.65734266
154
+ 144 42.36111111
155
+ 145 42.06896552
156
+ 146 42.46575342
157
+ 147 42.17687075
158
+ 148 42.56756757
159
+ 149 42.95302013
160
+ 150 42.66666667
161
+ 151 42.38410596
162
+ 152 42.76315789
163
+ 153 42.48366013
164
+ 154 42.20779221
165
+ 155 41.93548387
166
+ 156 41.66666667
167
+ 157 42.03821656
168
+ 158 42.40506329
169
+ 159 42.13836478
170
+ 160 42.50000000
171
+ 161 42.23602484
172
+ 162 41.97530864
173
+ 163 41.71779141
174
+ 164 41.46341463
175
+ 165 41.81818182
176
+ 166 41.56626506
177
+ 167 41.91616766
178
+ 168 41.66666667
179
+ 169 42.01183432
180
+ 170 41.76470588
181
+ 171 41.52046784
182
+ 172 41.27906977
183
+ 173 41.04046243
184
+ 174 41.37931034
185
+ 175 41.14285714
186
+ 176 40.90909091
187
+ 177 40.67796610
188
+ 178 40.44943820
189
+ 179 40.78212291
190
+ 180 40.55555556
191
+ 181 40.33149171
192
+ 182 40.10989011
193
+ 183 40.43715847
194
+ 184 40.21739130
195
+ 185 40.54054054
196
+ 186 40.86021505
197
+ 187 40.64171123
198
+ 188 40.42553191
199
+ 189 40.74074074
200
+ 190 40.52631579
201
+ 191 40.83769634
202
+ 192 40.62500000
203
+ 193 40.41450777
204
+ 194 40.20618557
205
+ 195 40.51282051
206
+ 196 40.30612245
207
+ 197 40.60913706
208
+ 198 40.90909091
209
+ 199 40.70351759
210
+ 200 40.50000000
211
+ 201 40.29850746
212
+ 202 40.09900990
213
+ 203 39.90147783
214
+ 204 39.70588235
215
+ 205 40.00000000
216
+ 206 39.80582524
217
+ 207 40.09661836
218
+ 208 39.90384615
219
+ 209 39.71291866
220
+ 210 39.52380952
221
+ 211 39.81042654
222
+ 212 39.62264151
223
+ 213 39.43661972
224
+ 214 39.25233645
225
+ 215 39.53488372
226
+ 216 39.81481481
227
+ 217 39.63133641
228
+ 218 39.90825688
229
+ 219 40.18264840
230
+ 220 40.45454545
231
+ 221 40.72398190
232
+ 222 40.99099099
233
+ 223 41.25560538
234
+ 224 41.51785714
235
+ 225 41.77777778
236
+ 226 42.03539823
237
+ 227 41.85022026
238
+ 228 42.10526316
239
+ 229 41.92139738
240
+ 230 41.73913043
241
+ 231 41.55844156
242
+ 232 41.37931034
243
+ 233 41.20171674
244
+ 234 41.45299145
245
+ 235 41.27659574
246
+ 236 41.10169492
247
+ 237 40.92827004
248
+ 238 40.75630252
249
+ 239 40.58577406
250
+ 240 40.41666667
251
+ 241 40.66390041
252
+ 242 40.90909091
253
+ 243 41.15226337
254
+ 244 40.98360656
255
+ 245 41.22448980
256
+ 246 41.05691057
257
+ 247 41.29554656
258
+ 248 41.53225806
259
+ 249 41.36546185
260
+ 250 41.20000000
261
+ 251 41.03585657
262
+ 252 40.87301587
263
+ 253 40.71146245
264
+ 254 40.55118110
265
+ 255 40.39215686
266
+ 256 40.62500000
267
+ 257 40.85603113
268
+ 258 40.69767442
269
+ 259 40.54054054
270
+ 260 40.38461538
271
+ 261 40.22988506
272
+ 262 40.07633588
273
+ 263 39.92395437
274
+ 264 40.15151515
275
+ 265 40.00000000
276
+ 266 40.22556391
277
+ 267 40.07490637
278
+ 268 39.92537313
279
+ 269 39.77695167
280
+ 270 40.00000000
281
+ 271 39.85239852
282
+ 272 39.70588235
283
+ 273 39.56043956
284
+ 274 39.41605839
285
+ 275 39.27272727
286
+ 276 39.49275362
287
+ 277 39.71119134
288
+ 278 39.92805755
289
+ 279 39.78494624
290
+ 280 40.00000000
291
+ 281 39.85765125
292
+ 282 39.71631206
293
+ 283 39.57597173
294
+ 284 39.43661972
295
+ 285 39.29824561
296
+ 286 39.16083916
297
+ 287 39.37282230
298
+ 288 39.23611111
299
+ 289 39.10034602
300
+ 290 39.31034483
301
+ 291 39.51890034
302
+ 292 39.72602740
303
+ 293 39.93174061
304
+ 294 39.79591837
305
+ 295 39.66101695
306
+ 296 39.52702703
307
+ 297 39.39393939
308
+ 298 39.59731544
309
+ 299 39.79933110
310
+ 300 40.00000000
311
+ 301 39.86710963
312
+ 302 39.73509934
313
+ 303 39.60396040
314
+ 304 39.47368421
315
+ 305 39.67213115
316
+ 306 39.54248366
317
+ 307 39.73941368
318
+ 308 39.61038961
319
+ 309 39.48220065
320
+ 310 39.67741935
321
+ 311 39.54983923
322
+ 312 39.74358974
323
+ 313 39.93610224
324
+ 314 40.12738854
325
+ 315 40.31746032
326
+ 316 40.18987342
327
+ 317 40.06309148
328
+ 318 39.93710692
329
+ 319 40.12539185
330
+ 320 40.00000000
331
+ 321 39.87538941
332
+ 322 39.75155280
333
+ 323 39.62848297
334
+ 324 39.50617284
335
+ 325 39.69230769
336
+ 326 39.57055215
337
+ 327 39.75535168
338
+ 328 39.63414634
339
+ 329 39.51367781
340
+ 330 39.39393939
341
+ 331 39.27492447
342
+ 332 39.15662651
343
+ 333 39.03903904
344
+ 334 38.92215569
345
+ 335 39.10447761
346
+ 336 39.28571429
347
+ 337 39.16913947
348
+ 338 39.05325444
349
+ 339 38.93805310
350
+ 340 39.11764706
351
+ 341 39.29618768
352
+ 342 39.47368421
353
+ 343 39.35860058
354
+ 344 39.24418605
355
+ 345 39.13043478
356
+ 346 39.30635838
357
+ 347 39.48126801
358
+ 348 39.36781609
359
+ 349 39.54154728
360
+ 350 39.71428571
361
+ 351 39.60113960
362
+ 352 39.48863636
363
+ 353 39.66005666
364
+ 354 39.54802260
365
+ 355 39.43661972
366
+ 356 39.32584270
367
+ 357 39.21568627
368
+ 358 39.10614525
369
+ 359 38.99721448
370
+ 360 39.16666667
371
+ 361 39.05817175
372
+ 362 39.22651934
373
+ 363 39.39393939
374
+ 364 39.28571429
375
+ 365 39.17808219
376
+ 366 39.34426230
377
+ 367 39.23705722
378
+ 368 39.13043478
379
+ 369 39.02439024
380
+ 370 39.18918919
381
+ 371 39.35309973
382
+ 372 39.24731183
383
+ 373 39.14209115
384
+ 374 39.30481283
385
+ 375 39.46666667
386
+ 376 39.36170213
387
+ 377 39.25729443
388
+ 378 39.41798942
389
+ 379 39.31398417
390
+ 380 39.21052632
391
+ 381 39.37007874
392
+ 382 39.52879581
393
+ 383 39.68668407
394
+ 384 39.58333333
395
+ 385 39.74025974
396
+ 386 39.63730570
397
+ 387 39.53488372
398
+ 388 39.43298969
399
+ 389 39.58868895
400
+ 390 39.48717949
401
+ 391 39.64194373
402
+ 392 39.79591837
403
+ 393 39.69465649
404
+ 394 39.59390863
405
+ 395 39.74683544
406
+ 396 39.89898990
407
+ 397 40.05037783
408
+ 398 39.94974874
409
+ 399 40.10025063
410
+ 400 40.25000000
411
+ 401 40.39900249
412
+ 402 40.29850746
413
+ 403 40.19851117
414
+ 404 40.09900990
415
+ 405 40.00000000
416
+ 406 39.90147783
417
+ 407 39.80343980
418
+ 408 39.70588235
419
+ 409 39.85330073
420
+ 410 39.75609756
421
+ 411 39.65936740
422
+ 412 39.56310680
423
+ 413 39.46731235
424
+ 414 39.37198068
425
+ 415 39.51807229
426
+ 416 39.42307692
427
+ 417 39.32853717
428
+ 418 39.23444976
429
+ 419 39.37947494
430
+ 420 39.52380952
431
+ 421 39.42992874
432
+ 422 39.33649289
433
+ 423 39.24349882
434
+ 424 39.38679245
435
+ 425 39.29411765
436
+ 426 39.43661972
437
+ 427 39.34426230
438
+ 428 39.25233645
439
+ 429 39.39393939
440
+ 430 39.30232558
441
+ 431 39.44315545
442
+ 432 39.58333333
443
+ 433 39.49191686
444
+ 434 39.40092166
445
+ 435 39.31034483
446
+ 436 39.22018349
447
+ 437 39.13043478
448
+ 438 39.26940639
449
+ 439 39.40774487
450
+ 440 39.31818182
451
+ 441 39.22902494
452
+ 442 39.14027149
453
+ 443 39.05191874
454
+ 444 38.96396396
455
+ 445 38.87640449
456
+ 446 38.78923767
457
+ 447 38.92617450
458
+ 448 38.83928571
459
+ 449 38.75278396
460
+ 450 38.88888889
461
+ 451 38.80266075
462
+ 452 38.93805310
463
+ 453 39.07284768
464
+ 454 38.98678414
465
+ 455 38.90109890
466
+ 456 39.03508772
467
+ 457 39.16849015
468
+ 458 39.08296943
469
+ 459 39.21568627
470
+ 460 39.34782609
471
+ 461 39.26247289
472
+ 462 39.17748918
473
+ 463 39.09287257
474
+ 464 39.22413793
475
+ 465 39.13978495
476
+ 466 39.05579399
477
+ 467 39.18629550
478
+ 468 39.10256410
479
+ 469 39.01918977
480
+ 470 38.93617021
481
+ 471 38.85350318
482
+ 472 38.77118644
483
+ 473 38.68921776
484
+ 474 38.60759494
485
+ 475 38.52631579
486
+ 476 38.65546218
487
+ 477 38.57442348
488
+ 478 38.70292887
489
+ 479 38.83089770
490
+ 480 38.95833333
491
+ 481 38.87733888
492
+ 482 38.79668050
493
+ 483 38.71635611
494
+ 484 38.63636364
495
+ 485 38.55670103
496
+ 486 38.47736626
497
+ 487 38.60369610
498
+ 488 38.52459016
499
+ 489 38.44580777
500
+ 490 38.57142857
501
+ 491 38.49287169
502
+ 492 38.41463415
503
+ 493 38.53955375
504
+ 494 38.66396761
505
+ 495 38.78787879
506
+ 496 38.91129032
507
+ 497 38.83299799
508
+ 498 38.75502008
509
+ 499 38.67735471
510
+ 500 38.60000000
511
+ 501 38.52295409
512
+ 502 38.44621514
513
+ 503 38.36978131
514
+ 504 38.49206349
515
+ 505 38.41584158
516
+ 506 38.53754941
517
+ 507 38.46153846
518
+ 508 38.58267717
519
+ 509 38.50687623
520
+ 510 38.62745098
521
+ 511 38.55185910
522
+ 512 38.47656250
523
+ 513 38.40155945
524
+ 514 38.32684825
525
+ 515 38.25242718
526
+ 516 38.37209302
527
+ 517 38.49129594
528
+ 518 38.61003861
529
+ 519 38.53564547
530
+ 520 38.65384615
531
+ 521 38.57965451
532
+ 522 38.50574713
533
+ 523 38.43212237
534
+ 524 38.54961832
535
+ 525 38.47619048
536
+ 526 38.40304183
537
+ 527 38.51992410
538
+ 528 38.63636364
539
+ 529 38.56332703
540
+ 530 38.67924528
541
+ 531 38.60640301
542
+ 532 38.53383459
543
+ 533 38.46153846
544
+ 534 38.38951311
545
+ 535 38.50467290
546
+ 536 38.43283582
547
+ 537 38.54748603
548
+ 538 38.66171004
549
+ 539 38.58998145
550
+ 540 38.51851852
551
+ 541 38.44731978
552
+ 542 38.56088561
553
+ 543 38.48987109
554
+ 544 38.41911765
555
+ 545 38.34862385
556
+ 546 38.27838828
557
+ 547 38.20840951
558
+ 548 38.32116788
559
+ 549 38.25136612
560
+ 550 38.18181818
561
+ 551 38.29401089
562
+ 552 38.22463768
563
+ 553 38.15551537
564
+ 554 38.08664260
565
+ 555 38.01801802
566
+ 556 38.12949640
567
+ 557 38.06104129
568
+ 558 38.17204301
569
+ 559 38.10375671
570
+ 560 38.03571429
571
+ 561 37.96791444
572
+ 562 38.07829181
573
+ 563 38.18827709
574
+ 564 38.12056738
575
+ 565 38.23008850
576
+ 566 38.16254417
577
+ 567 38.09523810
578
+ 568 38.02816901
579
+ 569 37.96133568
580
+ 570 37.89473684
581
+ 571 37.82837128
582
+ 572 37.93706294
583
+ 573 37.87085515
584
+ 574 37.97909408
585
+ 575 37.91304348
586
+ 576 38.02083333
587
+ 577 38.12824957
588
+ 578 38.06228374
589
+ 579 37.99654577
590
+ 580 38.10344828
591
+ 581 38.03786575
592
+ 582 37.97250859
593
+ 583 38.07890223
594
+ 584 38.18493151
595
+ 585 38.11965812
596
+ 586 38.05460751
597
+ 587 37.98977853
598
+ 588 37.92517007
599
+ 589 37.86078098
600
+ 590 37.79661017
601
+ 591 37.73265651
602
+ 592 37.66891892
603
+ 593 37.60539629
604
+ 594 37.71043771
605
+ 595 37.64705882
606
+ 596 37.75167785
607
+ 597 37.68844221
608
+ 598 37.62541806
609
+ 599 37.72954925
610
+ 600 37.66666667
611
+ 601 37.77038270
612
+ 602 37.70764120
613
+ 603 37.81094527
614
+ 604 37.74834437
615
+ 605 37.68595041
616
+ 606 37.62376238
617
+ 607 37.72652389
618
+ 608 37.66447368
619
+ 609 37.60262726
620
+ 610 37.54098361
621
+ 611 37.64320786
622
+ 612 37.58169935
623
+ 613 37.52039152
624
+ 614 37.45928339
625
+ 615 37.39837398
626
+ 616 37.50000000
627
+ 617 37.43922204
628
+ 618 37.37864078
629
+ 619 37.31825525
630
+ 620 37.25806452
631
+ 621 37.19806763
632
+ 622 37.13826367
633
+ 623 37.07865169
634
+ 624 37.01923077
635
+ 625 36.96000000
636
+ 626 37.06070288
637
+ 627 37.00159490
638
+ 628 36.94267516
639
+ 629 37.04292528
640
+ 630 36.98412698
641
+ 631 37.08399366
642
+ 632 37.18354430
643
+ 633 37.12480253
644
+ 634 37.06624606
645
+ 635 37.00787402
646
+ 636 36.94968553
647
+ 637 37.04866562
648
+ 638 36.99059561
649
+ 639 37.08920188
650
+ 640 37.18750000
651
+ 641 37.12948518
652
+ 642 37.22741433
653
+ 643 37.16951788
654
+ 644 37.26708075
655
+ 645 37.20930233
656
+ 646 37.15170279
657
+ 647 37.09428130
658
+ 648 37.03703704
659
+ 649 36.97996918
660
+ 650 37.07692308
661
+ 651 37.01996928
662
+ 652 37.11656442
663
+ 653 37.05972435
664
+ 654 37.15596330
665
+ 655 37.09923664
666
+ 656 37.04268293
667
+ 657 36.98630137
668
+ 658 37.08206687
669
+ 659 37.02579666
670
+ 660 36.96969697
671
+ 661 36.91376702
672
+ 662 36.85800604
673
+ 663 36.80241327
674
+ 664 36.89759036
675
+ 665 36.99248120
676
+ 666 36.93693694
677
+ 667 37.03148426
678
+ 668 37.12574850
679
+ 669 37.21973094
680
+ 670 37.31343284
681
+ 671 37.25782414
682
+ 672 37.20238095
683
+ 673 37.14710253
684
+ 674 37.09198813
685
+ 675 37.03703704
686
+ 676 36.98224852
687
+ 677 36.92762186
688
+ 678 36.87315634
689
+ 679 36.81885125
690
+ 680 36.91176471
691
+ 681 37.00440529
692
+ 682 36.95014663
693
+ 683 36.89604685
694
+ 684 36.84210526
695
+ 685 36.78832117
696
+ 686 36.73469388
697
+ 687 36.68122271
698
+ 688 36.62790698
699
+ 689 36.71988389
700
+ 690 36.66666667
701
+ 691 36.75832127
702
+ 692 36.70520231
703
+ 693 36.65223665
704
+ 694 36.59942363
705
+ 695 36.54676259
706
+ 696 36.63793103
707
+ 697 36.72883788
708
+ 698 36.67621777
709
+ 699 36.76680973
710
+ 700 36.71428571
711
+ 701 36.66191155
712
+ 702 36.60968661
713
+ 703 36.69985775
714
+ 704 36.64772727
715
+ 705 36.59574468
716
+ 706 36.54390935
717
+ 707 36.49222065
718
+ 708 36.44067797
719
+ 709 36.53032440
720
+ 710 36.61971831
721
+ 711 36.70886076
722
+ 712 36.65730337
723
+ 713 36.74614306
724
+ 714 36.83473389
725
+ 715 36.78321678
726
+ 716 36.87150838
727
+ 717 36.82008368
728
+ 718 36.76880223
729
+ 719 36.85674548
730
+ 720 36.94444444
731
+ 721 36.89320388
732
+ 722 36.98060942
733
+ 723 37.06777317
734
+ 724 37.01657459
735
+ 725 37.10344828
736
+ 726 37.05234160
737
+ 727 37.00137552
738
+ 728 36.95054945
739
+ 729 36.89986283
740
+ 730 36.84931507
741
+ 731 36.79890561
742
+ 732 36.88524590
743
+ 733 36.97135061
744
+ 734 37.05722071
745
+ 735 37.00680272
746
+ 736 36.95652174
747
+ 737 37.04206242
748
+ 738 36.99186992
749
+ 739 37.07713126
750
+ 740 37.02702703
751
+ 741 36.97705803
752
+ 742 37.06199461
753
+ 743 37.14670256
754
+ 744 37.23118280
755
+ 745 37.18120805
756
+ 746 37.13136729
757
+ 747 37.08165997
758
+ 748 37.03208556
759
+ 749 36.98264352
760
+ 750 37.06666667
761
+
762
+ Final result: 37.0667 +/- 1.7648
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-q3_k_s.ppx ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ====== Perplexity statistics ======
2
+ Mean PPL(Q) : 10.928250 ± 0.079514
3
+ Mean PPL(base) : 9.366577 ± 0.066397
4
+ Cor(ln(PPL(Q)), ln(PPL(base))): 97.35%
5
+ Mean ln(PPL(Q)/PPL(base)) : 0.154203 ± 0.001663
6
+ Mean PPL(Q)/PPL(base) : 1.166728 ± 0.001940
7
+ Mean PPL(Q)-PPL(base) : 1.561672 ± 0.021251
8
+
9
+ ====== KL divergence statistics ======
10
+ Mean KLD: 0.161692 ± 0.000497
11
+ Maximum KLD: 5.067591
12
+ 99.9% KLD: 1.801306
13
+ 99.0% KLD: 0.884262
14
+ 99.0% KLD: 0.884262
15
+ Median KLD: 0.116811
16
+ 10.0% KLD: 0.002901
17
+ 5.0% KLD: 0.000632
18
+ 1.0% KLD: 0.000051
19
+ Minimum KLD: -0.000015
20
+
21
+ ====== Token probability statistics ======
22
+ Mean Δp: -2.618 ± 0.027 %
23
+ Maximum Δp: 85.600%
24
+ 99.9% Δp: 43.032%
25
+ 99.0% Δp: 23.641%
26
+ 95.0% Δp: 10.359%
27
+ 90.0% Δp: 5.034%
28
+ 75.0% Δp: 0.301%
29
+ Median Δp: -0.242%
30
+ 25.0% Δp: -4.577%
31
+ 10.0% Δp: -13.964%
32
+ 5.0% Δp: -21.284%
33
+ 1.0% Δp: -39.535%
34
+ 0.1% Δp: -67.698%
35
+ Minimum Δp: -95.341%
36
+ RMS Δp : 10.577 ± 0.045 %
37
+ Same top p: 79.239 ± 0.105 %
scores/Hammer2.1-7b-q3_k_s.tqa ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 817 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 817 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 14.28571429
18
+ 8 12.50000000
19
+ 9 11.11111111
20
+ 10 10.00000000
21
+ 11 9.09090909
22
+ 12 16.66666667
23
+ 13 15.38461538
24
+ 14 14.28571429
25
+ 15 13.33333333
26
+ 16 12.50000000
27
+ 17 11.76470588
28
+ 18 11.11111111
29
+ 19 15.78947368
30
+ 20 15.00000000
31
+ 21 14.28571429
32
+ 22 13.63636364
33
+ 23 13.04347826
34
+ 24 16.66666667
35
+ 25 16.00000000
36
+ 26 15.38461538
37
+ 27 14.81481481
38
+ 28 14.28571429
39
+ 29 17.24137931
40
+ 30 16.66666667
41
+ 31 16.12903226
42
+ 32 15.62500000
43
+ 33 18.18181818
44
+ 34 17.64705882
45
+ 35 17.14285714
46
+ 36 16.66666667
47
+ 37 18.91891892
48
+ 38 18.42105263
49
+ 39 20.51282051
50
+ 40 20.00000000
51
+ 41 19.51219512
52
+ 42 19.04761905
53
+ 43 20.93023256
54
+ 44 20.45454545
55
+ 45 22.22222222
56
+ 46 23.91304348
57
+ 47 25.53191489
58
+ 48 27.08333333
59
+ 49 26.53061224
60
+ 50 28.00000000
61
+ 51 29.41176471
62
+ 52 28.84615385
63
+ 53 28.30188679
64
+ 54 29.62962963
65
+ 55 30.90909091
66
+ 56 30.35714286
67
+ 57 29.82456140
68
+ 58 29.31034483
69
+ 59 28.81355932
70
+ 60 28.33333333
71
+ 61 27.86885246
72
+ 62 29.03225806
73
+ 63 28.57142857
74
+ 64 29.68750000
75
+ 65 29.23076923
76
+ 66 28.78787879
77
+ 67 28.35820896
78
+ 68 27.94117647
79
+ 69 28.98550725
80
+ 70 30.00000000
81
+ 71 29.57746479
82
+ 72 30.55555556
83
+ 73 30.13698630
84
+ 74 29.72972973
85
+ 75 29.33333333
86
+ 76 28.94736842
87
+ 77 28.57142857
88
+ 78 28.20512821
89
+ 79 27.84810127
90
+ 80 28.75000000
91
+ 81 28.39506173
92
+ 82 29.26829268
93
+ 83 28.91566265
94
+ 84 28.57142857
95
+ 85 28.23529412
96
+ 86 27.90697674
97
+ 87 27.58620690
98
+ 88 27.27272727
99
+ 89 28.08988764
100
+ 90 27.77777778
101
+ 91 27.47252747
102
+ 92 27.17391304
103
+ 93 26.88172043
104
+ 94 26.59574468
105
+ 95 27.36842105
106
+ 96 28.12500000
107
+ 97 28.86597938
108
+ 98 28.57142857
109
+ 99 29.29292929
110
+ 100 30.00000000
111
+ 101 30.69306931
112
+ 102 30.39215686
113
+ 103 30.09708738
114
+ 104 29.80769231
115
+ 105 29.52380952
116
+ 106 29.24528302
117
+ 107 29.90654206
118
+ 108 30.55555556
119
+ 109 30.27522936
120
+ 110 30.90909091
121
+ 111 30.63063063
122
+ 112 30.35714286
123
+ 113 30.08849558
124
+ 114 30.70175439
125
+ 115 30.43478261
126
+ 116 30.17241379
127
+ 117 29.91452991
128
+ 118 29.66101695
129
+ 119 29.41176471
130
+ 120 30.00000000
131
+ 121 30.57851240
132
+ 122 30.32786885
133
+ 123 30.08130081
134
+ 124 30.64516129
135
+ 125 31.20000000
136
+ 126 30.95238095
137
+ 127 30.70866142
138
+ 128 30.46875000
139
+ 129 30.23255814
140
+ 130 30.00000000
141
+ 131 29.77099237
142
+ 132 29.54545455
143
+ 133 29.32330827
144
+ 134 29.85074627
145
+ 135 30.37037037
146
+ 136 30.14705882
147
+ 137 30.65693431
148
+ 138 31.15942029
149
+ 139 31.65467626
150
+ 140 31.42857143
151
+ 141 31.20567376
152
+ 142 30.98591549
153
+ 143 30.76923077
154
+ 144 31.25000000
155
+ 145 31.03448276
156
+ 146 30.82191781
157
+ 147 30.61224490
158
+ 148 30.40540541
159
+ 149 30.20134228
160
+ 150 30.00000000
161
+ 151 30.46357616
162
+ 152 30.26315789
163
+ 153 30.06535948
164
+ 154 29.87012987
165
+ 155 30.32258065
166
+ 156 30.76923077
167
+ 157 30.57324841
168
+ 158 31.01265823
169
+ 159 30.81761006
170
+ 160 30.62500000
171
+ 161 30.43478261
172
+ 162 30.24691358
173
+ 163 30.06134969
174
+ 164 29.87804878
175
+ 165 30.30303030
176
+ 166 30.72289157
177
+ 167 30.53892216
178
+ 168 30.35714286
179
+ 169 30.17751479
180
+ 170 30.00000000
181
+ 171 29.82456140
182
+ 172 29.65116279
183
+ 173 29.47976879
184
+ 174 29.88505747
185
+ 175 29.71428571
186
+ 176 29.54545455
187
+ 177 29.37853107
188
+ 178 29.21348315
189
+ 179 29.60893855
190
+ 180 29.44444444
191
+ 181 29.28176796
192
+ 182 29.12087912
193
+ 183 28.96174863
194
+ 184 28.80434783
195
+ 185 28.64864865
196
+ 186 28.49462366
197
+ 187 28.34224599
198
+ 188 28.72340426
199
+ 189 28.57142857
200
+ 190 28.42105263
201
+ 191 28.27225131
202
+ 192 28.64583333
203
+ 193 28.49740933
204
+ 194 28.86597938
205
+ 195 28.71794872
206
+ 196 28.57142857
207
+ 197 28.93401015
208
+ 198 29.29292929
209
+ 199 29.14572864
210
+ 200 29.00000000
211
+ 201 28.85572139
212
+ 202 29.20792079
213
+ 203 29.06403941
214
+ 204 28.92156863
215
+ 205 28.78048780
216
+ 206 28.64077670
217
+ 207 28.98550725
218
+ 208 28.84615385
219
+ 209 28.70813397
220
+ 210 28.57142857
221
+ 211 28.43601896
222
+ 212 28.30188679
223
+ 213 28.16901408
224
+ 214 28.03738318
225
+ 215 27.90697674
226
+ 216 28.24074074
227
+ 217 28.11059908
228
+ 218 28.44036697
229
+ 219 28.31050228
230
+ 220 28.18181818
231
+ 221 28.05429864
232
+ 222 27.92792793
233
+ 223 27.80269058
234
+ 224 28.12500000
235
+ 225 28.00000000
236
+ 226 27.87610619
237
+ 227 28.19383260
238
+ 228 28.07017544
239
+ 229 27.94759825
240
+ 230 27.82608696
241
+ 231 27.70562771
242
+ 232 28.01724138
243
+ 233 28.32618026
244
+ 234 28.20512821
245
+ 235 28.08510638
246
+ 236 28.38983051
247
+ 237 28.69198312
248
+ 238 28.99159664
249
+ 239 28.87029289
250
+ 240 28.75000000
251
+ 241 28.63070539
252
+ 242 28.51239669
253
+ 243 28.39506173
254
+ 244 28.27868852
255
+ 245 28.16326531
256
+ 246 28.04878049
257
+ 247 27.93522267
258
+ 248 27.82258065
259
+ 249 27.71084337
260
+ 250 27.60000000
261
+ 251 27.88844622
262
+ 252 27.77777778
263
+ 253 27.66798419
264
+ 254 27.55905512
265
+ 255 27.84313725
266
+ 256 27.73437500
267
+ 257 27.62645914
268
+ 258 27.51937984
269
+ 259 27.41312741
270
+ 260 27.30769231
271
+ 261 27.58620690
272
+ 262 27.48091603
273
+ 263 27.37642586
274
+ 264 27.65151515
275
+ 265 27.92452830
276
+ 266 27.81954887
277
+ 267 28.08988764
278
+ 268 27.98507463
279
+ 269 27.88104089
280
+ 270 27.77777778
281
+ 271 27.67527675
282
+ 272 27.57352941
283
+ 273 27.47252747
284
+ 274 27.73722628
285
+ 275 27.63636364
286
+ 276 27.53623188
287
+ 277 27.43682310
288
+ 278 27.33812950
289
+ 279 27.24014337
290
+ 280 27.50000000
291
+ 281 27.75800712
292
+ 282 27.65957447
293
+ 283 27.56183746
294
+ 284 27.81690141
295
+ 285 27.71929825
296
+ 286 27.97202797
297
+ 287 28.22299652
298
+ 288 28.12500000
299
+ 289 28.02768166
300
+ 290 27.93103448
301
+ 291 28.17869416
302
+ 292 28.08219178
303
+ 293 28.32764505
304
+ 294 28.23129252
305
+ 295 28.47457627
306
+ 296 28.71621622
307
+ 297 28.95622896
308
+ 298 28.85906040
309
+ 299 29.09698997
310
+ 300 29.00000000
311
+ 301 28.90365449
312
+ 302 28.80794702
313
+ 303 28.71287129
314
+ 304 28.61842105
315
+ 305 28.52459016
316
+ 306 28.43137255
317
+ 307 28.33876221
318
+ 308 28.57142857
319
+ 309 28.47896440
320
+ 310 28.38709677
321
+ 311 28.29581994
322
+ 312 28.52564103
323
+ 313 28.43450479
324
+ 314 28.66242038
325
+ 315 28.57142857
326
+ 316 28.48101266
327
+ 317 28.39116719
328
+ 318 28.30188679
329
+ 319 28.21316614
330
+ 320 28.12500000
331
+ 321 28.03738318
332
+ 322 28.26086957
333
+ 323 28.17337461
334
+ 324 28.08641975
335
+ 325 28.00000000
336
+ 326 27.91411043
337
+ 327 28.13455657
338
+ 328 28.35365854
339
+ 329 28.26747720
340
+ 330 28.48484848
341
+ 331 28.39879154
342
+ 332 28.61445783
343
+ 333 28.52852853
344
+ 334 28.44311377
345
+ 335 28.65671642
346
+ 336 28.86904762
347
+ 337 28.78338279
348
+ 338 28.69822485
349
+ 339 28.90855457
350
+ 340 29.11764706
351
+ 341 29.32551320
352
+ 342 29.23976608
353
+ 343 29.15451895
354
+ 344 29.36046512
355
+ 345 29.56521739
356
+ 346 29.76878613
357
+ 347 29.97118156
358
+ 348 29.88505747
359
+ 349 29.79942693
360
+ 350 29.71428571
361
+ 351 29.62962963
362
+ 352 29.54545455
363
+ 353 29.46175637
364
+ 354 29.37853107
365
+ 355 29.57746479
366
+ 356 29.49438202
367
+ 357 29.41176471
368
+ 358 29.32960894
369
+ 359 29.24791086
370
+ 360 29.16666667
371
+ 361 29.36288089
372
+ 362 29.28176796
373
+ 363 29.47658402
374
+ 364 29.39560440
375
+ 365 29.58904110
376
+ 366 29.50819672
377
+ 367 29.70027248
378
+ 368 29.61956522
379
+ 369 29.53929539
380
+ 370 29.45945946
381
+ 371 29.38005391
382
+ 372 29.30107527
383
+ 373 29.22252011
384
+ 374 29.41176471
385
+ 375 29.33333333
386
+ 376 29.25531915
387
+ 377 29.17771883
388
+ 378 29.10052910
389
+ 379 29.02374670
390
+ 380 28.94736842
391
+ 381 28.87139108
392
+ 382 29.05759162
393
+ 383 28.98172324
394
+ 384 28.90625000
395
+ 385 28.83116883
396
+ 386 28.75647668
397
+ 387 28.68217054
398
+ 388 28.86597938
399
+ 389 28.79177378
400
+ 390 28.71794872
401
+ 391 28.64450128
402
+ 392 28.57142857
403
+ 393 28.49872774
404
+ 394 28.68020305
405
+ 395 28.60759494
406
+ 396 28.53535354
407
+ 397 28.71536524
408
+ 398 28.64321608
409
+ 399 28.57142857
410
+ 400 28.50000000
411
+ 401 28.42892768
412
+ 402 28.60696517
413
+ 403 28.78411911
414
+ 404 28.71287129
415
+ 405 28.64197531
416
+ 406 28.57142857
417
+ 407 28.50122850
418
+ 408 28.43137255
419
+ 409 28.36185819
420
+ 410 28.29268293
421
+ 411 28.22384428
422
+ 412 28.39805825
423
+ 413 28.32929782
424
+ 414 28.50241546
425
+ 415 28.67469880
426
+ 416 28.60576923
427
+ 417 28.53717026
428
+ 418 28.46889952
429
+ 419 28.40095465
430
+ 420 28.57142857
431
+ 421 28.74109264
432
+ 422 28.67298578
433
+ 423 28.60520095
434
+ 424 28.53773585
435
+ 425 28.47058824
436
+ 426 28.40375587
437
+ 427 28.33723653
438
+ 428 28.27102804
439
+ 429 28.43822844
440
+ 430 28.37209302
441
+ 431 28.30626450
442
+ 432 28.47222222
443
+ 433 28.63741339
444
+ 434 28.80184332
445
+ 435 28.73563218
446
+ 436 28.66972477
447
+ 437 28.60411899
448
+ 438 28.53881279
449
+ 439 28.47380410
450
+ 440 28.40909091
451
+ 441 28.34467120
452
+ 442 28.28054299
453
+ 443 28.21670429
454
+ 444 28.37837838
455
+ 445 28.31460674
456
+ 446 28.25112108
457
+ 447 28.18791946
458
+ 448 28.12500000
459
+ 449 28.06236080
460
+ 450 28.00000000
461
+ 451 27.93791574
462
+ 452 27.87610619
463
+ 453 27.81456954
464
+ 454 27.97356828
465
+ 455 27.91208791
466
+ 456 27.85087719
467
+ 457 27.78993435
468
+ 458 27.72925764
469
+ 459 27.66884532
470
+ 460 27.60869565
471
+ 461 27.54880694
472
+ 462 27.48917749
473
+ 463 27.64578834
474
+ 464 27.80172414
475
+ 465 27.95698925
476
+ 466 27.89699571
477
+ 467 27.83725910
478
+ 468 27.99145299
479
+ 469 27.93176972
480
+ 470 27.87234043
481
+ 471 28.02547771
482
+ 472 27.96610169
483
+ 473 27.90697674
484
+ 474 27.84810127
485
+ 475 28.00000000
486
+ 476 27.94117647
487
+ 477 28.09224319
488
+ 478 28.03347280
489
+ 479 27.97494781
490
+ 480 27.91666667
491
+ 481 27.85862786
492
+ 482 28.00829876
493
+ 483 27.95031056
494
+ 484 28.09917355
495
+ 485 28.24742268
496
+ 486 28.18930041
497
+ 487 28.13141684
498
+ 488 28.07377049
499
+ 489 28.22085890
500
+ 490 28.36734694
501
+ 491 28.51323829
502
+ 492 28.65853659
503
+ 493 28.60040568
504
+ 494 28.54251012
505
+ 495 28.68686869
506
+ 496 28.62903226
507
+ 497 28.57142857
508
+ 498 28.51405622
509
+ 499 28.65731463
510
+ 500 28.60000000
511
+ 501 28.74251497
512
+ 502 28.68525896
513
+ 503 28.62823062
514
+ 504 28.76984127
515
+ 505 28.71287129
516
+ 506 28.65612648
517
+ 507 28.79684418
518
+ 508 28.74015748
519
+ 509 28.88015717
520
+ 510 29.01960784
521
+ 511 28.96281800
522
+ 512 28.90625000
523
+ 513 28.84990253
524
+ 514 28.79377432
525
+ 515 28.93203883
526
+ 516 28.87596899
527
+ 517 29.01353965
528
+ 518 28.95752896
529
+ 519 28.90173410
530
+ 520 28.84615385
531
+ 521 28.98272553
532
+ 522 28.92720307
533
+ 523 28.87189293
534
+ 524 28.81679389
535
+ 525 28.76190476
536
+ 526 28.70722433
537
+ 527 28.65275142
538
+ 528 28.59848485
539
+ 529 28.54442344
540
+ 530 28.49056604
541
+ 531 28.62523540
542
+ 532 28.57142857
543
+ 533 28.70544090
544
+ 534 28.65168539
545
+ 535 28.59813084
546
+ 536 28.54477612
547
+ 537 28.67783985
548
+ 538 28.62453532
549
+ 539 28.57142857
550
+ 540 28.51851852
551
+ 541 28.46580407
552
+ 542 28.41328413
553
+ 543 28.36095764
554
+ 544 28.30882353
555
+ 545 28.25688073
556
+ 546 28.38827839
557
+ 547 28.51919561
558
+ 548 28.64963504
559
+ 549 28.59744991
560
+ 550 28.54545455
561
+ 551 28.49364791
562
+ 552 28.62318841
563
+ 553 28.75226040
564
+ 554 28.70036101
565
+ 555 28.64864865
566
+ 556 28.59712230
567
+ 557 28.54578097
568
+ 558 28.49462366
569
+ 559 28.44364937
570
+ 560 28.57142857
571
+ 561 28.69875223
572
+ 562 28.64768683
573
+ 563 28.59680284
574
+ 564 28.72340426
575
+ 565 28.84955752
576
+ 566 28.97526502
577
+ 567 28.92416226
578
+ 568 28.87323944
579
+ 569 28.99824253
580
+ 570 28.94736842
581
+ 571 28.89667250
582
+ 572 28.84615385
583
+ 573 28.79581152
584
+ 574 28.91986063
585
+ 575 29.04347826
586
+ 576 29.16666667
587
+ 577 29.11611785
588
+ 578 29.23875433
589
+ 579 29.36096718
590
+ 580 29.48275862
591
+ 581 29.43201377
592
+ 582 29.55326460
593
+ 583 29.50257290
594
+ 584 29.62328767
595
+ 585 29.57264957
596
+ 586 29.69283276
597
+ 587 29.64224872
598
+ 588 29.59183673
599
+ 589 29.54159593
600
+ 590 29.66101695
601
+ 591 29.61082910
602
+ 592 29.56081081
603
+ 593 29.51096121
604
+ 594 29.62962963
605
+ 595 29.57983193
606
+ 596 29.53020134
607
+ 597 29.64824121
608
+ 598 29.59866221
609
+ 599 29.54924875
610
+ 600 29.50000000
611
+ 601 29.61730449
612
+ 602 29.56810631
613
+ 603 29.68490879
614
+ 604 29.63576159
615
+ 605 29.58677686
616
+ 606 29.70297030
617
+ 607 29.65403624
618
+ 608 29.60526316
619
+ 609 29.72085386
620
+ 610 29.67213115
621
+ 611 29.62356792
622
+ 612 29.73856209
623
+ 613 29.69004894
624
+ 614 29.64169381
625
+ 615 29.59349593
626
+ 616 29.70779221
627
+ 617 29.82171799
628
+ 618 29.77346278
629
+ 619 29.88691438
630
+ 620 29.83870968
631
+ 621 29.79066023
632
+ 622 29.90353698
633
+ 623 29.85553772
634
+ 624 29.80769231
635
+ 625 29.76000000
636
+ 626 29.71246006
637
+ 627 29.82456140
638
+ 628 29.77707006
639
+ 629 29.72972973
640
+ 630 29.68253968
641
+ 631 29.79397781
642
+ 632 29.74683544
643
+ 633 29.69984202
644
+ 634 29.81072555
645
+ 635 29.76377953
646
+ 636 29.71698113
647
+ 637 29.67032967
648
+ 638 29.62382445
649
+ 639 29.57746479
650
+ 640 29.53125000
651
+ 641 29.48517941
652
+ 642 29.59501558
653
+ 643 29.54898911
654
+ 644 29.50310559
655
+ 645 29.61240310
656
+ 646 29.56656347
657
+ 647 29.52086553
658
+ 648 29.62962963
659
+ 649 29.58397535
660
+ 650 29.69230769
661
+ 651 29.80030722
662
+ 652 29.90797546
663
+ 653 30.01531394
664
+ 654 29.96941896
665
+ 655 29.92366412
666
+ 656 30.03048780
667
+ 657 30.13698630
668
+ 658 30.09118541
669
+ 659 30.04552352
670
+ 660 30.00000000
671
+ 661 29.95461422
672
+ 662 29.90936556
673
+ 663 29.86425339
674
+ 664 29.81927711
675
+ 665 29.77443609
676
+ 666 29.87987988
677
+ 667 29.83508246
678
+ 668 29.79041916
679
+ 669 29.74588939
680
+ 670 29.85074627
681
+ 671 29.80625931
682
+ 672 29.76190476
683
+ 673 29.71768202
684
+ 674 29.67359050
685
+ 675 29.77777778
686
+ 676 29.88165680
687
+ 677 29.83751846
688
+ 678 29.79351032
689
+ 679 29.74963181
690
+ 680 29.70588235
691
+ 681 29.80910426
692
+ 682 29.76539589
693
+ 683 29.72181552
694
+ 684 29.82456140
695
+ 685 29.78102190
696
+ 686 29.73760933
697
+ 687 29.69432314
698
+ 688 29.65116279
699
+ 689 29.75326560
700
+ 690 29.71014493
701
+ 691 29.81186686
702
+ 692 29.91329480
703
+ 693 30.01443001
704
+ 694 29.97118156
705
+ 695 29.92805755
706
+ 696 30.02873563
707
+ 697 29.98565280
708
+ 698 29.94269341
709
+ 699 29.89985694
710
+ 700 30.00000000
711
+ 701 29.95720399
712
+ 702 29.91452991
713
+ 703 29.87197724
714
+ 704 29.82954545
715
+ 705 29.78723404
716
+ 706 29.74504249
717
+ 707 29.84441301
718
+ 708 29.80225989
719
+ 709 29.76022567
720
+ 710 29.71830986
721
+ 711 29.67651195
722
+ 712 29.63483146
723
+ 713 29.59326788
724
+ 714 29.69187675
725
+ 715 29.79020979
726
+ 716 29.74860335
727
+ 717 29.70711297
728
+ 718 29.66573816
729
+ 719 29.62447844
730
+ 720 29.58333333
731
+ 721 29.54230236
732
+ 722 29.50138504
733
+ 723 29.46058091
734
+ 724 29.55801105
735
+ 725 29.51724138
736
+ 726 29.61432507
737
+ 727 29.57359010
738
+ 728 29.53296703
739
+ 729 29.62962963
740
+ 730 29.58904110
741
+ 731 29.54856361
742
+ 732 29.64480874
743
+ 733 29.60436562
744
+ 734 29.56403270
745
+ 735 29.52380952
746
+ 736 29.48369565
747
+ 737 29.57937585
748
+ 738 29.53929539
749
+ 739 29.63464141
750
+ 740 29.59459459
751
+ 741 29.68960864
752
+ 742 29.64959569
753
+ 743 29.60969044
754
+ 744 29.70430108
755
+ 745 29.66442953
756
+ 746 29.75871314
757
+ 747 29.85274431
758
+ 748 29.81283422
759
+ 749 29.90654206
760
+ 750 29.86666667
761
+
762
+ Final result: 29.8667 +/- 1.6723
763
+ Random chance: 19.8992 +/- 1.4588
scores/Hammer2.1-7b-q3_k_s.wng ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ winogrande_score : loaded 1267 tasks from prompt.
6
+ winogrande_score : selecting 750 random tasks
7
+ winogrande_score : tokenizing selected tasks
8
+ winogrande_score : calculating winogrande score over selected tasks.
9
+ 1 100.0000 -4.119040 -4.953426 1 1
10
+ 2 100.0000 -2.152979 -2.772682 1 1
11
+ 3 100.0000 -2.903926 -2.347759 2 2
12
+ 4 75.0000 -4.682671 -4.277689 2 1
13
+ 5 60.0000 -2.591209 -3.251495 1 2
14
+ 6 66.6667 -4.691504 -4.969282 1 1
15
+ 7 71.4286 -4.044793 -3.000865 2 2
16
+ 8 62.5000 -1.934140 -2.288882 1 2
17
+ 9 66.6667 -3.430774 -5.620167 1 1
18
+ 10 70.0000 -1.795254 -1.338229 2 2
19
+ 11 72.7273 -10.584093 -5.449966 2 2
20
+ 12 75.0000 -4.444959 -3.881480 2 2
21
+ 13 76.9231 -3.406690 -6.481477 1 1
22
+ 14 71.4286 -4.203177 -4.058416 2 1
23
+ 15 66.6667 -3.642724 -5.530904 1 2
24
+ 16 68.7500 -4.540655 -4.346828 2 2
25
+ 17 64.7059 -5.998508 -6.585410 1 2
26
+ 18 66.6667 -1.150037 -2.496882 1 1
27
+ 19 68.4211 -4.297691 -5.530589 1 1
28
+ 20 70.0000 -3.310517 -4.171738 1 1
29
+ 21 71.4286 -2.592171 -3.192350 1 1
30
+ 22 72.7273 -2.722203 -2.333129 2 2
31
+ 23 73.9130 -5.225068 -5.346047 1 1
32
+ 24 70.8333 -3.000350 -2.621286 2 1
33
+ 25 72.0000 -2.794179 -2.569979 2 2
34
+ 26 73.0769 -3.323428 -4.559001 1 1
35
+ 27 74.0741 -5.333671 -5.218637 2 2
36
+ 28 75.0000 -4.679924 -5.347994 1 1
37
+ 29 75.8621 -4.025941 -4.428203 1 1
38
+ 30 73.3333 -1.596673 -1.872998 1 2
39
+ 31 70.9677 -5.803344 -5.765169 2 1
40
+ 32 71.8750 -4.456345 -5.505601 1 1
41
+ 33 72.7273 -3.560781 -6.272218 1 1
42
+ 34 70.5882 -2.411501 -2.277698 2 1
43
+ 35 68.5714 -4.565212 -4.703618 1 2
44
+ 36 69.4444 -4.761726 -3.976420 2 2
45
+ 37 67.5676 -3.215184 -3.164115 2 1
46
+ 38 68.4211 -3.505661 -3.702075 1 1
47
+ 39 69.2308 -4.121650 -3.691478 2 2
48
+ 40 67.5000 -4.220479 -5.384696 1 2
49
+ 41 68.2927 -2.866182 -3.082478 1 1
50
+ 42 69.0476 -1.832843 -2.628542 1 1
51
+ 43 69.7674 -4.203981 -4.167098 2 2
52
+ 44 68.1818 -2.209972 -2.257470 1 2
53
+ 45 68.8889 -0.055066 -14.781285 1 1
54
+ 46 69.5652 -2.655833 -2.842741 1 1
55
+ 47 68.0851 -4.141097 -4.629007 1 2
56
+ 48 68.7500 -5.173315 -4.996133 2 2
57
+ 49 67.3469 -2.658713 -0.806359 2 1
58
+ 50 68.0000 -2.445598 -2.445320 2 2
59
+ 51 68.6275 -2.390509 -2.828995 1 1
60
+ 52 69.2308 -3.321334 -2.640553 2 2
61
+ 53 69.8113 -3.976798 -4.819649 1 1
62
+ 54 70.3704 -4.179716 -3.887785 2 2
63
+ 55 70.9091 -3.168009 -3.061637 2 2
64
+ 56 69.6429 -4.616586 -5.250845 1 2
65
+ 57 68.4211 -2.382087 -3.607181 1 2
66
+ 58 68.9655 -3.308786 -4.080441 1 1
67
+ 59 69.4915 -3.212260 -3.012318 2 2
68
+ 60 70.0000 -4.735128 -4.402984 2 2
69
+ 61 70.4918 -4.138747 -4.430203 1 1
70
+ 62 69.3548 -7.424768 -7.392012 2 1
71
+ 63 68.2540 -3.916372 -4.691247 1 2
72
+ 64 68.7500 -4.128912 -4.344546 1 1
73
+ 65 69.2308 -4.433792 -3.865923 2 2
74
+ 66 69.6970 -2.598315 -3.611124 1 1
75
+ 67 70.1493 -2.458281 -2.092956 2 2
76
+ 68 70.5882 -4.998003 -4.772248 2 2
77
+ 69 71.0145 -4.470690 -7.602764 1 1
78
+ 70 71.4286 -3.594879 -2.944967 2 2
79
+ 71 70.4225 -3.242654 -3.119311 2 1
80
+ 72 69.4444 -2.779251 -3.386760 1 2
81
+ 73 69.8630 -3.196433 -3.377886 1 1
82
+ 74 68.9189 -2.457440 -2.360056 2 1
83
+ 75 69.3333 -1.414575 -2.032116 1 1
84
+ 76 69.7368 -1.828307 -3.885588 1 1
85
+ 77 68.8312 -2.730789 -3.266736 1 2
86
+ 78 69.2308 -1.500520 -1.620784 1 1
87
+ 79 68.3544 -2.284757 -2.596019 1 2
88
+ 80 68.7500 -2.393337 -3.430923 1 1
89
+ 81 69.1358 -3.541601 -3.536059 2 2
90
+ 82 68.2927 -2.399644 -2.456350 1 2
91
+ 83 67.4699 -6.156652 -6.538261 1 2
92
+ 84 67.8571 -2.565144 -2.561048 2 2
93
+ 85 68.2353 -5.283227 -5.792438 1 1
94
+ 86 67.4419 -5.746194 -6.035998 1 2
95
+ 87 66.6667 -1.724562 -1.409913 2 1
96
+ 88 65.9091 -4.409174 -4.214208 2 1
97
+ 89 66.2921 -3.526163 -3.646141 1 1
98
+ 90 66.6667 -5.251731 -4.623703 2 2
99
+ 91 65.9341 -4.785767 -4.359718 2 1
100
+ 92 66.3043 -5.495564 -4.439840 2 2
101
+ 93 66.6667 -2.576762 -2.558395 2 2
102
+ 94 67.0213 -3.760118 -3.034305 2 2
103
+ 95 67.3684 -5.196754 -5.747170 1 1
104
+ 96 67.7083 -2.215294 -3.058990 1 1
105
+ 97 67.0103 -5.332895 -5.128736 2 1
106
+ 98 67.3469 -5.590093 -3.923111 2 2
107
+ 99 66.6667 -3.032924 -2.462443 2 1
108
+ 100 66.0000 -6.251631 -6.586548 1 2
109
+ 101 65.3465 -4.746802 -4.771928 1 2
110
+ 102 65.6863 -4.234385 -5.329082 1 1
111
+ 103 65.0485 -2.704895 -3.209301 1 2
112
+ 104 65.3846 -8.477869 -8.349337 2 2
113
+ 105 65.7143 -1.300832 -2.316796 1 1
114
+ 106 66.0377 -3.218453 -2.931591 2 2
115
+ 107 66.3551 -3.358842 -2.751559 2 2
116
+ 108 66.6667 -3.885504 -3.108573 2 2
117
+ 109 66.0550 -5.749473 -5.925405 1 2
118
+ 110 66.3636 -4.115342 -3.602263 2 2
119
+ 111 66.6667 -3.922155 -3.600750 2 2
120
+ 112 66.0714 -3.562842 -3.326718 2 1
121
+ 113 65.4867 -4.307579 -4.408918 1 2
122
+ 114 65.7895 -3.948758 -4.354202 1 1
123
+ 115 66.0870 -2.490397 -3.262757 1 1
124
+ 116 66.3793 -3.772047 -2.820382 2 2
125
+ 117 65.8120 -4.684731 -5.306394 1 2
126
+ 118 66.1017 -0.931907 -2.544792 1 1
127
+ 119 66.3866 -2.802864 -3.151513 1 1
128
+ 120 65.8333 -2.692800 -2.334203 2 1
129
+ 121 66.1157 -1.530843 -0.986987 2 2
130
+ 122 66.3934 -6.118963 -4.714104 2 2
131
+ 123 65.8537 -3.648946 -3.193305 2 1
132
+ 124 66.1290 -2.745819 -2.530603 2 2
133
+ 125 66.4000 -3.991487 -3.204974 2 2
134
+ 126 65.8730 -6.913576 -7.266268 1 2
135
+ 127 66.1417 -1.827075 -1.176281 2 2
136
+ 128 66.4062 -2.165206 -2.743753 1 1
137
+ 129 66.6667 -2.368397 -1.410115 2 2
138
+ 130 66.9231 -5.702368 -4.792842 2 2
139
+ 131 66.4122 -3.703346 -3.960279 1 2
140
+ 132 65.9091 -2.829936 -3.032965 1 2
141
+ 133 66.1654 -1.437933 -3.981975 1 1
142
+ 134 66.4179 -5.286392 -4.478492 2 2
143
+ 135 66.6667 -4.890313 -5.573819 1 1
144
+ 136 66.9118 -4.747858 -4.156353 2 2
145
+ 137 66.4234 -6.975203 -6.811253 2 1
146
+ 138 65.9420 -4.156419 -4.563094 1 2
147
+ 139 65.4676 -4.445571 -3.685870 2 1
148
+ 140 65.7143 -1.302323 -1.166301 2 2
149
+ 141 65.9574 -4.623712 -3.804766 2 2
150
+ 142 65.4930 -3.056935 -3.180724 1 2
151
+ 143 65.7343 -1.855740 -5.502988 1 1
152
+ 144 65.9722 -2.020842 -2.062857 1 1
153
+ 145 65.5172 -1.685936 -14.865612 1 2
154
+ 146 65.7534 -4.536154 -4.151566 2 2
155
+ 147 65.3061 -4.647116 -5.331117 1 2
156
+ 148 65.5405 -2.108584 -2.470444 1 1
157
+ 149 65.7718 -3.690295 -3.552885 2 2
158
+ 150 66.0000 -4.583680 -3.310037 2 2
159
+ 151 65.5629 -2.573539 -2.438594 2 1
160
+ 152 65.1316 -3.858858 -3.994769 1 2
161
+ 153 64.7059 -3.735412 -3.641260 2 1
162
+ 154 64.9351 -3.523874 -3.650497 1 1
163
+ 155 65.1613 -2.218454 -2.170681 2 2
164
+ 156 64.7436 -2.744646 -2.294788 2 1
165
+ 157 64.9682 -3.723223 -5.011077 1 1
166
+ 158 65.1899 -4.485235 -3.991328 2 2
167
+ 159 65.4088 -3.391440 -5.198628 1 1
168
+ 160 65.6250 -3.815957 -3.040412 2 2
169
+ 161 65.8385 -2.669938 -2.857910 1 1
170
+ 162 65.4321 -0.159388 -14.549849 1 2
171
+ 163 65.6442 -1.885306 -1.983909 1 1
172
+ 164 65.8537 -4.964379 -3.905995 2 2
173
+ 165 66.0606 -4.892967 -4.873906 2 2
174
+ 166 66.2651 -4.197970 -3.703866 2 2
175
+ 167 65.8683 -4.996759 -4.526893 2 1
176
+ 168 65.4762 -3.564034 -4.093298 1 2
177
+ 169 65.0888 -3.821340 -4.271723 1 2
178
+ 170 64.7059 -4.460695 -4.185160 2 1
179
+ 171 64.9123 -3.565052 -4.478635 1 1
180
+ 172 65.1163 -4.976737 -5.921089 1 1
181
+ 173 65.3179 -3.479258 -2.922445 2 2
182
+ 174 65.5172 -6.927096 -6.398138 2 2
183
+ 175 65.1429 -5.004393 -4.689488 2 1
184
+ 176 65.3409 -1.819200 -3.322490 1 1
185
+ 177 65.5367 -5.499838 -5.350595 2 2
186
+ 178 65.1685 -2.558537 -2.338853 2 1
187
+ 179 65.3631 -2.951920 -3.146093 1 1
188
+ 180 65.5556 -5.678107 -5.201027 2 2
189
+ 181 65.7459 -4.237196 -2.945247 2 2
190
+ 182 65.9341 -4.373675 -3.914551 2 2
191
+ 183 66.1202 -6.658811 -6.211316 2 2
192
+ 184 66.3043 -3.016652 -3.749676 1 1
193
+ 185 65.9459 -5.816429 -5.886169 1 2
194
+ 186 66.1290 -3.493778 -3.771653 1 1
195
+ 187 66.3102 -5.216105 -4.865311 2 2
196
+ 188 65.9574 -3.854935 -3.458735 2 1
197
+ 189 65.6085 -5.648031 -5.873412 1 2
198
+ 190 65.7895 -5.330613 -4.879379 2 2
199
+ 191 65.9686 -3.734523 -3.980816 1 1
200
+ 192 66.1458 -5.903465 -3.803017 2 2
201
+ 193 66.3212 -4.117201 -4.006114 2 2
202
+ 194 66.4948 -2.183942 -1.344388 2 2
203
+ 195 66.6667 -4.025940 -4.144062 1 1
204
+ 196 66.8367 -4.308707 -2.923939 2 2
205
+ 197 67.0051 -1.075024 -2.673824 1 1
206
+ 198 67.1717 -6.539963 -6.791079 1 1
207
+ 199 67.3367 -5.090072 -4.022496 2 2
208
+ 200 67.5000 -3.609570 -3.307964 2 2
209
+ 201 67.6617 -3.129874 -3.813590 1 1
210
+ 202 67.8218 -7.393616 -7.286055 2 2
211
+ 203 67.4877 -3.207811 -3.257059 1 2
212
+ 204 67.1569 -3.353348 -3.324518 2 1
213
+ 205 67.3171 -4.213583 -3.474006 2 2
214
+ 206 66.9903 -5.894502 -6.007359 1 2
215
+ 207 66.6667 -3.606097 -3.343614 2 1
216
+ 208 66.8269 -4.469544 -3.206951 2 2
217
+ 209 66.9856 -4.265326 -3.853011 2 2
218
+ 210 66.6667 -8.232447 -9.034203 1 2
219
+ 211 66.3507 -6.167807 -5.875509 2 1
220
+ 212 66.5094 -4.281104 -3.519435 2 2
221
+ 213 66.1972 -5.215059 -6.061242 1 2
222
+ 214 66.3551 -3.154540 -3.368610 1 1
223
+ 215 66.5116 -5.305395 -4.817237 2 2
224
+ 216 66.2037 -3.692814 -3.504212 2 1
225
+ 217 66.3594 -1.397384 -2.041169 1 1
226
+ 218 66.5138 -2.459344 -2.363328 2 2
227
+ 219 66.6667 -2.764815 -3.095966 1 1
228
+ 220 66.3636 -3.462782 -3.626017 1 2
229
+ 221 66.5158 -2.757576 -8.227878 1 1
230
+ 222 66.2162 -0.237790 -6.438212 1 2
231
+ 223 66.3677 -4.540858 -4.111977 2 2
232
+ 224 66.0714 -6.212807 -6.571941 1 2
233
+ 225 66.2222 -3.914199 -3.016356 2 2
234
+ 226 66.3717 -3.083620 -3.447282 1 1
235
+ 227 66.0793 -4.619951 -3.737379 2 1
236
+ 228 66.2281 -1.963143 -3.938937 1 1
237
+ 229 65.9389 -4.634389 -4.777243 1 2
238
+ 230 66.0870 -3.317139 -3.328218 1 1
239
+ 231 66.2338 -2.167796 -2.459982 1 1
240
+ 232 66.3793 -2.540612 -1.505249 2 2
241
+ 233 66.5236 -2.255637 -2.567622 1 1
242
+ 234 66.6667 -4.129432 -3.315158 2 2
243
+ 235 66.8085 -3.998524 -3.471815 2 2
244
+ 236 66.5254 -2.506889 -2.281112 2 1
245
+ 237 66.6667 -1.231014 -1.404838 1 1
246
+ 238 66.8067 -3.348594 -2.964873 2 2
247
+ 239 66.9456 -2.885958 -2.820506 2 2
248
+ 240 66.6667 -7.938212 -8.205932 1 2
249
+ 241 66.8050 -4.379840 -3.669141 2 2
250
+ 242 66.9421 -6.014084 -5.744549 2 2
251
+ 243 66.6667 -4.089905 -4.078334 2 1
252
+ 244 66.8033 -2.063435 -3.511763 1 1
253
+ 245 66.9388 -4.858649 -5.601179 1 1
254
+ 246 67.0732 -2.445678 -3.306882 1 1
255
+ 247 66.8016 -5.062375 -5.764852 1 2
256
+ 248 66.9355 -3.431197 -3.149415 2 2
257
+ 249 67.0683 -3.114865 -2.052910 2 2
258
+ 250 67.2000 -3.344029 -3.883015 1 1
259
+ 251 66.9323 -0.809202 -0.803816 2 1
260
+ 252 67.0635 -6.522077 -6.161349 2 2
261
+ 253 67.1937 -5.189310 -5.233347 1 1
262
+ 254 67.3228 -4.235708 -3.297791 2 2
263
+ 255 67.4510 -4.349571 -5.809328 1 1
264
+ 256 67.5781 -4.109083 -4.302270 1 1
265
+ 257 67.3152 -2.433883 -2.131589 2 1
266
+ 258 67.0543 -3.512397 -3.974688 1 2
267
+ 259 66.7954 -4.455541 -4.402041 2 1
268
+ 260 66.9231 -4.597591 -5.199492 1 1
269
+ 261 66.6667 -3.789287 -3.812892 1 2
270
+ 262 66.4122 -5.018269 -5.025813 1 2
271
+ 263 66.5399 -3.676963 -5.164465 1 1
272
+ 264 66.6667 -4.445817 -4.327982 2 2
273
+ 265 66.7925 -1.298241 -2.043411 1 1
274
+ 266 66.9173 -6.135880 -7.940711 1 1
275
+ 267 67.0412 -3.796568 -3.799939 1 1
276
+ 268 66.7910 -2.939496 -2.714191 2 1
277
+ 269 66.9145 -3.413215 -3.137582 2 2
278
+ 270 67.0370 -2.579144 -4.621944 1 1
279
+ 271 67.1587 -3.155984 -2.929951 2 2
280
+ 272 67.2794 -4.501288 -3.654016 2 2
281
+ 273 67.3993 -3.974936 -3.638017 2 2
282
+ 274 67.5182 -3.093249 -2.164695 2 2
283
+ 275 67.6364 -1.505726 -2.081639 1 1
284
+ 276 67.3913 -2.495108 -2.471533 2 1
285
+ 277 67.5090 -2.707324 -2.982638 1 1
286
+ 278 67.6259 -2.818228 -2.872809 1 1
287
+ 279 67.7419 -4.750406 -3.142676 2 2
288
+ 280 67.8571 -4.155001 -3.881682 2 2
289
+ 281 67.9715 -2.356517 -1.813305 2 2
290
+ 282 68.0851 -4.090209 -3.654321 2 2
291
+ 283 68.1979 -3.317068 -3.176625 2 2
292
+ 284 67.9577 -3.186491 -2.852852 2 1
293
+ 285 68.0702 -5.512442 -6.135068 1 1
294
+ 286 68.1818 -3.585490 -3.613865 1 1
295
+ 287 68.2927 -2.130836 -3.967619 1 1
296
+ 288 68.4028 -3.793437 -2.390291 2 2
297
+ 289 68.5121 -2.647650 -2.952434 1 1
298
+ 290 68.6207 -5.305147 -5.598302 1 1
299
+ 291 68.7285 -1.747695 -3.289903 1 1
300
+ 292 68.4932 -1.772394 -2.280097 1 2
301
+ 293 68.6007 -3.727729 -2.975117 2 2
302
+ 294 68.3673 -5.168380 -3.625096 2 1
303
+ 295 68.4746 -1.904788 -2.142709 1 1
304
+ 296 68.5811 -4.836918 -4.645945 2 2
305
+ 297 68.3502 -1.086555 -1.331717 1 2
306
+ 298 68.1208 -5.325511 -6.519003 1 2
307
+ 299 68.2274 -2.495087 -2.135695 2 2
308
+ 300 68.3333 -3.161014 -4.178363 1 1
309
+ 301 68.4385 -4.807544 -5.933651 1 1
310
+ 302 68.5430 -4.639703 -4.429126 2 2
311
+ 303 68.3168 -9.319123 -8.101205 2 1
312
+ 304 68.4211 -3.519437 -3.312485 2 2
313
+ 305 68.1967 -3.081527 -3.234392 1 2
314
+ 306 68.3007 -2.539967 -2.075151 2 2
315
+ 307 68.4039 -3.307505 -3.566945 1 1
316
+ 308 68.5065 -4.526750 -4.140408 2 2
317
+ 309 68.6084 -3.121014 -3.836145 1 1
318
+ 310 68.7097 -5.013524 -4.950958 2 2
319
+ 311 68.8103 -1.219201 -4.639068 1 1
320
+ 312 68.5897 -6.106842 -4.660099 2 1
321
+ 313 68.3706 -4.227564 -5.228488 1 2
322
+ 314 68.4713 -5.828557 -5.051184 2 2
323
+ 315 68.5714 -2.655104 -3.722694 1 1
324
+ 316 68.6709 -7.068049 -5.228759 2 2
325
+ 317 68.4543 -4.704809 -4.407104 2 1
326
+ 318 68.5535 -4.959691 -4.605947 2 2
327
+ 319 68.6520 -2.358463 -2.992898 1 1
328
+ 320 68.7500 -3.992119 -4.895058 1 1
329
+ 321 68.5358 -2.908246 -3.038358 1 2
330
+ 322 68.6335 -2.513189 -2.482261 2 2
331
+ 323 68.7307 -2.813194 -3.426953 1 1
332
+ 324 68.8272 -4.138490 -4.337070 1 1
333
+ 325 68.9231 -4.806634 -4.039858 2 2
334
+ 326 69.0184 -2.969993 -1.879777 2 2
335
+ 327 69.1131 -2.686991 -3.644337 1 1
336
+ 328 69.2073 -4.100884 -4.433055 1 1
337
+ 329 69.3009 -4.029080 -10.358334 1 1
338
+ 330 69.3939 -1.416111 -1.792089 1 1
339
+ 331 69.1843 -5.756417 -4.000678 2 1
340
+ 332 69.2771 -1.926718 -2.989571 1 1
341
+ 333 69.3694 -2.776821 -1.713076 2 2
342
+ 334 69.4611 -2.020469 -5.526848 1 1
343
+ 335 69.5522 -4.396586 -4.506852 1 1
344
+ 336 69.3452 -3.038359 -3.702801 1 2
345
+ 337 69.4362 -1.894927 -1.827609 2 2
346
+ 338 69.2308 -3.836983 -3.442337 2 1
347
+ 339 69.3215 -6.262496 -5.222195 2 2
348
+ 340 69.4118 -5.752817 -5.915502 1 1
349
+ 341 69.5015 -2.999874 -4.715856 1 1
350
+ 342 69.5906 -5.986037 -5.813611 2 2
351
+ 343 69.6793 -3.620819 -4.085831 1 1
352
+ 344 69.4767 -2.867541 -2.575648 2 1
353
+ 345 69.5652 -1.807732 -3.681900 1 1
354
+ 346 69.3642 -4.144271 -6.579382 1 2
355
+ 347 69.4524 -4.553447 -3.799202 2 2
356
+ 348 69.5402 -1.035349 -2.529210 1 1
357
+ 349 69.3410 -3.327778 -5.090682 1 2
358
+ 350 69.1429 -0.918399 -4.233040 1 2
359
+ 351 69.2308 -2.345509 -3.520691 1 1
360
+ 352 69.3182 -3.940553 -2.830747 2 2
361
+ 353 69.1218 -2.249279 -1.816164 2 1
362
+ 354 69.2090 -5.308864 -4.070614 2 2
363
+ 355 69.2958 -3.539070 -2.657887 2 2
364
+ 356 69.1011 -3.310520 -3.344909 1 2
365
+ 357 69.1877 -4.302419 -4.065842 2 2
366
+ 358 69.2737 -2.494355 -2.352356 2 2
367
+ 359 69.3593 -6.224672 -4.050157 2 2
368
+ 360 69.1667 -2.594617 -3.089776 1 2
369
+ 361 69.2521 -3.230363 -1.833229 2 2
370
+ 362 69.3370 -5.071345 -4.569281 2 2
371
+ 363 69.1460 -2.542558 -2.364528 2 1
372
+ 364 69.2308 -6.115178 -6.803136 1 1
373
+ 365 69.3151 -5.552992 -5.992963 1 1
374
+ 366 69.1257 -3.918864 -9.811050 1 2
375
+ 367 69.2098 -5.421923 -5.863742 1 1
376
+ 368 69.2935 -3.177343 -1.364347 2 2
377
+ 369 69.1057 -4.203430 -4.384639 1 2
378
+ 370 69.1892 -4.851214 -5.030416 1 1
379
+ 371 69.2722 -4.184753 -4.822342 1 1
380
+ 372 69.0860 -2.866881 -2.531271 2 1
381
+ 373 69.1689 -5.202392 -5.856781 1 1
382
+ 374 69.2513 -2.136806 -3.792428 1 1
383
+ 375 69.3333 -3.508407 -2.656872 2 2
384
+ 376 69.4149 -5.250884 -4.104066 2 2
385
+ 377 69.2308 -3.764323 -4.063556 1 2
386
+ 378 69.0476 -3.328649 -3.614419 1 2
387
+ 379 68.8654 -1.662039 -2.502674 1 2
388
+ 380 68.6842 -7.692616 -7.167867 2 1
389
+ 381 68.7664 -11.283637 -9.349103 2 2
390
+ 382 68.5864 -2.046420 -2.122339 1 2
391
+ 383 68.6684 -1.574827 -0.847104 2 2
392
+ 384 68.4896 -4.130069 -3.174278 2 1
393
+ 385 68.3117 -1.291849 -1.031912 2 1
394
+ 386 68.3938 -1.548496 -9.984342 1 1
395
+ 387 68.4755 -1.603152 -1.991260 1 1
396
+ 388 68.2990 -5.102114 -5.958730 1 2
397
+ 389 68.3805 -2.399786 -1.934994 2 2
398
+ 390 68.4615 -2.703519 -3.600629 1 1
399
+ 391 68.2864 -3.657918 -5.591425 1 2
400
+ 392 68.1122 -6.000915 -3.954105 2 1
401
+ 393 68.1934 -7.043935 -5.978241 2 2
402
+ 394 68.2741 -2.002365 -4.763342 1 1
403
+ 395 68.1013 -4.330986 -4.552716 1 2
404
+ 396 67.9293 -6.673991 -6.123300 2 1
405
+ 397 67.7582 -7.656551 -7.034382 2 1
406
+ 398 67.8392 -3.624301 -6.738328 1 1
407
+ 399 67.9198 -4.951066 -5.124933 1 1
408
+ 400 68.0000 -1.832807 -4.714470 1 1
409
+ 401 68.0798 -2.769887 -2.896578 1 1
410
+ 402 67.9104 -4.079042 -4.293630 1 2
411
+ 403 67.9901 -6.888195 -6.539958 2 2
412
+ 404 68.0693 -4.012791 -2.678674 2 2
413
+ 405 68.1481 -3.130586 -4.199491 1 1
414
+ 406 68.2266 -8.225260 -9.481021 1 1
415
+ 407 68.3047 -5.293781 -5.098678 2 2
416
+ 408 68.3824 -3.478577 -3.436351 2 2
417
+ 409 68.4597 -4.167972 -4.411210 1 1
418
+ 410 68.5366 -5.541625 -6.213157 1 1
419
+ 411 68.6131 -5.440713 -6.034681 1 1
420
+ 412 68.6893 -4.426443 -3.029172 2 2
421
+ 413 68.7651 -4.286037 -3.062996 2 2
422
+ 414 68.5990 -5.099631 -5.205265 1 2
423
+ 415 68.4337 -2.672635 -2.773336 1 2
424
+ 416 68.5096 -5.575631 -3.290507 2 2
425
+ 417 68.5851 -5.113293 -5.685486 1 1
426
+ 418 68.6603 -5.736723 -7.174960 1 1
427
+ 419 68.4964 -4.468493 -5.049629 1 2
428
+ 420 68.3333 -3.318200 -3.339440 1 2
429
+ 421 68.4086 -4.427686 -3.719290 2 2
430
+ 422 68.2464 -2.361789 -2.118036 2 1
431
+ 423 68.0851 -3.549772 -3.432178 2 1
432
+ 424 68.1604 -5.162621 -5.517593 1 1
433
+ 425 68.2353 -3.606486 -2.487066 2 2
434
+ 426 68.0751 -2.146521 -2.681085 1 2
435
+ 427 68.1499 -3.778451 -5.047907 1 1
436
+ 428 67.9907 -3.022687 -2.752265 2 1
437
+ 429 68.0653 -3.987441 -3.986847 2 2
438
+ 430 67.9070 -6.607077 -5.911957 2 1
439
+ 431 67.9814 -5.284007 -6.054695 1 1
440
+ 432 67.8241 -4.392149 -4.529019 1 2
441
+ 433 67.8984 -1.833172 -1.724333 2 2
442
+ 434 67.9724 -7.347567 -7.889950 1 1
443
+ 435 68.0460 -3.393203 -3.829139 1 1
444
+ 436 68.1193 -0.724927 -2.528411 1 1
445
+ 437 68.1922 -3.761232 -3.712604 2 2
446
+ 438 68.2648 -3.993209 -3.714343 2 2
447
+ 439 68.1093 -3.793046 -3.230899 2 1
448
+ 440 68.1818 -4.472368 -5.281980 1 1
449
+ 441 68.2540 -3.724891 -5.181984 1 1
450
+ 442 68.3258 -4.836992 -5.364379 1 1
451
+ 443 68.3973 -1.574216 -3.223924 1 1
452
+ 444 68.2432 -5.674445 -7.172248 1 2
453
+ 445 68.3146 -1.644237 -3.134385 1 1
454
+ 446 68.3857 -5.817714 -5.046960 2 2
455
+ 447 68.2327 -2.023469 -1.938441 2 1
456
+ 448 68.3036 -3.792578 -3.508832 2 2
457
+ 449 68.3742 -3.222422 -3.080277 2 2
458
+ 450 68.4444 -3.810568 -3.364422 2 2
459
+ 451 68.5144 -4.104343 -3.445942 2 2
460
+ 452 68.5841 -4.321023 -5.114030 1 1
461
+ 453 68.6534 -5.278905 -4.682106 2 2
462
+ 454 68.7225 -3.201258 -3.622505 1 1
463
+ 455 68.5714 -1.087056 -7.562391 1 2
464
+ 456 68.6404 -4.712631 -4.560905 2 2
465
+ 457 68.4902 -11.223601 -10.860411 2 1
466
+ 458 68.5590 -4.517668 -4.183098 2 2
467
+ 459 68.6275 -4.743351 -4.569104 2 2
468
+ 460 68.6957 -1.688734 -6.779734 1 1
469
+ 461 68.7636 -2.584044 -3.764947 1 1
470
+ 462 68.8312 -5.300207 -5.319726 1 1
471
+ 463 68.8985 -1.009516 -1.201537 1 1
472
+ 464 68.9655 -2.823210 -2.855354 1 1
473
+ 465 69.0323 -3.211841 -3.069345 2 2
474
+ 466 69.0987 -5.368622 -5.057118 2 2
475
+ 467 69.1649 -4.931166 -5.288261 1 1
476
+ 468 69.2308 -5.110211 -4.354724 2 2
477
+ 469 69.0832 -1.797240 -1.770743 2 1
478
+ 470 69.1489 -1.910937 -1.612257 2 2
479
+ 471 69.0021 -2.297094 -1.861999 2 1
480
+ 472 69.0678 -4.424694 -3.814764 2 2
481
+ 473 69.1332 -4.459025 -3.181363 2 2
482
+ 474 69.1983 -5.481959 -5.859749 1 1
483
+ 475 69.2632 -1.445959 -1.068387 2 2
484
+ 476 69.1176 -4.497791 -4.551610 1 2
485
+ 477 68.9727 -4.908286 -5.321484 1 2
486
+ 478 68.8285 -4.727144 -4.146483 2 1
487
+ 479 68.8935 -3.466094 -3.633478 1 1
488
+ 480 68.9583 -5.507957 -5.282809 2 2
489
+ 481 69.0229 -2.433033 -2.924565 1 1
490
+ 482 68.8797 -4.907882 -5.389348 1 2
491
+ 483 68.7371 -5.484952 -4.854682 2 1
492
+ 484 68.5950 -2.526872 -1.586370 2 1
493
+ 485 68.6598 -1.672623 -2.497812 1 1
494
+ 486 68.7243 -2.796244 -3.313833 1 1
495
+ 487 68.7885 -2.519754 -1.774177 2 2
496
+ 488 68.8525 -2.984470 -2.206957 2 2
497
+ 489 68.9162 -1.975868 -2.455865 1 1
498
+ 490 68.9796 -3.482192 -3.910794 1 1
499
+ 491 69.0428 -5.127148 -4.578103 2 2
500
+ 492 69.1057 -5.785787 -5.372174 2 2
501
+ 493 69.1684 -1.506477 -3.115538 1 1
502
+ 494 69.2308 -5.054993 -5.036638 2 2
503
+ 495 69.2929 -4.190526 -4.330722 1 1
504
+ 496 69.3548 -3.720472 -3.537952 2 2
505
+ 497 69.2153 -5.540328 -4.760826 2 1
506
+ 498 69.0763 -1.504464 -1.754027 1 2
507
+ 499 69.1383 -3.141105 -3.401851 1 1
508
+ 500 69.2000 -5.380219 -4.399515 2 2
509
+ 501 69.2615 -1.161664 -2.216163 1 1
510
+ 502 69.3227 -1.517995 -1.821065 1 1
511
+ 503 69.3837 -3.075836 -2.992557 2 2
512
+ 504 69.2460 -2.719119 -2.658581 2 1
513
+ 505 69.1089 -7.055962 -8.371962 1 2
514
+ 506 68.9723 -6.407734 -6.044674 2 1
515
+ 507 69.0335 -4.824501 -4.921940 1 1
516
+ 508 69.0945 -6.489491 -2.865833 2 2
517
+ 509 69.1552 -3.255092 -8.592415 1 1
518
+ 510 69.0196 -4.022797 -3.442316 2 1
519
+ 511 69.0802 -5.307014 -4.019280 2 2
520
+ 512 69.1406 -3.756449 -4.196115 1 1
521
+ 513 69.2008 -3.948641 -3.827832 2 2
522
+ 514 69.0661 -6.062654 -5.745501 2 1
523
+ 515 69.1262 -7.584927 -7.588939 1 1
524
+ 516 68.9922 -4.349790 -4.229074 2 1
525
+ 517 68.8588 -1.588876 -1.560983 2 1
526
+ 518 68.9189 -3.019310 -1.942206 2 2
527
+ 519 68.9788 -10.012157 -9.362842 2 2
528
+ 520 69.0385 -2.731968 -2.393439 2 2
529
+ 521 69.0979 -1.201920 -1.316352 1 1
530
+ 522 69.1571 -3.674573 -3.040379 2 2
531
+ 523 69.2161 -3.787593 -3.737056 2 2
532
+ 524 69.2748 -6.574247 -6.643954 1 1
533
+ 525 69.3333 -4.692077 -2.135269 2 2
534
+ 526 69.3916 -0.561206 -16.804710 1 1
535
+ 527 69.4497 -6.741061 -5.124789 2 2
536
+ 528 69.5076 -6.650594 -5.593540 2 2
537
+ 529 69.5652 -4.864418 -5.760204 1 1
538
+ 530 69.6226 -5.403697 -4.344112 2 2
539
+ 531 69.4915 -4.123256 -3.830426 2 1
540
+ 532 69.5489 -2.280262 -3.037763 1 1
541
+ 533 69.6060 -3.745087 -3.211292 2 2
542
+ 534 69.6629 -2.935961 -3.548033 1 1
543
+ 535 69.5327 -3.196465 -3.279307 1 2
544
+ 536 69.4030 -2.269417 -2.004975 2 1
545
+ 537 69.4600 -5.986547 -6.248462 1 1
546
+ 538 69.5167 -2.580507 -2.685558 1 1
547
+ 539 69.5733 -4.666496 -2.752000 2 2
548
+ 540 69.4444 -2.674911 -3.027056 1 2
549
+ 541 69.5009 -2.783239 -2.981592 1 1
550
+ 542 69.3727 -3.682292 -4.674799 1 2
551
+ 543 69.4291 -5.655987 -5.020757 2 2
552
+ 544 69.4853 -2.562846 -4.087018 1 1
553
+ 545 69.3578 -1.208684 -4.272872 1 2
554
+ 546 69.4139 -2.569133 -3.100200 1 1
555
+ 547 69.4698 -6.035563 -4.870708 2 2
556
+ 548 69.5255 -4.416175 -4.526608 1 1
557
+ 549 69.3989 -5.740114 -5.461845 2 1
558
+ 550 69.4545 -3.470011 -3.391101 2 2
559
+ 551 69.3285 -1.499587 -7.063973 1 2
560
+ 552 69.3841 -3.073270 -3.598244 1 1
561
+ 553 69.4394 -5.993649 -5.642938 2 2
562
+ 554 69.3141 -4.684253 -5.008045 1 2
563
+ 555 69.3694 -3.059141 -2.253784 2 2
564
+ 556 69.4245 -5.973199 -4.837995 2 2
565
+ 557 69.4794 -4.100952 -5.058801 1 1
566
+ 558 69.5341 -1.972620 -3.433215 1 1
567
+ 559 69.5886 -2.450155 -2.318425 2 2
568
+ 560 69.4643 -2.441566 -2.010247 2 1
569
+ 561 69.5187 -2.190756 -2.619232 1 1
570
+ 562 69.5730 -0.729791 -1.505927 1 1
571
+ 563 69.4494 -1.574636 -2.667282 1 2
572
+ 564 69.5035 -2.992205 -3.578209 1 1
573
+ 565 69.3805 -3.561498 -3.607810 1 2
574
+ 566 69.4346 -4.045775 -4.680665 1 1
575
+ 567 69.3122 -2.710636 -2.813227 1 2
576
+ 568 69.3662 -7.259642 -4.924101 2 2
577
+ 569 69.2443 -5.610801 -5.266110 2 1
578
+ 570 69.2982 -4.017564 -6.004910 1 1
579
+ 571 69.3520 -4.796005 -3.480678 2 2
580
+ 572 69.4056 -2.801288 -2.763844 2 2
581
+ 573 69.2845 -4.708818 -5.232337 1 2
582
+ 574 69.3380 -3.893144 -2.560281 2 2
583
+ 575 69.3913 -5.034733 -4.696530 2 2
584
+ 576 69.4444 -2.606024 -3.232027 1 1
585
+ 577 69.4974 -4.201353 -4.037041 2 2
586
+ 578 69.5502 -3.187794 -2.922547 2 2
587
+ 579 69.4301 -5.438629 -5.742666 1 2
588
+ 580 69.4828 -1.846539 -3.337974 1 1
589
+ 581 69.5353 -5.767014 -7.901727 1 1
590
+ 582 69.4158 -5.123087 -5.668014 1 2
591
+ 583 69.4683 -4.484005 -4.407092 2 2
592
+ 584 69.5205 -4.801570 -4.839435 1 1
593
+ 585 69.5726 -0.482099 -1.521062 1 1
594
+ 586 69.6246 -3.791123 -4.291680 1 1
595
+ 587 69.5060 -5.170010 -4.840511 2 1
596
+ 588 69.5578 -4.265429 -2.710217 2 2
597
+ 589 69.4397 -3.223317 -3.509738 1 2
598
+ 590 69.4915 -4.351593 -3.832996 2 2
599
+ 591 69.3739 -3.599840 -3.828559 1 2
600
+ 592 69.4257 -3.231816 -4.257094 1 1
601
+ 593 69.4772 -4.455252 -4.366606 2 2
602
+ 594 69.3603 -4.906308 -3.997883 2 1
603
+ 595 69.4118 -1.998688 -1.887045 2 2
604
+ 596 69.4631 -4.648184 -4.577534 2 2
605
+ 597 69.5142 -2.512922 -2.524202 1 1
606
+ 598 69.5652 -3.324842 -3.722542 1 1
607
+ 599 69.6160 -2.865672 -4.243872 1 1
608
+ 600 69.6667 -3.814197 -3.612399 2 2
609
+ 601 69.7171 -3.255293 -2.879711 2 2
610
+ 602 69.7674 -3.301987 -5.544719 1 1
611
+ 603 69.8176 -6.656727 -5.416613 2 2
612
+ 604 69.7020 -3.230742 -3.260954 1 2
613
+ 605 69.5868 -2.609592 -1.872935 2 1
614
+ 606 69.6370 -3.063185 -2.862562 2 2
615
+ 607 69.5222 -2.917128 -2.497421 2 1
616
+ 608 69.4079 -3.443429 -2.912560 2 1
617
+ 609 69.2939 -7.372441 -7.239206 2 1
618
+ 610 69.3443 -3.005976 -3.313468 1 1
619
+ 611 69.2308 -4.606559 -4.536846 2 1
620
+ 612 69.2810 -4.966691 -6.636167 1 1
621
+ 613 69.3312 -2.863071 -2.095825 2 2
622
+ 614 69.2182 -5.249929 -5.498508 1 2
623
+ 615 69.2683 -2.920552 -2.547790 2 2
624
+ 616 69.1558 -6.347117 -6.393728 1 2
625
+ 617 69.2058 -10.482837 -5.915479 2 2
626
+ 618 69.0939 -3.871097 -4.100213 1 2
627
+ 619 69.1438 -3.785504 -3.149442 2 2
628
+ 620 69.0323 -5.035107 -4.629354 2 1
629
+ 621 69.0821 -3.476031 -2.131619 2 2
630
+ 622 69.1318 -3.552464 -3.998349 1 1
631
+ 623 69.0209 -3.484604 -3.392351 2 1
632
+ 624 69.0705 -4.643794 -3.775977 2 2
633
+ 625 69.1200 -4.100658 -6.588501 1 1
634
+ 626 69.0096 -1.625393 -1.199906 2 1
635
+ 627 69.0590 -4.085450 -4.337955 1 1
636
+ 628 69.1083 -3.266288 -2.500842 2 2
637
+ 629 69.1574 -1.934836 -2.537359 1 1
638
+ 630 69.2063 -1.211162 -1.708143 1 1
639
+ 631 69.2552 -2.451815 -3.866859 1 1
640
+ 632 69.3038 -2.033539 -1.363403 2 2
641
+ 633 69.3523 -2.180563 -2.157074 2 2
642
+ 634 69.4006 -2.691913 -3.776526 1 1
643
+ 635 69.2913 -4.782748 -5.188820 1 2
644
+ 636 69.3396 -1.376953 -3.238773 1 1
645
+ 637 69.3878 -3.429159 -6.122543 1 1
646
+ 638 69.4357 -5.931367 -6.452784 1 1
647
+ 639 69.4836 -1.654200 -2.507864 1 1
648
+ 640 69.3750 -1.964728 -1.991685 1 2
649
+ 641 69.2668 -6.545896 -5.180532 2 1
650
+ 642 69.3146 -4.883635 -3.281547 2 2
651
+ 643 69.3624 -2.120732 -1.586105 2 2
652
+ 644 69.4099 -3.183611 -3.946879 1 1
653
+ 645 69.4574 -1.028160 -3.247170 1 1
654
+ 646 69.3498 -3.944079 -3.593513 2 1
655
+ 647 69.2427 -3.684469 -2.424170 2 1
656
+ 648 69.2901 -3.780525 -2.972334 2 2
657
+ 649 69.3374 -3.684965 -3.912903 1 1
658
+ 650 69.3846 -4.071819 -4.380829 1 1
659
+ 651 69.2780 -6.064477 -6.348314 1 2
660
+ 652 69.3252 -5.715863 -4.989135 2 2
661
+ 653 69.3721 -5.052603 -3.711219 2 2
662
+ 654 69.2661 -3.554076 -3.677589 1 2
663
+ 655 69.1603 -2.768519 -2.960465 1 2
664
+ 656 69.2073 -3.649720 -2.340528 2 2
665
+ 657 69.1020 -5.431038 -5.832234 1 2
666
+ 658 69.1489 -4.616488 -2.496267 2 2
667
+ 659 69.1958 -2.672218 -1.979849 2 2
668
+ 660 69.0909 -3.362274 -3.949524 1 2
669
+ 661 69.1377 -3.221850 -3.447344 1 1
670
+ 662 69.1843 -3.409204 -4.481295 1 1
671
+ 663 69.0799 -6.286427 -6.399870 1 2
672
+ 664 68.9759 -4.039705 -3.661219 2 1
673
+ 665 69.0226 -3.683110 -3.531641 2 2
674
+ 666 69.0691 -3.145826 -4.531809 1 1
675
+ 667 69.1154 -7.208050 -6.368951 2 2
676
+ 668 69.0120 -2.497112 -2.481529 2 1
677
+ 669 69.0583 -2.677064 -3.179734 1 1
678
+ 670 68.9552 -3.520779 -3.207191 2 1
679
+ 671 69.0015 -3.247020 -4.723367 1 1
680
+ 672 68.8988 -3.108395 -3.110343 1 2
681
+ 673 68.9450 -2.018717 -1.805394 2 2
682
+ 674 68.8427 -4.845762 -4.527997 2 1
683
+ 675 68.7407 -3.591314 -3.321730 2 1
684
+ 676 68.6391 -2.087529 -2.730204 1 2
685
+ 677 68.6854 -3.714689 -4.379614 1 1
686
+ 678 68.5841 -5.608682 -6.105275 1 2
687
+ 679 68.6303 -5.857257 -5.057003 2 2
688
+ 680 68.6765 -2.822965 -4.188003 1 1
689
+ 681 68.7225 -2.704372 -3.185760 1 1
690
+ 682 68.7683 -2.327578 -2.302968 2 2
691
+ 683 68.8141 -2.245817 -3.409984 1 1
692
+ 684 68.8596 -5.069051 -7.099782 1 1
693
+ 685 68.9051 -4.443300 -2.216941 2 2
694
+ 686 68.9504 -5.971647 -5.397357 2 2
695
+ 687 68.9956 -4.381819 -3.309089 2 2
696
+ 688 69.0407 -8.276637 -8.277252 1 1
697
+ 689 69.0856 -2.548282 -2.274980 2 2
698
+ 690 68.9855 -2.768632 -2.826689 1 2
699
+ 691 68.8857 -2.827191 -2.428003 2 1
700
+ 692 68.9306 -1.728411 -2.626713 1 1
701
+ 693 68.9755 -4.668349 -5.183150 1 1
702
+ 694 68.8761 -3.282557 -3.518038 1 2
703
+ 695 68.9209 -4.639150 -5.203030 1 1
704
+ 696 68.9655 -4.825577 -5.570054 1 1
705
+ 697 69.0100 -2.035690 -2.377295 1 1
706
+ 698 69.0544 -4.027066 -3.822586 2 2
707
+ 699 69.0987 -3.595093 -4.690185 1 1
708
+ 700 69.1429 -4.474351 -4.021708 2 2
709
+ 701 69.0442 -2.446276 -2.195074 2 1
710
+ 702 68.9459 -4.186103 -4.919098 1 2
711
+ 703 68.9900 -3.476721 -3.928987 1 1
712
+ 704 69.0341 -1.128125 -1.428042 1 1
713
+ 705 68.9362 -3.344077 -3.544628 1 2
714
+ 706 68.9802 -3.292380 -2.802952 2 2
715
+ 707 68.8826 -4.785459 -4.567581 2 1
716
+ 708 68.9266 -3.533267 -2.973665 2 2
717
+ 709 68.9704 -2.608013 -2.164989 2 2
718
+ 710 69.0141 -4.178464 -4.952765 1 1
719
+ 711 68.9170 -1.627336 -2.069198 1 2
720
+ 712 68.8202 -2.350152 -3.424595 1 2
721
+ 713 68.8640 -3.438560 -3.429957 2 2
722
+ 714 68.7675 -3.200774 -3.251874 1 2
723
+ 715 68.8112 -4.454144 -3.727100 2 2
724
+ 716 68.8547 -4.391012 -4.311406 2 2
725
+ 717 68.8982 -2.253513 -0.341702 2 2
726
+ 718 68.9415 -4.706014 -4.723713 1 1
727
+ 719 68.8456 -5.222919 -5.079775 2 1
728
+ 720 68.7500 -2.391449 -4.716194 1 2
729
+ 721 68.6546 -4.882289 -5.646648 1 2
730
+ 722 68.5596 -2.277268 -1.767647 2 1
731
+ 723 68.6030 -2.400512 -2.639141 1 1
732
+ 724 68.6464 -4.527413 -4.425976 2 2
733
+ 725 68.5517 -2.630269 -3.486116 1 2
734
+ 726 68.5950 -4.253718 -3.854415 2 2
735
+ 727 68.6382 -3.601682 -2.865148 2 2
736
+ 728 68.6813 -3.221851 -3.295494 1 1
737
+ 729 68.7243 -1.309755 -2.639606 1 1
738
+ 730 68.7671 -3.916920 -4.241753 1 1
739
+ 731 68.6731 -5.752945 -4.787099 2 1
740
+ 732 68.7158 -2.914277 -2.032896 2 2
741
+ 733 68.6221 -6.265514 -6.685935 1 2
742
+ 734 68.6649 -0.786519 -11.688575 1 1
743
+ 735 68.5714 -3.422326 -3.063004 2 1
744
+ 736 68.6141 -2.530855 -3.258477 1 1
745
+ 737 68.6567 -4.899638 -5.403581 1 1
746
+ 738 68.5637 -4.333287 -4.517331 1 2
747
+ 739 68.6062 -0.869469 -6.449938 1 1
748
+ 740 68.6486 -4.177191 -3.175862 2 2
749
+ 741 68.5560 -5.001028 -4.341011 2 1
750
+ 742 68.5984 -2.467130 -2.322556 2 2
751
+ 743 68.5061 -4.132953 -4.390192 1 2
752
+ 744 68.5484 -4.798147 -4.007128 2 2
753
+ 745 68.5906 -2.593132 -3.448767 1 1
754
+ 746 68.4987 -7.596664 -7.189275 2 1
755
+ 747 68.5408 -5.636856 -6.323819 1 1
756
+ 748 68.5829 -4.870705 -3.881089 2 2
757
+ 749 68.6248 -4.192736 -5.121568 1 1
758
+ 750 68.6667 -2.125436 -7.431914 1 1
759
+
760
+ Final Winogrande score(750 tasks): 68.6667 +/- 1.6949
scores/Hammer2.1-7b-q4_k_m-naive.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 57.14285714
18
+ 8 62.50000000
19
+ 9 55.55555556
20
+ 10 60.00000000
21
+ 11 54.54545455
22
+ 12 58.33333333
23
+ 13 61.53846154
24
+ 14 64.28571429
25
+ 15 60.00000000
26
+ 16 62.50000000
27
+ 17 64.70588235
28
+ 18 61.11111111
29
+ 19 57.89473684
30
+ 20 55.00000000
31
+ 21 57.14285714
32
+ 22 59.09090909
33
+ 23 56.52173913
34
+ 24 58.33333333
35
+ 25 56.00000000
36
+ 26 53.84615385
37
+ 27 55.55555556
38
+ 28 53.57142857
39
+ 29 55.17241379
40
+ 30 56.66666667
41
+ 31 54.83870968
42
+ 32 53.12500000
43
+ 33 54.54545455
44
+ 34 52.94117647
45
+ 35 54.28571429
46
+ 36 52.77777778
47
+ 37 54.05405405
48
+ 38 55.26315789
49
+ 39 56.41025641
50
+ 40 57.50000000
51
+ 41 58.53658537
52
+ 42 59.52380952
53
+ 43 60.46511628
54
+ 44 59.09090909
55
+ 45 60.00000000
56
+ 46 60.86956522
57
+ 47 61.70212766
58
+ 48 60.41666667
59
+ 49 59.18367347
60
+ 50 60.00000000
61
+ 51 58.82352941
62
+ 52 59.61538462
63
+ 53 60.37735849
64
+ 54 59.25925926
65
+ 55 58.18181818
66
+ 56 58.92857143
67
+ 57 59.64912281
68
+ 58 60.34482759
69
+ 59 61.01694915
70
+ 60 61.66666667
71
+ 61 60.65573770
72
+ 62 59.67741935
73
+ 63 60.31746032
74
+ 64 60.93750000
75
+ 65 60.00000000
76
+ 66 60.60606061
77
+ 67 61.19402985
78
+ 68 61.76470588
79
+ 69 60.86956522
80
+ 70 60.00000000
81
+ 71 59.15492958
82
+ 72 58.33333333
83
+ 73 58.90410959
84
+ 74 59.45945946
85
+ 75 60.00000000
86
+ 76 60.52631579
87
+ 77 61.03896104
88
+ 78 61.53846154
89
+ 79 60.75949367
90
+ 80 61.25000000
91
+ 81 60.49382716
92
+ 82 60.97560976
93
+ 83 60.24096386
94
+ 84 59.52380952
95
+ 85 60.00000000
96
+ 86 60.46511628
97
+ 87 60.91954023
98
+ 88 61.36363636
99
+ 89 61.79775281
100
+ 90 61.11111111
101
+ 91 61.53846154
102
+ 92 61.95652174
103
+ 93 61.29032258
104
+ 94 60.63829787
105
+ 95 61.05263158
106
+ 96 61.45833333
107
+ 97 60.82474227
108
+ 98 60.20408163
109
+ 99 60.60606061
110
+ 100 61.00000000
111
+ 101 60.39603960
112
+ 102 60.78431373
113
+ 103 60.19417476
114
+ 104 59.61538462
115
+ 105 60.00000000
116
+ 106 59.43396226
117
+ 107 59.81308411
118
+ 108 59.25925926
119
+ 109 58.71559633
120
+ 110 59.09090909
121
+ 111 59.45945946
122
+ 112 59.82142857
123
+ 113 59.29203540
124
+ 114 59.64912281
125
+ 115 60.00000000
126
+ 116 59.48275862
127
+ 117 58.97435897
128
+ 118 59.32203390
129
+ 119 58.82352941
130
+ 120 59.16666667
131
+ 121 58.67768595
132
+ 122 59.01639344
133
+ 123 59.34959350
134
+ 124 58.87096774
135
+ 125 59.20000000
136
+ 126 59.52380952
137
+ 127 59.05511811
138
+ 128 59.37500000
139
+ 129 58.91472868
140
+ 130 58.46153846
141
+ 131 58.77862595
142
+ 132 59.09090909
143
+ 133 59.39849624
144
+ 134 58.95522388
145
+ 135 59.25925926
146
+ 136 58.82352941
147
+ 137 59.12408759
148
+ 138 59.42028986
149
+ 139 59.71223022
150
+ 140 59.28571429
151
+ 141 59.57446809
152
+ 142 59.15492958
153
+ 143 59.44055944
154
+ 144 59.02777778
155
+ 145 59.31034483
156
+ 146 58.90410959
157
+ 147 59.18367347
158
+ 148 59.45945946
159
+ 149 59.73154362
160
+ 150 60.00000000
161
+ 151 60.26490066
162
+ 152 60.52631579
163
+ 153 60.78431373
164
+ 154 60.38961039
165
+ 155 60.64516129
166
+ 156 60.89743590
167
+ 157 61.14649682
168
+ 158 61.39240506
169
+ 159 61.63522013
170
+ 160 61.25000000
171
+ 161 60.86956522
172
+ 162 61.11111111
173
+ 163 60.73619632
174
+ 164 60.36585366
175
+ 165 60.60606061
176
+ 166 60.84337349
177
+ 167 60.47904192
178
+ 168 60.71428571
179
+ 169 60.94674556
180
+ 170 60.58823529
181
+ 171 60.23391813
182
+ 172 60.46511628
183
+ 173 60.69364162
184
+ 174 60.91954023
185
+ 175 60.57142857
186
+ 176 60.22727273
187
+ 177 60.45197740
188
+ 178 60.11235955
189
+ 179 60.33519553
190
+ 180 60.00000000
191
+ 181 60.22099448
192
+ 182 60.43956044
193
+ 183 60.10928962
194
+ 184 59.78260870
195
+ 185 60.00000000
196
+ 186 60.21505376
197
+ 187 60.42780749
198
+ 188 60.10638298
199
+ 189 59.78835979
200
+ 190 59.47368421
201
+ 191 59.68586387
202
+ 192 59.37500000
203
+ 193 59.06735751
204
+ 194 59.27835052
205
+ 195 59.48717949
206
+ 196 59.69387755
207
+ 197 59.89847716
208
+ 198 59.59595960
209
+ 199 59.29648241
210
+ 200 59.50000000
211
+ 201 59.70149254
212
+ 202 59.90099010
213
+ 203 60.09852217
214
+ 204 60.29411765
215
+ 205 60.00000000
216
+ 206 59.70873786
217
+ 207 59.90338164
218
+ 208 60.09615385
219
+ 209 60.28708134
220
+ 210 60.47619048
221
+ 211 60.66350711
222
+ 212 60.37735849
223
+ 213 60.56338028
224
+ 214 60.74766355
225
+ 215 60.93023256
226
+ 216 60.64814815
227
+ 217 60.36866359
228
+ 218 60.09174312
229
+ 219 60.27397260
230
+ 220 60.00000000
231
+ 221 59.72850679
232
+ 222 59.90990991
233
+ 223 59.64125561
234
+ 224 59.82142857
235
+ 225 59.55555556
236
+ 226 59.29203540
237
+ 227 59.47136564
238
+ 228 59.64912281
239
+ 229 59.82532751
240
+ 230 60.00000000
241
+ 231 59.74025974
242
+ 232 59.48275862
243
+ 233 59.65665236
244
+ 234 59.82905983
245
+ 235 59.57446809
246
+ 236 59.74576271
247
+ 237 59.91561181
248
+ 238 60.08403361
249
+ 239 60.25104603
250
+ 240 60.41666667
251
+ 241 60.58091286
252
+ 242 60.33057851
253
+ 243 60.49382716
254
+ 244 60.24590164
255
+ 245 60.40816327
256
+ 246 60.56910569
257
+ 247 60.72874494
258
+ 248 60.88709677
259
+ 249 60.64257028
260
+ 250 60.80000000
261
+ 251 60.95617530
262
+ 252 61.11111111
263
+ 253 61.26482213
264
+ 254 61.02362205
265
+ 255 61.17647059
266
+ 256 61.32812500
267
+ 257 61.47859922
268
+ 258 61.62790698
269
+ 259 61.38996139
270
+ 260 61.53846154
271
+ 261 61.68582375
272
+ 262 61.45038168
273
+ 263 61.59695817
274
+ 264 61.74242424
275
+ 265 61.88679245
276
+ 266 61.65413534
277
+ 267 61.79775281
278
+ 268 61.94029851
279
+ 269 62.08178439
280
+ 270 62.22222222
281
+ 271 62.36162362
282
+ 272 62.13235294
283
+ 273 61.90476190
284
+ 274 61.67883212
285
+ 275 61.45454545
286
+ 276 61.59420290
287
+ 277 61.73285199
288
+ 278 61.51079137
289
+ 279 61.64874552
290
+ 280 61.78571429
291
+ 281 61.92170819
292
+ 282 62.05673759
293
+ 283 61.83745583
294
+ 284 61.97183099
295
+ 285 62.10526316
296
+ 286 62.23776224
297
+ 287 62.02090592
298
+ 288 62.15277778
299
+ 289 61.93771626
300
+ 290 62.06896552
301
+ 291 62.19931271
302
+ 292 61.98630137
303
+ 293 62.11604096
304
+ 294 61.90476190
305
+ 295 62.03389831
306
+ 296 62.16216216
307
+ 297 62.28956229
308
+ 298 62.41610738
309
+ 299 62.54180602
310
+ 300 62.33333333
311
+ 301 62.12624585
312
+ 302 62.25165563
313
+ 303 62.37623762
314
+ 304 62.50000000
315
+ 305 62.62295082
316
+ 306 62.41830065
317
+ 307 62.21498371
318
+ 308 62.33766234
319
+ 309 62.45954693
320
+ 310 62.25806452
321
+ 311 62.37942122
322
+ 312 62.17948718
323
+ 313 62.30031949
324
+ 314 62.10191083
325
+ 315 61.90476190
326
+ 316 62.02531646
327
+ 317 62.14511041
328
+ 318 61.94968553
329
+ 319 61.75548589
330
+ 320 61.56250000
331
+ 321 61.37071651
332
+ 322 61.49068323
333
+ 323 61.60990712
334
+ 324 61.41975309
335
+ 325 61.23076923
336
+ 326 61.34969325
337
+ 327 61.16207951
338
+ 328 61.28048780
339
+ 329 61.39817629
340
+ 330 61.51515152
341
+ 331 61.63141994
342
+ 332 61.74698795
343
+ 333 61.86186186
344
+ 334 61.67664671
345
+ 335 61.49253731
346
+ 336 61.30952381
347
+ 337 61.12759644
348
+ 338 60.94674556
349
+ 339 61.06194690
350
+ 340 61.17647059
351
+ 341 61.29032258
352
+ 342 61.40350877
353
+ 343 61.51603499
354
+ 344 61.62790698
355
+ 345 61.44927536
356
+ 346 61.56069364
357
+ 347 61.38328530
358
+ 348 61.20689655
359
+ 349 61.31805158
360
+ 350 61.14285714
361
+ 351 61.25356125
362
+ 352 61.36363636
363
+ 353 61.47308782
364
+ 354 61.29943503
365
+ 355 61.12676056
366
+ 356 60.95505618
367
+ 357 60.78431373
368
+ 358 60.89385475
369
+ 359 60.72423398
370
+ 360 60.83333333
371
+ 361 60.94182825
372
+ 362 61.04972376
373
+ 363 61.15702479
374
+ 364 60.98901099
375
+ 365 60.82191781
376
+ 366 60.92896175
377
+ 367 61.03542234
378
+ 368 61.14130435
379
+ 369 60.97560976
380
+ 370 60.81081081
381
+ 371 60.64690027
382
+ 372 60.48387097
383
+ 373 60.32171582
384
+ 374 60.42780749
385
+ 375 60.53333333
386
+ 376 60.37234043
387
+ 377 60.21220159
388
+ 378 60.31746032
389
+ 379 60.42216359
390
+ 380 60.52631579
391
+ 381 60.36745407
392
+ 382 60.20942408
393
+ 383 60.05221932
394
+ 384 60.15625000
395
+ 385 60.25974026
396
+ 386 60.36269430
397
+ 387 60.20671835
398
+ 388 60.05154639
399
+ 389 60.15424165
400
+ 390 60.00000000
401
+ 391 59.84654731
402
+ 392 59.94897959
403
+ 393 59.79643766
404
+ 394 59.89847716
405
+ 395 60.00000000
406
+ 396 59.84848485
407
+ 397 59.94962217
408
+ 398 59.79899497
409
+ 399 59.89974937
410
+ 400 60.00000000
411
+ 401 60.09975062
412
+ 402 59.95024876
413
+ 403 60.04962779
414
+ 404 60.14851485
415
+ 405 60.24691358
416
+ 406 60.34482759
417
+ 407 60.44226044
418
+ 408 60.53921569
419
+ 409 60.39119804
420
+ 410 60.48780488
421
+ 411 60.58394161
422
+ 412 60.67961165
423
+ 413 60.53268765
424
+ 414 60.62801932
425
+ 415 60.48192771
426
+ 416 60.33653846
427
+ 417 60.19184652
428
+ 418 60.28708134
429
+ 419 60.14319809
430
+ 420 60.00000000
431
+ 421 60.09501188
432
+ 422 60.18957346
433
+ 423 60.28368794
434
+ 424 60.14150943
435
+ 425 60.23529412
436
+ 426 60.09389671
437
+ 427 60.18735363
438
+ 428 60.04672897
439
+ 429 60.13986014
440
+ 430 60.00000000
441
+ 431 60.09280742
442
+ 432 60.18518519
443
+ 433 60.04618938
444
+ 434 60.13824885
445
+ 435 60.22988506
446
+ 436 60.32110092
447
+ 437 60.18306636
448
+ 438 60.04566210
449
+ 439 60.13667426
450
+ 440 60.00000000
451
+ 441 60.09070295
452
+ 442 59.95475113
453
+ 443 59.81941309
454
+ 444 59.90990991
455
+ 445 60.00000000
456
+ 446 60.08968610
457
+ 447 60.17897092
458
+ 448 60.04464286
459
+ 449 60.13363029
460
+ 450 60.22222222
461
+ 451 60.31042129
462
+ 452 60.39823009
463
+ 453 60.26490066
464
+ 454 60.13215859
465
+ 455 60.21978022
466
+ 456 60.08771930
467
+ 457 60.17505470
468
+ 458 60.26200873
469
+ 459 60.34858388
470
+ 460 60.21739130
471
+ 461 60.08676790
472
+ 462 60.17316017
473
+ 463 60.04319654
474
+ 464 59.91379310
475
+ 465 59.78494624
476
+ 466 59.87124464
477
+ 467 59.95717345
478
+ 468 60.04273504
479
+ 469 59.91471215
480
+ 470 60.00000000
481
+ 471 60.08492569
482
+ 472 60.16949153
483
+ 473 60.04228330
484
+ 474 60.12658228
485
+ 475 60.00000000
486
+ 476 59.87394958
487
+ 477 59.95807128
488
+ 478 60.04184100
489
+ 479 60.12526096
490
+ 480 60.20833333
491
+ 481 60.08316008
492
+ 482 60.16597510
493
+ 483 60.24844720
494
+ 484 60.12396694
495
+ 485 60.20618557
496
+ 486 60.28806584
497
+ 487 60.16427105
498
+ 488 60.24590164
499
+ 489 60.12269939
500
+ 490 60.00000000
501
+ 491 60.08146640
502
+ 492 60.16260163
503
+ 493 60.04056795
504
+ 494 60.12145749
505
+ 495 60.00000000
506
+ 496 60.08064516
507
+ 497 60.16096579
508
+ 498 60.24096386
509
+ 499 60.32064128
510
+ 500 60.40000000
511
+ 501 60.27944112
512
+ 502 60.35856574
513
+ 503 60.43737575
514
+ 504 60.31746032
515
+ 505 60.39603960
516
+ 506 60.27667984
517
+ 507 60.15779093
518
+ 508 60.03937008
519
+ 509 59.92141454
520
+ 510 60.00000000
521
+ 511 59.88258317
522
+ 512 59.96093750
523
+ 513 59.84405458
524
+ 514 59.92217899
525
+ 515 60.00000000
526
+ 516 59.88372093
527
+ 517 59.96131528
528
+ 518 60.03861004
529
+ 519 59.92292871
530
+ 520 59.80769231
531
+ 521 59.88483685
532
+ 522 59.77011494
533
+ 523 59.65583174
534
+ 524 59.54198473
535
+ 525 59.42857143
536
+ 526 59.31558935
537
+ 527 59.20303605
538
+ 528 59.09090909
539
+ 529 58.97920605
540
+ 530 59.05660377
541
+ 531 59.13370998
542
+ 532 59.21052632
543
+ 533 59.28705441
544
+ 534 59.17602996
545
+ 535 59.06542056
546
+ 536 59.14179104
547
+ 537 59.21787709
548
+ 538 59.10780669
549
+ 539 59.18367347
550
+ 540 59.07407407
551
+ 541 59.14972274
552
+ 542 59.04059041
553
+ 543 59.11602210
554
+ 544 59.00735294
555
+ 545 59.08256881
556
+ 546 58.97435897
557
+ 547 58.86654479
558
+ 548 58.94160584
559
+ 549 59.01639344
560
+ 550 59.09090909
561
+ 551 58.98366606
562
+ 552 58.87681159
563
+ 553 58.95117541
564
+ 554 59.02527076
565
+ 555 59.09909910
566
+ 556 58.99280576
567
+ 557 59.06642729
568
+ 558 58.96057348
569
+ 559 59.03398927
570
+ 560 59.10714286
571
+ 561 59.18003565
572
+ 562 59.25266904
573
+ 563 59.32504440
574
+ 564 59.21985816
575
+ 565 59.11504425
576
+ 566 59.18727915
577
+ 567 59.08289242
578
+ 568 59.15492958
579
+ 569 59.05096661
580
+ 570 58.94736842
581
+ 571 59.01926445
582
+ 572 58.91608392
583
+ 573 58.81326353
584
+ 574 58.88501742
585
+ 575 58.95652174
586
+ 576 59.02777778
587
+ 577 59.09878683
588
+ 578 59.16955017
589
+ 579 59.24006908
590
+ 580 59.13793103
591
+ 581 59.03614458
592
+ 582 59.10652921
593
+ 583 59.17667238
594
+ 584 59.24657534
595
+ 585 59.31623932
596
+ 586 59.38566553
597
+ 587 59.28449744
598
+ 588 59.18367347
599
+ 589 59.25297114
600
+ 590 59.32203390
601
+ 591 59.39086294
602
+ 592 59.29054054
603
+ 593 59.35919056
604
+ 594 59.42760943
605
+ 595 59.32773109
606
+ 596 59.22818792
607
+ 597 59.12897822
608
+ 598 59.03010033
609
+ 599 59.09849750
610
+ 600 59.16666667
611
+ 601 59.23460899
612
+ 602 59.13621262
613
+ 603 59.03814262
614
+ 604 58.94039735
615
+ 605 58.84297521
616
+ 606 58.91089109
617
+ 607 58.97858320
618
+ 608 59.04605263
619
+ 609 58.94909688
620
+ 610 59.01639344
621
+ 611 58.91980360
622
+ 612 58.82352941
623
+ 613 58.72756933
624
+ 614 58.63192182
625
+ 615 58.53658537
626
+ 616 58.44155844
627
+ 617 58.50891410
628
+ 618 58.41423948
629
+ 619 58.31987076
630
+ 620 58.38709677
631
+ 621 58.45410628
632
+ 622 58.36012862
633
+ 623 58.42696629
634
+ 624 58.33333333
635
+ 625 58.24000000
636
+ 626 58.14696486
637
+ 627 58.21371611
638
+ 628 58.12101911
639
+ 629 58.02861685
640
+ 630 58.09523810
641
+ 631 58.00316957
642
+ 632 57.91139241
643
+ 633 57.81990521
644
+ 634 57.88643533
645
+ 635 57.95275591
646
+ 636 58.01886792
647
+ 637 57.92778650
648
+ 638 57.99373041
649
+ 639 57.90297340
650
+ 640 57.81250000
651
+ 641 57.72230889
652
+ 642 57.63239875
653
+ 643 57.54276827
654
+ 644 57.60869565
655
+ 645 57.51937984
656
+ 646 57.58513932
657
+ 647 57.65069552
658
+ 648 57.56172840
659
+ 649 57.47303544
660
+ 650 57.38461538
661
+ 651 57.45007680
662
+ 652 57.36196319
663
+ 653 57.42725881
664
+ 654 57.49235474
665
+ 655 57.40458015
666
+ 656 57.46951220
667
+ 657 57.38203957
668
+ 658 57.29483283
669
+ 659 57.35963581
670
+ 660 57.42424242
671
+ 661 57.33736762
672
+ 662 57.40181269
673
+ 663 57.46606335
674
+ 664 57.53012048
675
+ 665 57.59398496
676
+ 666 57.50750751
677
+ 667 57.57121439
678
+ 668 57.48502994
679
+ 669 57.39910314
680
+ 670 57.46268657
681
+ 671 57.52608048
682
+ 672 57.44047619
683
+ 673 57.35512630
684
+ 674 57.41839763
685
+ 675 57.48148148
686
+ 676 57.39644970
687
+ 677 57.45937962
688
+ 678 57.52212389
689
+ 679 57.58468336
690
+ 680 57.64705882
691
+ 681 57.56240822
692
+ 682 57.47800587
693
+ 683 57.39385066
694
+ 684 57.30994152
695
+ 685 57.37226277
696
+ 686 57.28862974
697
+ 687 57.35080058
698
+ 688 57.41279070
699
+ 689 57.47460087
700
+ 690 57.53623188
701
+ 691 57.59768452
702
+ 692 57.65895954
703
+ 693 57.72005772
704
+ 694 57.78097983
705
+ 695 57.69784173
706
+ 696 57.61494253
707
+ 697 57.67575323
708
+ 698 57.59312321
709
+ 699 57.51072961
710
+ 700 57.57142857
711
+ 701 57.48930100
712
+ 702 57.54985755
713
+ 703 57.61024182
714
+ 704 57.67045455
715
+ 705 57.73049645
716
+ 706 57.79036827
717
+ 707 57.85007072
718
+ 708 57.90960452
719
+ 709 57.96897038
720
+ 710 58.02816901
721
+ 711 58.08720113
722
+ 712 58.14606742
723
+ 713 58.20476858
724
+ 714 58.12324930
725
+ 715 58.04195804
726
+ 716 58.10055866
727
+ 717 58.01952580
728
+ 718 57.93871866
729
+ 719 57.85813630
730
+ 720 57.91666667
731
+ 721 57.83633842
732
+ 722 57.89473684
733
+ 723 57.95297372
734
+ 724 58.01104972
735
+ 725 58.06896552
736
+ 726 57.98898072
737
+ 727 57.90921596
738
+ 728 57.96703297
739
+ 729 58.02469136
740
+ 730 58.08219178
741
+ 731 58.13953488
742
+ 732 58.19672131
743
+ 733 58.25375171
744
+ 734 58.31062670
745
+ 735 58.23129252
746
+ 736 58.28804348
747
+ 737 58.20895522
748
+ 738 58.13008130
749
+ 739 58.05142084
750
+ 740 57.97297297
751
+ 741 58.02968961
752
+ 742 58.08625337
753
+ 743 58.14266487
754
+ 744 58.06451613
755
+ 745 58.12080537
756
+ 746 58.04289544
757
+ 747 57.96519411
758
+ 748 57.88770053
759
+ 749 57.94392523
760
+ 750 58.00000000
761
+
762
+ Final result: 58.0000 +/- 1.8034
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-q4_k_m-naive.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 77.77777778
20
+ 10 80.00000000
21
+ 11 81.81818182
22
+ 12 75.00000000
23
+ 13 76.92307692
24
+ 14 78.57142857
25
+ 15 80.00000000
26
+ 16 81.25000000
27
+ 17 76.47058824
28
+ 18 72.22222222
29
+ 19 68.42105263
30
+ 20 70.00000000
31
+ 21 71.42857143
32
+ 22 72.72727273
33
+ 23 73.91304348
34
+ 24 70.83333333
35
+ 25 68.00000000
36
+ 26 65.38461538
37
+ 27 62.96296296
38
+ 28 64.28571429
39
+ 29 65.51724138
40
+ 30 63.33333333
41
+ 31 64.51612903
42
+ 32 62.50000000
43
+ 33 63.63636364
44
+ 34 64.70588235
45
+ 35 65.71428571
46
+ 36 63.88888889
47
+ 37 62.16216216
48
+ 38 63.15789474
49
+ 39 64.10256410
50
+ 40 65.00000000
51
+ 41 65.85365854
52
+ 42 64.28571429
53
+ 43 62.79069767
54
+ 44 63.63636364
55
+ 45 64.44444444
56
+ 46 63.04347826
57
+ 47 61.70212766
58
+ 48 60.41666667
59
+ 49 61.22448980
60
+ 50 60.00000000
61
+ 51 60.78431373
62
+ 52 59.61538462
63
+ 53 58.49056604
64
+ 54 57.40740741
65
+ 55 58.18181818
66
+ 56 57.14285714
67
+ 57 57.89473684
68
+ 58 56.89655172
69
+ 59 57.62711864
70
+ 60 58.33333333
71
+ 61 57.37704918
72
+ 62 56.45161290
73
+ 63 57.14285714
74
+ 64 57.81250000
75
+ 65 56.92307692
76
+ 66 56.06060606
77
+ 67 56.71641791
78
+ 68 57.35294118
79
+ 69 57.97101449
80
+ 70 58.57142857
81
+ 71 57.74647887
82
+ 72 58.33333333
83
+ 73 58.90410959
84
+ 74 59.45945946
85
+ 75 58.66666667
86
+ 76 59.21052632
87
+ 77 59.74025974
88
+ 78 60.25641026
89
+ 79 60.75949367
90
+ 80 61.25000000
91
+ 81 61.72839506
92
+ 82 62.19512195
93
+ 83 62.65060241
94
+ 84 63.09523810
95
+ 85 62.35294118
96
+ 86 62.79069767
97
+ 87 63.21839080
98
+ 88 63.63636364
99
+ 89 62.92134831
100
+ 90 63.33333333
101
+ 91 63.73626374
102
+ 92 63.04347826
103
+ 93 63.44086022
104
+ 94 63.82978723
105
+ 95 64.21052632
106
+ 96 64.58333333
107
+ 97 64.94845361
108
+ 98 65.30612245
109
+ 99 65.65656566
110
+ 100 65.00000000
111
+ 101 65.34653465
112
+ 102 64.70588235
113
+ 103 64.07766990
114
+ 104 64.42307692
115
+ 105 64.76190476
116
+ 106 65.09433962
117
+ 107 65.42056075
118
+ 108 65.74074074
119
+ 109 65.13761468
120
+ 110 65.45454545
121
+ 111 65.76576577
122
+ 112 66.07142857
123
+ 113 66.37168142
124
+ 114 66.66666667
125
+ 115 66.95652174
126
+ 116 66.37931034
127
+ 117 65.81196581
128
+ 118 66.10169492
129
+ 119 66.38655462
130
+ 120 65.83333333
131
+ 121 65.28925620
132
+ 122 65.57377049
133
+ 123 65.85365854
134
+ 124 65.32258065
135
+ 125 65.60000000
136
+ 126 65.87301587
137
+ 127 66.14173228
138
+ 128 65.62500000
139
+ 129 65.89147287
140
+ 130 65.38461538
141
+ 131 65.64885496
142
+ 132 65.15151515
143
+ 133 65.41353383
144
+ 134 65.67164179
145
+ 135 65.92592593
146
+ 136 66.17647059
147
+ 137 66.42335766
148
+ 138 66.66666667
149
+ 139 66.90647482
150
+ 140 67.14285714
151
+ 141 67.37588652
152
+ 142 67.60563380
153
+ 143 67.83216783
154
+ 144 68.05555556
155
+ 145 68.27586207
156
+ 146 68.49315068
157
+ 147 68.70748299
158
+ 148 68.91891892
159
+ 149 69.12751678
160
+ 150 69.33333333
161
+ 151 68.87417219
162
+ 152 68.42105263
163
+ 153 68.62745098
164
+ 154 68.83116883
165
+ 155 69.03225806
166
+ 156 68.58974359
167
+ 157 68.15286624
168
+ 158 68.35443038
169
+ 159 67.92452830
170
+ 160 68.12500000
171
+ 161 68.32298137
172
+ 162 68.51851852
173
+ 163 68.71165644
174
+ 164 68.90243902
175
+ 165 69.09090909
176
+ 166 69.27710843
177
+ 167 69.46107784
178
+ 168 69.64285714
179
+ 169 69.23076923
180
+ 170 69.41176471
181
+ 171 69.59064327
182
+ 172 69.18604651
183
+ 173 68.78612717
184
+ 174 68.39080460
185
+ 175 68.57142857
186
+ 176 68.75000000
187
+ 177 68.92655367
188
+ 178 69.10112360
189
+ 179 68.71508380
190
+ 180 68.88888889
191
+ 181 68.50828729
192
+ 182 68.68131868
193
+ 183 68.85245902
194
+ 184 68.47826087
195
+ 185 68.64864865
196
+ 186 68.27956989
197
+ 187 67.91443850
198
+ 188 67.55319149
199
+ 189 67.19576720
200
+ 190 67.36842105
201
+ 191 67.53926702
202
+ 192 67.70833333
203
+ 193 67.35751295
204
+ 194 67.52577320
205
+ 195 67.17948718
206
+ 196 67.34693878
207
+ 197 67.00507614
208
+ 198 66.66666667
209
+ 199 66.83417085
210
+ 200 67.00000000
211
+ 201 67.16417910
212
+ 202 66.83168317
213
+ 203 66.50246305
214
+ 204 66.66666667
215
+ 205 66.34146341
216
+ 206 66.50485437
217
+ 207 66.66666667
218
+ 208 66.34615385
219
+ 209 66.50717703
220
+ 210 66.66666667
221
+ 211 66.82464455
222
+ 212 66.98113208
223
+ 213 67.13615023
224
+ 214 67.28971963
225
+ 215 66.97674419
226
+ 216 67.12962963
227
+ 217 66.82027650
228
+ 218 66.97247706
229
+ 219 67.12328767
230
+ 220 67.27272727
231
+ 221 66.96832579
232
+ 222 67.11711712
233
+ 223 67.26457399
234
+ 224 67.41071429
235
+ 225 67.55555556
236
+ 226 67.69911504
237
+ 227 67.40088106
238
+ 228 67.54385965
239
+ 229 67.68558952
240
+ 230 67.39130435
241
+ 231 67.09956710
242
+ 232 66.81034483
243
+ 233 66.52360515
244
+ 234 66.66666667
245
+ 235 66.38297872
246
+ 236 66.10169492
247
+ 237 66.24472574
248
+ 238 66.38655462
249
+ 239 66.52719665
250
+ 240 66.66666667
251
+ 241 66.80497925
252
+ 242 66.94214876
253
+ 243 67.07818930
254
+ 244 66.80327869
255
+ 245 66.93877551
256
+ 246 67.07317073
257
+ 247 67.20647773
258
+ 248 67.33870968
259
+ 249 67.46987952
260
+ 250 67.60000000
261
+ 251 67.72908367
262
+ 252 67.85714286
263
+ 253 67.58893281
264
+ 254 67.71653543
265
+ 255 67.84313725
266
+ 256 67.96875000
267
+ 257 67.70428016
268
+ 258 67.82945736
269
+ 259 67.95366795
270
+ 260 68.07692308
271
+ 261 68.19923372
272
+ 262 67.93893130
273
+ 263 68.06083650
274
+ 264 68.18181818
275
+ 265 68.30188679
276
+ 266 68.04511278
277
+ 267 68.16479401
278
+ 268 68.28358209
279
+ 269 68.40148699
280
+ 270 68.51851852
281
+ 271 68.26568266
282
+ 272 68.38235294
283
+ 273 68.13186813
284
+ 274 68.24817518
285
+ 275 68.00000000
286
+ 276 68.11594203
287
+ 277 68.23104693
288
+ 278 68.34532374
289
+ 279 68.45878136
290
+ 280 68.57142857
291
+ 281 68.68327402
292
+ 282 68.79432624
293
+ 283 68.55123675
294
+ 284 68.66197183
295
+ 285 68.42105263
296
+ 286 68.53146853
297
+ 287 68.64111498
298
+ 288 68.75000000
299
+ 289 68.85813149
300
+ 290 68.96551724
301
+ 291 69.07216495
302
+ 292 68.83561644
303
+ 293 68.94197952
304
+ 294 69.04761905
305
+ 295 69.15254237
306
+ 296 69.25675676
307
+ 297 69.36026936
308
+ 298 69.46308725
309
+ 299 69.23076923
310
+ 300 69.00000000
311
+ 301 68.77076412
312
+ 302 68.87417219
313
+ 303 68.64686469
314
+ 304 68.42105263
315
+ 305 68.52459016
316
+ 306 68.62745098
317
+ 307 68.40390879
318
+ 308 68.50649351
319
+ 309 68.60841424
320
+ 310 68.70967742
321
+ 311 68.81028939
322
+ 312 68.91025641
323
+ 313 69.00958466
324
+ 314 68.78980892
325
+ 315 68.88888889
326
+ 316 68.98734177
327
+ 317 69.08517350
328
+ 318 68.86792453
329
+ 319 68.65203762
330
+ 320 68.75000000
331
+ 321 68.53582555
332
+ 322 68.32298137
333
+ 323 68.42105263
334
+ 324 68.51851852
335
+ 325 68.61538462
336
+ 326 68.40490798
337
+ 327 68.19571865
338
+ 328 68.29268293
339
+ 329 68.38905775
340
+ 330 68.48484848
341
+ 331 68.58006042
342
+ 332 68.37349398
343
+ 333 68.46846847
344
+ 334 68.26347305
345
+ 335 68.35820896
346
+ 336 68.45238095
347
+ 337 68.24925816
348
+ 338 68.04733728
349
+ 339 68.14159292
350
+ 340 68.23529412
351
+ 341 68.32844575
352
+ 342 68.42105263
353
+ 343 68.51311953
354
+ 344 68.31395349
355
+ 345 68.40579710
356
+ 346 68.20809249
357
+ 347 68.29971182
358
+ 348 68.10344828
359
+ 349 67.90830946
360
+ 350 68.00000000
361
+ 351 68.09116809
362
+ 352 68.18181818
363
+ 353 68.27195467
364
+ 354 68.36158192
365
+ 355 68.45070423
366
+ 356 68.53932584
367
+ 357 68.62745098
368
+ 358 68.71508380
369
+ 359 68.80222841
370
+ 360 68.61111111
371
+ 361 68.69806094
372
+ 362 68.78453039
373
+ 363 68.87052342
374
+ 364 68.95604396
375
+ 365 69.04109589
376
+ 366 69.12568306
377
+ 367 69.20980926
378
+ 368 69.29347826
379
+ 369 69.37669377
380
+ 370 69.18918919
381
+ 371 69.00269542
382
+ 372 69.08602151
383
+ 373 68.90080429
384
+ 374 68.98395722
385
+ 375 69.06666667
386
+ 376 69.14893617
387
+ 377 68.96551724
388
+ 378 69.04761905
389
+ 379 69.12928760
390
+ 380 69.21052632
391
+ 381 69.29133858
392
+ 382 69.10994764
393
+ 383 69.19060052
394
+ 384 69.27083333
395
+ 385 69.35064935
396
+ 386 69.43005181
397
+ 387 69.50904393
398
+ 388 69.32989691
399
+ 389 69.40874036
400
+ 390 69.48717949
401
+ 391 69.30946292
402
+ 392 69.38775510
403
+ 393 69.46564885
404
+ 394 69.54314721
405
+ 395 69.62025316
406
+ 396 69.69696970
407
+ 397 69.77329975
408
+ 398 69.84924623
409
+ 399 69.92481203
410
+ 400 69.75000000
411
+ 401 69.82543641
412
+ 402 69.90049751
413
+ 403 69.72704715
414
+ 404 69.80198020
415
+ 405 69.62962963
416
+ 406 69.70443350
417
+ 407 69.77886978
418
+ 408 69.85294118
419
+ 409 69.92665037
420
+ 410 69.75609756
421
+ 411 69.58637470
422
+ 412 69.41747573
423
+ 413 69.49152542
424
+ 414 69.56521739
425
+ 415 69.39759036
426
+ 416 69.47115385
427
+ 417 69.54436451
428
+ 418 69.61722488
429
+ 419 69.68973747
430
+ 420 69.76190476
431
+ 421 69.59619952
432
+ 422 69.66824645
433
+ 423 69.73995272
434
+ 424 69.81132075
435
+ 425 69.88235294
436
+ 426 69.95305164
437
+ 427 69.78922717
438
+ 428 69.85981308
439
+ 429 69.69696970
440
+ 430 69.76744186
441
+ 431 69.83758701
442
+ 432 69.90740741
443
+ 433 69.97690531
444
+ 434 70.04608295
445
+ 435 69.88505747
446
+ 436 69.95412844
447
+ 437 69.79405034
448
+ 438 69.63470320
449
+ 439 69.70387244
450
+ 440 69.77272727
451
+ 441 69.84126984
452
+ 442 69.90950226
453
+ 443 69.97742664
454
+ 444 70.04504505
455
+ 445 70.11235955
456
+ 446 70.17937220
457
+ 447 70.24608501
458
+ 448 70.31250000
459
+ 449 70.37861915
460
+ 450 70.44444444
461
+ 451 70.28824834
462
+ 452 70.35398230
463
+ 453 70.19867550
464
+ 454 70.26431718
465
+ 455 70.10989011
466
+ 456 70.17543860
467
+ 457 70.24070022
468
+ 458 70.30567686
469
+ 459 70.37037037
470
+ 460 70.43478261
471
+ 461 70.49891540
472
+ 462 70.56277056
473
+ 463 70.62634989
474
+ 464 70.68965517
475
+ 465 70.53763441
476
+ 466 70.60085837
477
+ 467 70.66381156
478
+ 468 70.51282051
479
+ 469 70.36247335
480
+ 470 70.21276596
481
+ 471 70.06369427
482
+ 472 70.12711864
483
+ 473 70.19027484
484
+ 474 70.25316456
485
+ 475 70.31578947
486
+ 476 70.16806723
487
+ 477 70.02096436
488
+ 478 70.08368201
489
+ 479 70.14613779
490
+ 480 70.20833333
491
+ 481 70.27027027
492
+ 482 70.12448133
493
+ 483 70.18633540
494
+ 484 70.04132231
495
+ 485 69.89690722
496
+ 486 69.95884774
497
+ 487 70.02053388
498
+ 488 69.87704918
499
+ 489 69.93865031
500
+ 490 70.00000000
501
+ 491 70.06109980
502
+ 492 70.12195122
503
+ 493 70.18255578
504
+ 494 70.24291498
505
+ 495 70.30303030
506
+ 496 70.36290323
507
+ 497 70.42253521
508
+ 498 70.28112450
509
+ 499 70.14028056
510
+ 500 70.20000000
511
+ 501 70.25948104
512
+ 502 70.31872510
513
+ 503 70.37773360
514
+ 504 70.43650794
515
+ 505 70.49504950
516
+ 506 70.55335968
517
+ 507 70.61143984
518
+ 508 70.66929134
519
+ 509 70.72691552
520
+ 510 70.78431373
521
+ 511 70.84148728
522
+ 512 70.70312500
523
+ 513 70.56530214
524
+ 514 70.62256809
525
+ 515 70.67961165
526
+ 516 70.54263566
527
+ 517 70.40618956
528
+ 518 70.46332046
529
+ 519 70.32755299
530
+ 520 70.38461538
531
+ 521 70.24952015
532
+ 522 70.11494253
533
+ 523 70.17208413
534
+ 524 70.22900763
535
+ 525 70.28571429
536
+ 526 70.15209125
537
+ 527 70.01897533
538
+ 528 70.07575758
539
+ 529 70.13232514
540
+ 530 70.18867925
541
+ 531 70.24482109
542
+ 532 70.30075188
543
+ 533 70.16885553
544
+ 534 70.22471910
545
+ 535 70.28037383
546
+ 536 70.33582090
547
+ 537 70.39106145
548
+ 538 70.44609665
549
+ 539 70.31539889
550
+ 540 70.37037037
551
+ 541 70.42513863
552
+ 542 70.47970480
553
+ 543 70.53406998
554
+ 544 70.58823529
555
+ 545 70.64220183
556
+ 546 70.69597070
557
+ 547 70.74954296
558
+ 548 70.80291971
559
+ 549 70.85610200
560
+ 550 70.72727273
561
+ 551 70.78039927
562
+ 552 70.65217391
563
+ 553 70.70524412
564
+ 554 70.75812274
565
+ 555 70.63063063
566
+ 556 70.68345324
567
+ 557 70.55655296
568
+ 558 70.43010753
569
+ 559 70.30411449
570
+ 560 70.35714286
571
+ 561 70.40998217
572
+ 562 70.28469751
573
+ 563 70.33747780
574
+ 564 70.39007092
575
+ 565 70.26548673
576
+ 566 70.14134276
577
+ 567 70.19400353
578
+ 568 70.24647887
579
+ 569 70.12302285
580
+ 570 70.17543860
581
+ 571 70.22767075
582
+ 572 70.27972028
583
+ 573 70.33158813
584
+ 574 70.38327526
585
+ 575 70.43478261
586
+ 576 70.48611111
587
+ 577 70.36395147
588
+ 578 70.41522491
589
+ 579 70.46632124
590
+ 580 70.34482759
591
+ 581 70.22375215
592
+ 582 70.27491409
593
+ 583 70.32590051
594
+ 584 70.37671233
595
+ 585 70.42735043
596
+ 586 70.47781570
597
+ 587 70.52810903
598
+ 588 70.40816327
599
+ 589 70.28862479
600
+ 590 70.16949153
601
+ 591 70.21996616
602
+ 592 70.27027027
603
+ 593 70.15177066
604
+ 594 70.03367003
605
+ 595 70.08403361
606
+ 596 70.13422819
607
+ 597 70.18425461
608
+ 598 70.23411371
609
+ 599 70.11686144
610
+ 600 70.16666667
611
+ 601 70.21630616
612
+ 602 70.26578073
613
+ 603 70.14925373
614
+ 604 70.19867550
615
+ 605 70.24793388
616
+ 606 70.13201320
617
+ 607 70.18121911
618
+ 608 70.06578947
619
+ 609 70.11494253
620
+ 610 70.16393443
621
+ 611 70.21276596
622
+ 612 70.09803922
623
+ 613 70.14681892
624
+ 614 70.03257329
625
+ 615 70.08130081
626
+ 616 69.96753247
627
+ 617 70.01620746
628
+ 618 70.06472492
629
+ 619 69.95153473
630
+ 620 70.00000000
631
+ 621 69.88727858
632
+ 622 69.93569132
633
+ 623 69.98394864
634
+ 624 70.03205128
635
+ 625 70.08000000
636
+ 626 69.96805112
637
+ 627 70.01594896
638
+ 628 70.06369427
639
+ 629 69.95230525
640
+ 630 70.00000000
641
+ 631 70.04754358
642
+ 632 69.93670886
643
+ 633 69.98420221
644
+ 634 70.03154574
645
+ 635 70.07874016
646
+ 636 70.12578616
647
+ 637 70.17268446
648
+ 638 70.21943574
649
+ 639 70.26604069
650
+ 640 70.15625000
651
+ 641 70.20280811
652
+ 642 70.09345794
653
+ 643 70.13996890
654
+ 644 70.03105590
655
+ 645 69.92248062
656
+ 646 69.96904025
657
+ 647 70.01545595
658
+ 648 70.06172840
659
+ 649 69.95377504
660
+ 650 70.00000000
661
+ 651 70.04608295
662
+ 652 70.09202454
663
+ 653 70.13782542
664
+ 654 70.18348624
665
+ 655 70.22900763
666
+ 656 70.27439024
667
+ 657 70.31963470
668
+ 658 70.21276596
669
+ 659 70.10622155
670
+ 660 70.00000000
671
+ 661 70.04538578
672
+ 662 70.09063444
673
+ 663 70.13574661
674
+ 664 70.18072289
675
+ 665 70.22556391
676
+ 666 70.27027027
677
+ 667 70.31484258
678
+ 668 70.35928144
679
+ 669 70.40358744
680
+ 670 70.44776119
681
+ 671 70.49180328
682
+ 672 70.53571429
683
+ 673 70.57949480
684
+ 674 70.62314540
685
+ 675 70.66666667
686
+ 676 70.56213018
687
+ 677 70.60561300
688
+ 678 70.64896755
689
+ 679 70.69219440
690
+ 680 70.73529412
691
+ 681 70.63142438
692
+ 682 70.67448680
693
+ 683 70.71742313
694
+ 684 70.76023392
695
+ 685 70.65693431
696
+ 686 70.55393586
697
+ 687 70.59679767
698
+ 688 70.63953488
699
+ 689 70.68214804
700
+ 690 70.72463768
701
+ 691 70.76700434
702
+ 692 70.80924855
703
+ 693 70.70707071
704
+ 694 70.74927954
705
+ 695 70.79136691
706
+ 696 70.83333333
707
+ 697 70.87517934
708
+ 698 70.91690544
709
+ 699 70.95851216
710
+ 700 71.00000000
711
+ 701 71.04136947
712
+ 702 71.08262108
713
+ 703 71.12375533
714
+ 704 71.16477273
715
+ 705 71.20567376
716
+ 706 71.24645892
717
+ 707 71.28712871
718
+ 708 71.32768362
719
+ 709 71.36812412
720
+ 710 71.40845070
721
+ 711 71.30801688
722
+ 712 71.34831461
723
+ 713 71.38849930
724
+ 714 71.42857143
725
+ 715 71.32867133
726
+ 716 71.36871508
727
+ 717 71.40864714
728
+ 718 71.44846797
729
+ 719 71.34909597
730
+ 720 71.38888889
731
+ 721 71.28987517
732
+ 722 71.32963989
733
+ 723 71.36929461
734
+ 724 71.40883978
735
+ 725 71.44827586
736
+ 726 71.48760331
737
+ 727 71.52682256
738
+ 728 71.56593407
739
+ 729 71.60493827
740
+ 730 71.50684932
741
+ 731 71.54582763
742
+ 732 71.44808743
743
+ 733 71.48703956
744
+ 734 71.52588556
745
+ 735 71.56462585
746
+ 736 71.60326087
747
+ 737 71.50610583
748
+ 738 71.40921409
749
+ 739 71.31258457
750
+ 740 71.35135135
751
+ 741 71.39001350
752
+ 742 71.42857143
753
+ 743 71.46702557
754
+ 744 71.50537634
755
+ 745 71.40939597
756
+ 746 71.44772118
757
+ 747 71.35207497
758
+ 748 71.39037433
759
+ 749 71.42857143
760
+ 750 71.46666667
scores/Hammer2.1-7b-q4_k_m-naive.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 50.00000000
15
+ 5 40.00000000
16
+ 6 33.33333333
17
+ 7 42.85714286
18
+ 8 50.00000000
19
+ 9 44.44444444
20
+ 10 40.00000000
21
+ 11 36.36363636
22
+ 12 41.66666667
23
+ 13 46.15384615
24
+ 14 42.85714286
25
+ 15 40.00000000
26
+ 16 43.75000000
27
+ 17 47.05882353
28
+ 18 44.44444444
29
+ 19 47.36842105
30
+ 20 50.00000000
31
+ 21 52.38095238
32
+ 22 50.00000000
33
+ 23 47.82608696
34
+ 24 45.83333333
35
+ 25 48.00000000
36
+ 26 46.15384615
37
+ 27 48.14814815
38
+ 28 50.00000000
39
+ 29 51.72413793
40
+ 30 53.33333333
41
+ 31 54.83870968
42
+ 32 56.25000000
43
+ 33 54.54545455
44
+ 34 55.88235294
45
+ 35 57.14285714
46
+ 36 55.55555556
47
+ 37 54.05405405
48
+ 38 55.26315789
49
+ 39 53.84615385
50
+ 40 52.50000000
51
+ 41 53.65853659
52
+ 42 52.38095238
53
+ 43 53.48837209
54
+ 44 52.27272727
55
+ 45 53.33333333
56
+ 46 52.17391304
57
+ 47 51.06382979
58
+ 48 52.08333333
59
+ 49 53.06122449
60
+ 50 52.00000000
61
+ 51 50.98039216
62
+ 52 50.00000000
63
+ 53 49.05660377
64
+ 54 48.14814815
65
+ 55 49.09090909
66
+ 56 50.00000000
67
+ 57 49.12280702
68
+ 58 50.00000000
69
+ 59 50.84745763
70
+ 60 50.00000000
71
+ 61 49.18032787
72
+ 62 48.38709677
73
+ 63 47.61904762
74
+ 64 46.87500000
75
+ 65 46.15384615
76
+ 66 46.96969697
77
+ 67 46.26865672
78
+ 68 45.58823529
79
+ 69 44.92753623
80
+ 70 45.71428571
81
+ 71 46.47887324
82
+ 72 45.83333333
83
+ 73 45.20547945
84
+ 74 45.94594595
85
+ 75 46.66666667
86
+ 76 47.36842105
87
+ 77 48.05194805
88
+ 78 48.71794872
89
+ 79 48.10126582
90
+ 80 48.75000000
91
+ 81 49.38271605
92
+ 82 48.78048780
93
+ 83 49.39759036
94
+ 84 50.00000000
95
+ 85 50.58823529
96
+ 86 50.00000000
97
+ 87 50.57471264
98
+ 88 51.13636364
99
+ 89 50.56179775
100
+ 90 50.00000000
101
+ 91 49.45054945
102
+ 92 48.91304348
103
+ 93 48.38709677
104
+ 94 48.93617021
105
+ 95 48.42105263
106
+ 96 48.95833333
107
+ 97 48.45360825
108
+ 98 47.95918367
109
+ 99 47.47474747
110
+ 100 47.00000000
111
+ 101 47.52475248
112
+ 102 47.05882353
113
+ 103 46.60194175
114
+ 104 47.11538462
115
+ 105 47.61904762
116
+ 106 48.11320755
117
+ 107 47.66355140
118
+ 108 48.14814815
119
+ 109 47.70642202
120
+ 110 47.27272727
121
+ 111 46.84684685
122
+ 112 47.32142857
123
+ 113 46.90265487
124
+ 114 47.36842105
125
+ 115 47.82608696
126
+ 116 48.27586207
127
+ 117 47.86324786
128
+ 118 47.45762712
129
+ 119 47.89915966
130
+ 120 47.50000000
131
+ 121 47.93388430
132
+ 122 48.36065574
133
+ 123 47.96747967
134
+ 124 47.58064516
135
+ 125 48.00000000
136
+ 126 47.61904762
137
+ 127 47.24409449
138
+ 128 46.87500000
139
+ 129 46.51162791
140
+ 130 46.15384615
141
+ 131 46.56488550
142
+ 132 46.21212121
143
+ 133 45.86466165
144
+ 134 45.52238806
145
+ 135 45.18518519
146
+ 136 45.58823529
147
+ 137 45.98540146
148
+ 138 45.65217391
149
+ 139 46.04316547
150
+ 140 46.42857143
151
+ 141 46.09929078
152
+ 142 46.47887324
153
+ 143 46.85314685
154
+ 144 46.52777778
155
+ 145 46.89655172
156
+ 146 47.26027397
157
+ 147 46.93877551
158
+ 148 47.29729730
159
+ 149 47.65100671
160
+ 150 47.33333333
161
+ 151 47.01986755
162
+ 152 47.36842105
163
+ 153 47.05882353
164
+ 154 46.75324675
165
+ 155 46.45161290
166
+ 156 46.15384615
167
+ 157 46.49681529
168
+ 158 46.83544304
169
+ 159 46.54088050
170
+ 160 46.87500000
171
+ 161 46.58385093
172
+ 162 46.29629630
173
+ 163 46.01226994
174
+ 164 45.73170732
175
+ 165 46.06060606
176
+ 166 45.78313253
177
+ 167 46.10778443
178
+ 168 45.83333333
179
+ 169 46.15384615
180
+ 170 45.88235294
181
+ 171 45.61403509
182
+ 172 45.34883721
183
+ 173 45.08670520
184
+ 174 45.40229885
185
+ 175 45.14285714
186
+ 176 44.88636364
187
+ 177 44.63276836
188
+ 178 44.38202247
189
+ 179 44.69273743
190
+ 180 44.44444444
191
+ 181 44.19889503
192
+ 182 43.95604396
193
+ 183 44.26229508
194
+ 184 44.02173913
195
+ 185 44.32432432
196
+ 186 44.62365591
197
+ 187 44.38502674
198
+ 188 44.14893617
199
+ 189 44.44444444
200
+ 190 44.21052632
201
+ 191 43.97905759
202
+ 192 43.75000000
203
+ 193 43.52331606
204
+ 194 43.29896907
205
+ 195 43.58974359
206
+ 196 43.36734694
207
+ 197 43.14720812
208
+ 198 42.92929293
209
+ 199 42.71356784
210
+ 200 42.50000000
211
+ 201 42.28855721
212
+ 202 42.07920792
213
+ 203 41.87192118
214
+ 204 41.66666667
215
+ 205 41.95121951
216
+ 206 41.74757282
217
+ 207 42.02898551
218
+ 208 41.82692308
219
+ 209 41.62679426
220
+ 210 41.42857143
221
+ 211 41.70616114
222
+ 212 41.50943396
223
+ 213 41.31455399
224
+ 214 41.12149533
225
+ 215 41.39534884
226
+ 216 41.66666667
227
+ 217 41.47465438
228
+ 218 41.74311927
229
+ 219 42.00913242
230
+ 220 42.27272727
231
+ 221 42.53393665
232
+ 222 42.34234234
233
+ 223 42.60089686
234
+ 224 42.85714286
235
+ 225 43.11111111
236
+ 226 43.36283186
237
+ 227 43.17180617
238
+ 228 43.42105263
239
+ 229 43.23144105
240
+ 230 43.04347826
241
+ 231 42.85714286
242
+ 232 42.67241379
243
+ 233 42.48927039
244
+ 234 42.73504274
245
+ 235 42.55319149
246
+ 236 42.37288136
247
+ 237 42.19409283
248
+ 238 42.01680672
249
+ 239 41.84100418
250
+ 240 42.08333333
251
+ 241 41.90871369
252
+ 242 42.14876033
253
+ 243 42.38683128
254
+ 244 42.21311475
255
+ 245 42.44897959
256
+ 246 42.27642276
257
+ 247 42.51012146
258
+ 248 42.74193548
259
+ 249 42.57028112
260
+ 250 42.40000000
261
+ 251 42.23107570
262
+ 252 42.06349206
263
+ 253 41.89723320
264
+ 254 41.73228346
265
+ 255 41.56862745
266
+ 256 41.40625000
267
+ 257 41.63424125
268
+ 258 41.47286822
269
+ 259 41.31274131
270
+ 260 41.15384615
271
+ 261 40.99616858
272
+ 262 40.83969466
273
+ 263 40.68441065
274
+ 264 40.90909091
275
+ 265 40.75471698
276
+ 266 40.97744361
277
+ 267 40.82397004
278
+ 268 40.67164179
279
+ 269 40.89219331
280
+ 270 41.11111111
281
+ 271 40.95940959
282
+ 272 40.80882353
283
+ 273 40.65934066
284
+ 274 40.51094891
285
+ 275 40.36363636
286
+ 276 40.57971014
287
+ 277 40.79422383
288
+ 278 41.00719424
289
+ 279 40.86021505
290
+ 280 41.07142857
291
+ 281 40.92526690
292
+ 282 40.78014184
293
+ 283 40.63604240
294
+ 284 40.49295775
295
+ 285 40.35087719
296
+ 286 40.20979021
297
+ 287 40.41811847
298
+ 288 40.27777778
299
+ 289 40.13840830
300
+ 290 40.34482759
301
+ 291 40.54982818
302
+ 292 40.75342466
303
+ 293 40.95563140
304
+ 294 40.81632653
305
+ 295 40.67796610
306
+ 296 40.54054054
307
+ 297 40.40404040
308
+ 298 40.60402685
309
+ 299 40.80267559
310
+ 300 41.00000000
311
+ 301 40.86378738
312
+ 302 41.05960265
313
+ 303 40.92409241
314
+ 304 40.78947368
315
+ 305 40.98360656
316
+ 306 40.84967320
317
+ 307 41.04234528
318
+ 308 40.90909091
319
+ 309 40.77669903
320
+ 310 40.96774194
321
+ 311 41.15755627
322
+ 312 41.34615385
323
+ 313 41.53354633
324
+ 314 41.71974522
325
+ 315 41.90476190
326
+ 316 41.77215190
327
+ 317 41.64037855
328
+ 318 41.50943396
329
+ 319 41.69278997
330
+ 320 41.56250000
331
+ 321 41.43302181
332
+ 322 41.30434783
333
+ 323 41.17647059
334
+ 324 41.04938272
335
+ 325 41.23076923
336
+ 326 41.10429448
337
+ 327 41.28440367
338
+ 328 41.46341463
339
+ 329 41.33738602
340
+ 330 41.21212121
341
+ 331 41.08761329
342
+ 332 41.26506024
343
+ 333 41.14114114
344
+ 334 41.01796407
345
+ 335 41.19402985
346
+ 336 41.36904762
347
+ 337 41.24629080
348
+ 338 41.12426036
349
+ 339 41.00294985
350
+ 340 41.17647059
351
+ 341 41.34897361
352
+ 342 41.52046784
353
+ 343 41.39941691
354
+ 344 41.27906977
355
+ 345 41.15942029
356
+ 346 41.32947977
357
+ 347 41.49855908
358
+ 348 41.37931034
359
+ 349 41.54727794
360
+ 350 41.71428571
361
+ 351 41.59544160
362
+ 352 41.47727273
363
+ 353 41.64305949
364
+ 354 41.52542373
365
+ 355 41.40845070
366
+ 356 41.57303371
367
+ 357 41.73669468
368
+ 358 41.89944134
369
+ 359 41.78272981
370
+ 360 41.94444444
371
+ 361 41.82825485
372
+ 362 41.98895028
373
+ 363 41.87327824
374
+ 364 41.75824176
375
+ 365 41.64383562
376
+ 366 41.80327869
377
+ 367 41.68937330
378
+ 368 41.57608696
379
+ 369 41.46341463
380
+ 370 41.62162162
381
+ 371 41.77897574
382
+ 372 41.66666667
383
+ 373 41.55495979
384
+ 374 41.71122995
385
+ 375 41.86666667
386
+ 376 41.75531915
387
+ 377 41.64456233
388
+ 378 41.79894180
389
+ 379 41.68865435
390
+ 380 41.57894737
391
+ 381 41.73228346
392
+ 382 41.88481675
393
+ 383 42.03655352
394
+ 384 41.92708333
395
+ 385 42.07792208
396
+ 386 41.96891192
397
+ 387 41.86046512
398
+ 388 41.75257732
399
+ 389 41.90231362
400
+ 390 42.05128205
401
+ 391 42.19948849
402
+ 392 42.34693878
403
+ 393 42.23918575
404
+ 394 42.13197970
405
+ 395 42.27848101
406
+ 396 42.42424242
407
+ 397 42.56926952
408
+ 398 42.46231156
409
+ 399 42.60651629
410
+ 400 42.50000000
411
+ 401 42.64339152
412
+ 402 42.53731343
413
+ 403 42.43176179
414
+ 404 42.32673267
415
+ 405 42.22222222
416
+ 406 42.11822660
417
+ 407 42.01474201
418
+ 408 41.91176471
419
+ 409 42.05378973
420
+ 410 41.95121951
421
+ 411 41.84914842
422
+ 412 41.74757282
423
+ 413 41.64648910
424
+ 414 41.78743961
425
+ 415 41.68674699
426
+ 416 41.58653846
427
+ 417 41.48681055
428
+ 418 41.38755981
429
+ 419 41.52744630
430
+ 420 41.66666667
431
+ 421 41.56769596
432
+ 422 41.46919431
433
+ 423 41.37115839
434
+ 424 41.50943396
435
+ 425 41.64705882
436
+ 426 41.78403756
437
+ 427 41.68618267
438
+ 428 41.58878505
439
+ 429 41.72494172
440
+ 430 41.62790698
441
+ 431 41.76334107
442
+ 432 41.89814815
443
+ 433 41.80138568
444
+ 434 41.70506912
445
+ 435 41.60919540
446
+ 436 41.51376147
447
+ 437 41.41876430
448
+ 438 41.55251142
449
+ 439 41.68564920
450
+ 440 41.59090909
451
+ 441 41.49659864
452
+ 442 41.40271493
453
+ 443 41.30925508
454
+ 444 41.21621622
455
+ 445 41.12359551
456
+ 446 41.03139013
457
+ 447 40.93959732
458
+ 448 40.84821429
459
+ 449 40.75723831
460
+ 450 40.88888889
461
+ 451 40.79822616
462
+ 452 40.92920354
463
+ 453 41.05960265
464
+ 454 40.96916300
465
+ 455 40.87912088
466
+ 456 41.00877193
467
+ 457 41.13785558
468
+ 458 41.04803493
469
+ 459 40.95860566
470
+ 460 41.08695652
471
+ 461 40.99783080
472
+ 462 40.90909091
473
+ 463 40.82073434
474
+ 464 40.94827586
475
+ 465 40.86021505
476
+ 466 40.77253219
477
+ 467 40.89935760
478
+ 468 40.81196581
479
+ 469 40.93816631
480
+ 470 40.85106383
481
+ 471 40.76433121
482
+ 472 40.67796610
483
+ 473 40.59196617
484
+ 474 40.50632911
485
+ 475 40.42105263
486
+ 476 40.54621849
487
+ 477 40.46121593
488
+ 478 40.58577406
489
+ 479 40.70981211
490
+ 480 40.83333333
491
+ 481 40.74844075
492
+ 482 40.66390041
493
+ 483 40.57971014
494
+ 484 40.49586777
495
+ 485 40.41237113
496
+ 486 40.32921811
497
+ 487 40.45174538
498
+ 488 40.36885246
499
+ 489 40.28629857
500
+ 490 40.40816327
501
+ 491 40.32586558
502
+ 492 40.24390244
503
+ 493 40.16227181
504
+ 494 40.28340081
505
+ 495 40.40404040
506
+ 496 40.52419355
507
+ 497 40.44265594
508
+ 498 40.36144578
509
+ 499 40.28056112
510
+ 500 40.20000000
511
+ 501 40.11976048
512
+ 502 40.03984064
513
+ 503 39.96023857
514
+ 504 40.07936508
515
+ 505 40.00000000
516
+ 506 40.11857708
517
+ 507 40.03944773
518
+ 508 40.15748031
519
+ 509 40.07858546
520
+ 510 40.19607843
521
+ 511 40.11741683
522
+ 512 40.03906250
523
+ 513 39.96101365
524
+ 514 39.88326848
525
+ 515 39.80582524
526
+ 516 39.92248062
527
+ 517 40.03868472
528
+ 518 40.15444015
529
+ 519 40.26974952
530
+ 520 40.38461538
531
+ 521 40.30710173
532
+ 522 40.22988506
533
+ 523 40.15296367
534
+ 524 40.26717557
535
+ 525 40.19047619
536
+ 526 40.11406844
537
+ 527 40.22770398
538
+ 528 40.15151515
539
+ 529 40.07561437
540
+ 530 40.18867925
541
+ 531 40.11299435
542
+ 532 40.03759398
543
+ 533 39.96247655
544
+ 534 39.88764045
545
+ 535 40.00000000
546
+ 536 39.92537313
547
+ 537 40.03724395
548
+ 538 40.14869888
549
+ 539 40.07421150
550
+ 540 40.00000000
551
+ 541 39.92606285
552
+ 542 40.03690037
553
+ 543 39.96316759
554
+ 544 39.88970588
555
+ 545 39.81651376
556
+ 546 39.74358974
557
+ 547 39.85374771
558
+ 548 39.96350365
559
+ 549 39.89071038
560
+ 550 39.81818182
561
+ 551 39.92740472
562
+ 552 39.85507246
563
+ 553 39.78300181
564
+ 554 39.89169675
565
+ 555 39.81981982
566
+ 556 39.92805755
567
+ 557 39.85637343
568
+ 558 39.96415771
569
+ 559 39.89266547
570
+ 560 39.82142857
571
+ 561 39.75044563
572
+ 562 39.85765125
573
+ 563 39.96447602
574
+ 564 39.89361702
575
+ 565 40.00000000
576
+ 566 39.92932862
577
+ 567 39.85890653
578
+ 568 39.96478873
579
+ 569 39.89455185
580
+ 570 40.00000000
581
+ 571 39.92994746
582
+ 572 40.03496503
583
+ 573 39.96509599
584
+ 574 39.89547038
585
+ 575 39.82608696
586
+ 576 39.93055556
587
+ 577 40.03466205
588
+ 578 39.96539792
589
+ 579 39.89637306
590
+ 580 40.00000000
591
+ 581 39.93115318
592
+ 582 39.86254296
593
+ 583 39.96569468
594
+ 584 40.06849315
595
+ 585 40.00000000
596
+ 586 40.10238908
597
+ 587 40.03407155
598
+ 588 39.96598639
599
+ 589 39.89813243
600
+ 590 39.83050847
601
+ 591 39.76311337
602
+ 592 39.69594595
603
+ 593 39.62900506
604
+ 594 39.73063973
605
+ 595 39.66386555
606
+ 596 39.76510067
607
+ 597 39.69849246
608
+ 598 39.63210702
609
+ 599 39.73288815
610
+ 600 39.66666667
611
+ 601 39.76705491
612
+ 602 39.70099668
613
+ 603 39.80099502
614
+ 604 39.73509934
615
+ 605 39.83471074
616
+ 606 39.76897690
617
+ 607 39.86820428
618
+ 608 39.80263158
619
+ 609 39.73727422
620
+ 610 39.67213115
621
+ 611 39.77086743
622
+ 612 39.70588235
623
+ 613 39.64110930
624
+ 614 39.73941368
625
+ 615 39.67479675
626
+ 616 39.77272727
627
+ 617 39.70826580
628
+ 618 39.80582524
629
+ 619 39.74151858
630
+ 620 39.67741935
631
+ 621 39.61352657
632
+ 622 39.71061093
633
+ 623 39.64686998
634
+ 624 39.58333333
635
+ 625 39.52000000
636
+ 626 39.61661342
637
+ 627 39.55342903
638
+ 628 39.49044586
639
+ 629 39.58664547
640
+ 630 39.52380952
641
+ 631 39.61965135
642
+ 632 39.71518987
643
+ 633 39.65244866
644
+ 634 39.58990536
645
+ 635 39.52755906
646
+ 636 39.46540881
647
+ 637 39.56043956
648
+ 638 39.49843260
649
+ 639 39.59311424
650
+ 640 39.68750000
651
+ 641 39.62558502
652
+ 642 39.71962617
653
+ 643 39.65785381
654
+ 644 39.75155280
655
+ 645 39.68992248
656
+ 646 39.62848297
657
+ 647 39.56723338
658
+ 648 39.50617284
659
+ 649 39.44530046
660
+ 650 39.38461538
661
+ 651 39.32411674
662
+ 652 39.41717791
663
+ 653 39.35681470
664
+ 654 39.44954128
665
+ 655 39.38931298
666
+ 656 39.48170732
667
+ 657 39.42161339
668
+ 658 39.51367781
669
+ 659 39.45371775
670
+ 660 39.39393939
671
+ 661 39.33434191
672
+ 662 39.27492447
673
+ 663 39.21568627
674
+ 664 39.30722892
675
+ 665 39.39849624
676
+ 666 39.33933934
677
+ 667 39.43028486
678
+ 668 39.52095808
679
+ 669 39.61136024
680
+ 670 39.55223881
681
+ 671 39.49329359
682
+ 672 39.43452381
683
+ 673 39.37592868
684
+ 674 39.31750742
685
+ 675 39.25925926
686
+ 676 39.20118343
687
+ 677 39.14327917
688
+ 678 39.08554572
689
+ 679 39.02798233
690
+ 680 39.11764706
691
+ 681 39.20704846
692
+ 682 39.14956012
693
+ 683 39.09224012
694
+ 684 39.03508772
695
+ 685 38.97810219
696
+ 686 38.92128280
697
+ 687 38.86462882
698
+ 688 38.80813953
699
+ 689 38.75181422
700
+ 690 38.69565217
701
+ 691 38.78437048
702
+ 692 38.72832370
703
+ 693 38.67243867
704
+ 694 38.61671470
705
+ 695 38.56115108
706
+ 696 38.64942529
707
+ 697 38.73744620
708
+ 698 38.68194842
709
+ 699 38.76967096
710
+ 700 38.71428571
711
+ 701 38.80171184
712
+ 702 38.88888889
713
+ 703 38.97581792
714
+ 704 38.92045455
715
+ 705 38.86524823
716
+ 706 38.81019830
717
+ 707 38.75530410
718
+ 708 38.70056497
719
+ 709 38.78702398
720
+ 710 38.87323944
721
+ 711 38.95921238
722
+ 712 38.90449438
723
+ 713 38.99018233
724
+ 714 39.07563025
725
+ 715 39.02097902
726
+ 716 39.10614525
727
+ 717 39.05160391
728
+ 718 38.99721448
729
+ 719 38.94297636
730
+ 720 38.88888889
731
+ 721 38.83495146
732
+ 722 38.91966759
733
+ 723 39.00414938
734
+ 724 38.95027624
735
+ 725 39.03448276
736
+ 726 38.98071625
737
+ 727 38.92709766
738
+ 728 38.87362637
739
+ 729 38.82030178
740
+ 730 38.76712329
741
+ 731 38.71409029
742
+ 732 38.79781421
743
+ 733 38.88130969
744
+ 734 38.96457766
745
+ 735 38.91156463
746
+ 736 38.85869565
747
+ 737 38.94165536
748
+ 738 38.88888889
749
+ 739 38.97158322
750
+ 740 38.91891892
751
+ 741 38.86639676
752
+ 742 38.94878706
753
+ 743 38.89636608
754
+ 744 38.97849462
755
+ 745 38.92617450
756
+ 746 38.87399464
757
+ 747 38.82195448
758
+ 748 38.77005348
759
+ 749 38.71829105
760
+ 750 38.80000000
761
+
762
+ Final result: 38.8000 +/- 1.7805
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-q4_k_m-naive.tqa ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 817 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 817 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 25.00000000
15
+ 5 20.00000000
16
+ 6 16.66666667
17
+ 7 14.28571429
18
+ 8 12.50000000
19
+ 9 11.11111111
20
+ 10 10.00000000
21
+ 11 9.09090909
22
+ 12 16.66666667
23
+ 13 23.07692308
24
+ 14 21.42857143
25
+ 15 20.00000000
26
+ 16 18.75000000
27
+ 17 17.64705882
28
+ 18 16.66666667
29
+ 19 21.05263158
30
+ 20 20.00000000
31
+ 21 19.04761905
32
+ 22 18.18181818
33
+ 23 17.39130435
34
+ 24 20.83333333
35
+ 25 20.00000000
36
+ 26 19.23076923
37
+ 27 18.51851852
38
+ 28 17.85714286
39
+ 29 20.68965517
40
+ 30 20.00000000
41
+ 31 19.35483871
42
+ 32 21.87500000
43
+ 33 24.24242424
44
+ 34 23.52941176
45
+ 35 22.85714286
46
+ 36 22.22222222
47
+ 37 24.32432432
48
+ 38 23.68421053
49
+ 39 25.64102564
50
+ 40 25.00000000
51
+ 41 24.39024390
52
+ 42 23.80952381
53
+ 43 25.58139535
54
+ 44 25.00000000
55
+ 45 26.66666667
56
+ 46 26.08695652
57
+ 47 25.53191489
58
+ 48 25.00000000
59
+ 49 24.48979592
60
+ 50 26.00000000
61
+ 51 27.45098039
62
+ 52 26.92307692
63
+ 53 26.41509434
64
+ 54 27.77777778
65
+ 55 29.09090909
66
+ 56 28.57142857
67
+ 57 28.07017544
68
+ 58 27.58620690
69
+ 59 27.11864407
70
+ 60 26.66666667
71
+ 61 26.22950820
72
+ 62 27.41935484
73
+ 63 26.98412698
74
+ 64 26.56250000
75
+ 65 26.15384615
76
+ 66 25.75757576
77
+ 67 25.37313433
78
+ 68 25.00000000
79
+ 69 26.08695652
80
+ 70 27.14285714
81
+ 71 26.76056338
82
+ 72 27.77777778
83
+ 73 27.39726027
84
+ 74 27.02702703
85
+ 75 26.66666667
86
+ 76 26.31578947
87
+ 77 25.97402597
88
+ 78 25.64102564
89
+ 79 25.31645570
90
+ 80 26.25000000
91
+ 81 25.92592593
92
+ 82 26.82926829
93
+ 83 26.50602410
94
+ 84 26.19047619
95
+ 85 25.88235294
96
+ 86 25.58139535
97
+ 87 25.28735632
98
+ 88 25.00000000
99
+ 89 25.84269663
100
+ 90 25.55555556
101
+ 91 26.37362637
102
+ 92 26.08695652
103
+ 93 25.80645161
104
+ 94 25.53191489
105
+ 95 26.31578947
106
+ 96 27.08333333
107
+ 97 26.80412371
108
+ 98 26.53061224
109
+ 99 27.27272727
110
+ 100 28.00000000
111
+ 101 28.71287129
112
+ 102 28.43137255
113
+ 103 28.15533981
114
+ 104 27.88461538
115
+ 105 27.61904762
116
+ 106 27.35849057
117
+ 107 28.03738318
118
+ 108 28.70370370
119
+ 109 28.44036697
120
+ 110 29.09090909
121
+ 111 28.82882883
122
+ 112 28.57142857
123
+ 113 28.31858407
124
+ 114 28.94736842
125
+ 115 28.69565217
126
+ 116 28.44827586
127
+ 117 28.20512821
128
+ 118 27.96610169
129
+ 119 27.73109244
130
+ 120 28.33333333
131
+ 121 28.92561983
132
+ 122 28.68852459
133
+ 123 29.26829268
134
+ 124 29.03225806
135
+ 125 29.60000000
136
+ 126 29.36507937
137
+ 127 29.13385827
138
+ 128 28.90625000
139
+ 129 28.68217054
140
+ 130 28.46153846
141
+ 131 28.24427481
142
+ 132 28.03030303
143
+ 133 27.81954887
144
+ 134 28.35820896
145
+ 135 28.88888889
146
+ 136 28.67647059
147
+ 137 29.19708029
148
+ 138 29.71014493
149
+ 139 30.21582734
150
+ 140 30.00000000
151
+ 141 29.78723404
152
+ 142 29.57746479
153
+ 143 29.37062937
154
+ 144 29.86111111
155
+ 145 29.65517241
156
+ 146 29.45205479
157
+ 147 29.25170068
158
+ 148 29.05405405
159
+ 149 28.85906040
160
+ 150 28.66666667
161
+ 151 29.13907285
162
+ 152 28.94736842
163
+ 153 28.75816993
164
+ 154 28.57142857
165
+ 155 29.03225806
166
+ 156 29.48717949
167
+ 157 29.29936306
168
+ 158 29.74683544
169
+ 159 29.55974843
170
+ 160 29.37500000
171
+ 161 29.19254658
172
+ 162 29.01234568
173
+ 163 28.83435583
174
+ 164 28.65853659
175
+ 165 29.09090909
176
+ 166 29.51807229
177
+ 167 29.34131737
178
+ 168 29.16666667
179
+ 169 28.99408284
180
+ 170 28.82352941
181
+ 171 28.65497076
182
+ 172 28.48837209
183
+ 173 28.32369942
184
+ 174 28.73563218
185
+ 175 28.57142857
186
+ 176 28.40909091
187
+ 177 28.24858757
188
+ 178 28.08988764
189
+ 179 28.49162011
190
+ 180 28.33333333
191
+ 181 28.17679558
192
+ 182 28.02197802
193
+ 183 27.86885246
194
+ 184 27.71739130
195
+ 185 27.56756757
196
+ 186 27.95698925
197
+ 187 27.80748663
198
+ 188 28.19148936
199
+ 189 28.57142857
200
+ 190 28.42105263
201
+ 191 28.27225131
202
+ 192 28.64583333
203
+ 193 28.49740933
204
+ 194 28.86597938
205
+ 195 28.71794872
206
+ 196 28.57142857
207
+ 197 28.93401015
208
+ 198 29.29292929
209
+ 199 29.14572864
210
+ 200 29.00000000
211
+ 201 28.85572139
212
+ 202 29.20792079
213
+ 203 29.06403941
214
+ 204 28.92156863
215
+ 205 28.78048780
216
+ 206 28.64077670
217
+ 207 28.98550725
218
+ 208 28.84615385
219
+ 209 28.70813397
220
+ 210 28.57142857
221
+ 211 28.43601896
222
+ 212 28.30188679
223
+ 213 28.16901408
224
+ 214 28.03738318
225
+ 215 27.90697674
226
+ 216 28.24074074
227
+ 217 28.11059908
228
+ 218 28.44036697
229
+ 219 28.31050228
230
+ 220 28.18181818
231
+ 221 28.05429864
232
+ 222 27.92792793
233
+ 223 27.80269058
234
+ 224 28.12500000
235
+ 225 28.00000000
236
+ 226 27.87610619
237
+ 227 28.19383260
238
+ 228 28.07017544
239
+ 229 27.94759825
240
+ 230 27.82608696
241
+ 231 27.70562771
242
+ 232 28.01724138
243
+ 233 28.32618026
244
+ 234 28.63247863
245
+ 235 28.51063830
246
+ 236 28.81355932
247
+ 237 29.11392405
248
+ 238 28.99159664
249
+ 239 28.87029289
250
+ 240 28.75000000
251
+ 241 28.63070539
252
+ 242 28.51239669
253
+ 243 28.39506173
254
+ 244 28.27868852
255
+ 245 28.16326531
256
+ 246 28.04878049
257
+ 247 27.93522267
258
+ 248 27.82258065
259
+ 249 27.71084337
260
+ 250 27.60000000
261
+ 251 27.88844622
262
+ 252 27.77777778
263
+ 253 27.66798419
264
+ 254 27.55905512
265
+ 255 27.84313725
266
+ 256 27.73437500
267
+ 257 27.62645914
268
+ 258 27.51937984
269
+ 259 27.41312741
270
+ 260 27.30769231
271
+ 261 27.58620690
272
+ 262 27.48091603
273
+ 263 27.37642586
274
+ 264 27.65151515
275
+ 265 27.92452830
276
+ 266 27.81954887
277
+ 267 28.08988764
278
+ 268 27.98507463
279
+ 269 27.88104089
280
+ 270 27.77777778
281
+ 271 27.67527675
282
+ 272 27.57352941
283
+ 273 27.47252747
284
+ 274 27.73722628
285
+ 275 27.63636364
286
+ 276 27.53623188
287
+ 277 27.43682310
288
+ 278 27.33812950
289
+ 279 27.24014337
290
+ 280 27.50000000
291
+ 281 27.75800712
292
+ 282 27.65957447
293
+ 283 27.56183746
294
+ 284 27.81690141
295
+ 285 27.71929825
296
+ 286 27.97202797
297
+ 287 28.22299652
298
+ 288 28.12500000
299
+ 289 28.02768166
300
+ 290 27.93103448
301
+ 291 28.17869416
302
+ 292 28.08219178
303
+ 293 28.32764505
304
+ 294 28.23129252
305
+ 295 28.47457627
306
+ 296 28.71621622
307
+ 297 28.95622896
308
+ 298 28.85906040
309
+ 299 28.76254181
310
+ 300 28.66666667
311
+ 301 28.57142857
312
+ 302 28.47682119
313
+ 303 28.38283828
314
+ 304 28.28947368
315
+ 305 28.19672131
316
+ 306 28.10457516
317
+ 307 28.01302932
318
+ 308 28.24675325
319
+ 309 28.15533981
320
+ 310 28.06451613
321
+ 311 27.97427653
322
+ 312 28.20512821
323
+ 313 28.11501597
324
+ 314 28.34394904
325
+ 315 28.25396825
326
+ 316 28.16455696
327
+ 317 28.07570978
328
+ 318 27.98742138
329
+ 319 27.89968652
330
+ 320 27.81250000
331
+ 321 27.72585670
332
+ 322 27.95031056
333
+ 323 27.86377709
334
+ 324 27.77777778
335
+ 325 27.69230769
336
+ 326 27.60736196
337
+ 327 27.82874618
338
+ 328 28.04878049
339
+ 329 27.96352584
340
+ 330 28.18181818
341
+ 331 28.09667674
342
+ 332 28.31325301
343
+ 333 28.22822823
344
+ 334 28.14371257
345
+ 335 28.35820896
346
+ 336 28.57142857
347
+ 337 28.48664688
348
+ 338 28.40236686
349
+ 339 28.61356932
350
+ 340 28.52941176
351
+ 341 28.73900293
352
+ 342 28.65497076
353
+ 343 28.57142857
354
+ 344 28.77906977
355
+ 345 28.98550725
356
+ 346 29.19075145
357
+ 347 29.39481268
358
+ 348 29.31034483
359
+ 349 29.22636103
360
+ 350 29.14285714
361
+ 351 29.05982906
362
+ 352 28.97727273
363
+ 353 28.89518414
364
+ 354 28.81355932
365
+ 355 29.01408451
366
+ 356 28.93258427
367
+ 357 29.13165266
368
+ 358 29.32960894
369
+ 359 29.24791086
370
+ 360 29.16666667
371
+ 361 29.36288089
372
+ 362 29.28176796
373
+ 363 29.47658402
374
+ 364 29.39560440
375
+ 365 29.58904110
376
+ 366 29.50819672
377
+ 367 29.70027248
378
+ 368 29.61956522
379
+ 369 29.53929539
380
+ 370 29.45945946
381
+ 371 29.38005391
382
+ 372 29.56989247
383
+ 373 29.49061662
384
+ 374 29.67914439
385
+ 375 29.60000000
386
+ 376 29.52127660
387
+ 377 29.44297082
388
+ 378 29.36507937
389
+ 379 29.28759894
390
+ 380 29.47368421
391
+ 381 29.39632546
392
+ 382 29.58115183
393
+ 383 29.50391645
394
+ 384 29.42708333
395
+ 385 29.35064935
396
+ 386 29.27461140
397
+ 387 29.19896641
398
+ 388 29.38144330
399
+ 389 29.30591260
400
+ 390 29.23076923
401
+ 391 29.15601023
402
+ 392 29.08163265
403
+ 393 29.26208651
404
+ 394 29.44162437
405
+ 395 29.36708861
406
+ 396 29.29292929
407
+ 397 29.47103275
408
+ 398 29.39698492
409
+ 399 29.32330827
410
+ 400 29.25000000
411
+ 401 29.17705736
412
+ 402 29.35323383
413
+ 403 29.52853598
414
+ 404 29.45544554
415
+ 405 29.38271605
416
+ 406 29.55665025
417
+ 407 29.48402948
418
+ 408 29.41176471
419
+ 409 29.33985330
420
+ 410 29.26829268
421
+ 411 29.19708029
422
+ 412 29.36893204
423
+ 413 29.29782082
424
+ 414 29.46859903
425
+ 415 29.63855422
426
+ 416 29.56730769
427
+ 417 29.49640288
428
+ 418 29.42583732
429
+ 419 29.35560859
430
+ 420 29.52380952
431
+ 421 29.69121140
432
+ 422 29.62085308
433
+ 423 29.55082742
434
+ 424 29.48113208
435
+ 425 29.41176471
436
+ 426 29.34272300
437
+ 427 29.27400468
438
+ 428 29.20560748
439
+ 429 29.37062937
440
+ 430 29.30232558
441
+ 431 29.23433875
442
+ 432 29.39814815
443
+ 433 29.56120092
444
+ 434 29.72350230
445
+ 435 29.65517241
446
+ 436 29.58715596
447
+ 437 29.51945080
448
+ 438 29.45205479
449
+ 439 29.38496583
450
+ 440 29.31818182
451
+ 441 29.25170068
452
+ 442 29.18552036
453
+ 443 29.11963883
454
+ 444 29.27927928
455
+ 445 29.21348315
456
+ 446 29.14798206
457
+ 447 29.08277405
458
+ 448 29.01785714
459
+ 449 29.17594655
460
+ 450 29.33333333
461
+ 451 29.26829268
462
+ 452 29.20353982
463
+ 453 29.13907285
464
+ 454 29.29515419
465
+ 455 29.23076923
466
+ 456 29.16666667
467
+ 457 29.10284464
468
+ 458 29.03930131
469
+ 459 28.97603486
470
+ 460 28.91304348
471
+ 461 28.85032538
472
+ 462 28.78787879
473
+ 463 28.94168467
474
+ 464 29.09482759
475
+ 465 29.24731183
476
+ 466 29.18454936
477
+ 467 29.12205567
478
+ 468 29.27350427
479
+ 469 29.21108742
480
+ 470 29.14893617
481
+ 471 29.29936306
482
+ 472 29.23728814
483
+ 473 29.17547569
484
+ 474 29.11392405
485
+ 475 29.26315789
486
+ 476 29.20168067
487
+ 477 29.35010482
488
+ 478 29.28870293
489
+ 479 29.22755741
490
+ 480 29.16666667
491
+ 481 29.10602911
492
+ 482 29.25311203
493
+ 483 29.19254658
494
+ 484 29.33884298
495
+ 485 29.48453608
496
+ 486 29.42386831
497
+ 487 29.56878850
498
+ 488 29.50819672
499
+ 489 29.65235174
500
+ 490 29.79591837
501
+ 491 29.93890020
502
+ 492 30.08130081
503
+ 493 30.02028398
504
+ 494 29.95951417
505
+ 495 30.10101010
506
+ 496 30.04032258
507
+ 497 29.97987928
508
+ 498 29.91967871
509
+ 499 30.06012024
510
+ 500 30.00000000
511
+ 501 30.13972056
512
+ 502 30.07968127
513
+ 503 30.01988072
514
+ 504 30.15873016
515
+ 505 30.09900990
516
+ 506 30.03952569
517
+ 507 30.17751479
518
+ 508 30.31496063
519
+ 509 30.45186640
520
+ 510 30.58823529
521
+ 511 30.52837573
522
+ 512 30.46875000
523
+ 513 30.40935673
524
+ 514 30.35019455
525
+ 515 30.48543689
526
+ 516 30.42635659
527
+ 517 30.56092843
528
+ 518 30.50193050
529
+ 519 30.44315992
530
+ 520 30.38461538
531
+ 521 30.32629559
532
+ 522 30.26819923
533
+ 523 30.21032505
534
+ 524 30.15267176
535
+ 525 30.09523810
536
+ 526 30.03802281
537
+ 527 30.17077799
538
+ 528 30.11363636
539
+ 529 30.05671078
540
+ 530 30.00000000
541
+ 531 29.94350282
542
+ 532 29.88721805
543
+ 533 30.01876173
544
+ 534 29.96254682
545
+ 535 29.90654206
546
+ 536 29.85074627
547
+ 537 29.98137803
548
+ 538 29.92565056
549
+ 539 29.87012987
550
+ 540 29.81481481
551
+ 541 29.75970425
552
+ 542 29.70479705
553
+ 543 29.65009208
554
+ 544 29.59558824
555
+ 545 29.54128440
556
+ 546 29.67032967
557
+ 547 29.79890311
558
+ 548 29.92700730
559
+ 549 29.87249545
560
+ 550 29.81818182
561
+ 551 29.76406534
562
+ 552 29.89130435
563
+ 553 30.01808318
564
+ 554 29.96389892
565
+ 555 29.90990991
566
+ 556 29.85611511
567
+ 557 29.80251346
568
+ 558 29.74910394
569
+ 559 29.69588551
570
+ 560 29.82142857
571
+ 561 29.94652406
572
+ 562 29.89323843
573
+ 563 29.84014210
574
+ 564 29.96453901
575
+ 565 30.08849558
576
+ 566 30.21201413
577
+ 567 30.15873016
578
+ 568 30.28169014
579
+ 569 30.40421793
580
+ 570 30.35087719
581
+ 571 30.29772329
582
+ 572 30.24475524
583
+ 573 30.19197208
584
+ 574 30.31358885
585
+ 575 30.43478261
586
+ 576 30.38194444
587
+ 577 30.32928943
588
+ 578 30.44982699
589
+ 579 30.39723661
590
+ 580 30.51724138
591
+ 581 30.46471601
592
+ 582 30.58419244
593
+ 583 30.53173242
594
+ 584 30.65068493
595
+ 585 30.59829060
596
+ 586 30.71672355
597
+ 587 30.66439523
598
+ 588 30.61224490
599
+ 589 30.56027165
600
+ 590 30.67796610
601
+ 591 30.62605753
602
+ 592 30.57432432
603
+ 593 30.52276560
604
+ 594 30.47138047
605
+ 595 30.42016807
606
+ 596 30.36912752
607
+ 597 30.31825796
608
+ 598 30.26755853
609
+ 599 30.21702838
610
+ 600 30.16666667
611
+ 601 30.28286190
612
+ 602 30.23255814
613
+ 603 30.34825871
614
+ 604 30.29801325
615
+ 605 30.24793388
616
+ 606 30.36303630
617
+ 607 30.31301483
618
+ 608 30.26315789
619
+ 609 30.21346470
620
+ 610 30.16393443
621
+ 611 30.11456628
622
+ 612 30.22875817
623
+ 613 30.17944535
624
+ 614 30.13029316
625
+ 615 30.08130081
626
+ 616 30.19480519
627
+ 617 30.30794165
628
+ 618 30.25889968
629
+ 619 30.37156704
630
+ 620 30.32258065
631
+ 621 30.27375201
632
+ 622 30.38585209
633
+ 623 30.33707865
634
+ 624 30.28846154
635
+ 625 30.24000000
636
+ 626 30.19169329
637
+ 627 30.30303030
638
+ 628 30.25477707
639
+ 629 30.20667727
640
+ 630 30.15873016
641
+ 631 30.26941363
642
+ 632 30.22151899
643
+ 633 30.17377567
644
+ 634 30.28391167
645
+ 635 30.23622047
646
+ 636 30.18867925
647
+ 637 30.14128728
648
+ 638 30.09404389
649
+ 639 30.04694836
650
+ 640 30.00000000
651
+ 641 29.95319813
652
+ 642 30.06230530
653
+ 643 30.01555210
654
+ 644 29.96894410
655
+ 645 30.07751938
656
+ 646 30.03095975
657
+ 647 29.98454405
658
+ 648 30.09259259
659
+ 649 30.04622496
660
+ 650 30.15384615
661
+ 651 30.26113671
662
+ 652 30.36809816
663
+ 653 30.47473201
664
+ 654 30.42813456
665
+ 655 30.38167939
666
+ 656 30.48780488
667
+ 657 30.59360731
668
+ 658 30.54711246
669
+ 659 30.65250379
670
+ 660 30.60606061
671
+ 661 30.55975794
672
+ 662 30.51359517
673
+ 663 30.46757164
674
+ 664 30.42168675
675
+ 665 30.37593985
676
+ 666 30.48048048
677
+ 667 30.43478261
678
+ 668 30.38922156
679
+ 669 30.34379671
680
+ 670 30.44776119
681
+ 671 30.40238450
682
+ 672 30.35714286
683
+ 673 30.31203566
684
+ 674 30.26706231
685
+ 675 30.37037037
686
+ 676 30.47337278
687
+ 677 30.42836041
688
+ 678 30.38348083
689
+ 679 30.33873343
690
+ 680 30.29411765
691
+ 681 30.24963289
692
+ 682 30.20527859
693
+ 683 30.16105417
694
+ 684 30.26315789
695
+ 685 30.21897810
696
+ 686 30.17492711
697
+ 687 30.13100437
698
+ 688 30.08720930
699
+ 689 30.18867925
700
+ 690 30.14492754
701
+ 691 30.24602026
702
+ 692 30.34682081
703
+ 693 30.44733045
704
+ 694 30.40345821
705
+ 695 30.35971223
706
+ 696 30.45977011
707
+ 697 30.41606887
708
+ 698 30.37249284
709
+ 699 30.32904149
710
+ 700 30.28571429
711
+ 701 30.24251070
712
+ 702 30.19943020
713
+ 703 30.15647226
714
+ 704 30.11363636
715
+ 705 30.07092199
716
+ 706 30.02832861
717
+ 707 30.12729844
718
+ 708 30.08474576
719
+ 709 30.04231312
720
+ 710 30.00000000
721
+ 711 29.95780591
722
+ 712 29.91573034
723
+ 713 29.87377279
724
+ 714 29.97198880
725
+ 715 30.06993007
726
+ 716 30.02793296
727
+ 717 29.98605300
728
+ 718 29.94428969
729
+ 719 29.90264256
730
+ 720 29.86111111
731
+ 721 29.81969487
732
+ 722 29.77839335
733
+ 723 29.73720609
734
+ 724 29.83425414
735
+ 725 29.79310345
736
+ 726 29.88980716
737
+ 727 29.84869326
738
+ 728 29.80769231
739
+ 729 29.90397805
740
+ 730 29.86301370
741
+ 731 29.82216142
742
+ 732 29.91803279
743
+ 733 29.87721692
744
+ 734 29.83651226
745
+ 735 29.79591837
746
+ 736 29.75543478
747
+ 737 29.85074627
748
+ 738 29.81029810
749
+ 739 29.90527740
750
+ 740 29.86486486
751
+ 741 29.95951417
752
+ 742 29.91913747
753
+ 743 29.87886945
754
+ 744 29.97311828
755
+ 745 29.93288591
756
+ 746 29.89276139
757
+ 747 29.98661312
758
+ 748 29.94652406
759
+ 749 30.04005340
760
+ 750 30.13333333
761
+
762
+ Final result: 30.1333 +/- 1.6766
763
+ Random chance: 19.8992 +/- 1.4588
scores/Hammer2.1-7b-q4_k_m-naive.wng ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ winogrande_score : loaded 1267 tasks from prompt.
6
+ winogrande_score : selecting 750 random tasks
7
+ winogrande_score : tokenizing selected tasks
8
+ winogrande_score : calculating winogrande score over selected tasks.
9
+ 1 100.0000 -4.073147 -4.341902 1 1
10
+ 2 100.0000 -2.002059 -2.392350 1 1
11
+ 3 100.0000 -2.806897 -2.357071 2 2
12
+ 4 75.0000 -5.631859 -5.101724 2 1
13
+ 5 60.0000 -2.256425 -3.186967 1 2
14
+ 6 50.0000 -5.195934 -5.140819 2 1
15
+ 7 57.1429 -4.245509 -3.176660 2 2
16
+ 8 62.5000 -2.135587 -2.013082 2 2
17
+ 9 66.6667 -3.411335 -6.202401 1 1
18
+ 10 70.0000 -1.869223 -1.222768 2 2
19
+ 11 72.7273 -10.794676 -5.541762 2 2
20
+ 12 75.0000 -3.865073 -2.960941 2 2
21
+ 13 76.9231 -2.691580 -5.744883 1 1
22
+ 14 78.5714 -3.923702 -4.103228 1 1
23
+ 15 73.3333 -4.297781 -5.646027 1 2
24
+ 16 75.0000 -4.208681 -3.935671 2 2
25
+ 17 70.5882 -6.457009 -7.022897 1 2
26
+ 18 72.2222 -1.235399 -3.177442 1 1
27
+ 19 73.6842 -4.058254 -4.988782 1 1
28
+ 20 75.0000 -2.973456 -3.761214 1 1
29
+ 21 76.1905 -2.419065 -3.072155 1 1
30
+ 22 77.2727 -2.700415 -2.045843 2 2
31
+ 23 78.2609 -4.777004 -4.838593 1 1
32
+ 24 75.0000 -2.933517 -2.728683 2 1
33
+ 25 76.0000 -2.904390 -2.753894 2 2
34
+ 26 76.9231 -3.520569 -4.226878 1 1
35
+ 27 77.7778 -5.252704 -5.082743 2 2
36
+ 28 78.5714 -4.599693 -5.546337 1 1
37
+ 29 79.3103 -4.136139 -4.345783 1 1
38
+ 30 76.6667 -1.295570 -1.689267 1 2
39
+ 31 77.4194 -5.251396 -5.382027 1 1
40
+ 32 78.1250 -4.145416 -6.048861 1 1
41
+ 33 78.7879 -3.315151 -6.532020 1 1
42
+ 34 76.4706 -2.508332 -2.388638 2 1
43
+ 35 77.1429 -4.704243 -4.662181 2 2
44
+ 36 77.7778 -4.219828 -3.357770 2 2
45
+ 37 75.6757 -3.406592 -3.052458 2 1
46
+ 38 76.3158 -3.334883 -3.621847 1 1
47
+ 39 76.9231 -4.563976 -4.229147 2 2
48
+ 40 75.0000 -4.137616 -5.519074 1 2
49
+ 41 75.6098 -2.549767 -2.771647 1 1
50
+ 42 76.1905 -1.776616 -2.334874 1 1
51
+ 43 74.4186 -4.085493 -4.181500 1 2
52
+ 44 75.0000 -1.811443 -1.711734 2 2
53
+ 45 75.5556 -0.074932 -15.186333 1 1
54
+ 46 76.0870 -2.495840 -2.858429 1 1
55
+ 47 74.4681 -3.997855 -4.515547 1 2
56
+ 48 75.0000 -4.947919 -4.596514 2 2
57
+ 49 73.4694 -3.101562 -1.024802 2 1
58
+ 50 72.0000 -2.673930 -2.729897 1 2
59
+ 51 72.5490 -2.205984 -2.605826 1 1
60
+ 52 73.0769 -2.916319 -2.220397 2 2
61
+ 53 73.5849 -4.119981 -5.144504 1 1
62
+ 54 74.0741 -3.375921 -3.017697 2 2
63
+ 55 74.5455 -3.361651 -3.062883 2 2
64
+ 56 73.2143 -4.712955 -5.223645 1 2
65
+ 57 71.9298 -1.681952 -2.846913 1 2
66
+ 58 72.4138 -3.195844 -4.299898 1 1
67
+ 59 72.8814 -3.319516 -3.026950 2 2
68
+ 60 73.3333 -4.907790 -4.521118 2 2
69
+ 61 73.7705 -3.834436 -4.462154 1 1
70
+ 62 74.1935 -6.632193 -6.637777 1 1
71
+ 63 73.0159 -4.033095 -4.874521 1 2
72
+ 64 73.4375 -3.966459 -4.133111 1 1
73
+ 65 73.8462 -4.996745 -4.254751 2 2
74
+ 66 74.2424 -2.423554 -3.624497 1 1
75
+ 67 74.6269 -2.241466 -1.972496 2 2
76
+ 68 73.5294 -4.824273 -4.976226 1 2
77
+ 69 73.9130 -4.413124 -7.828858 1 1
78
+ 70 74.2857 -3.479099 -2.758441 2 2
79
+ 71 73.2394 -3.282046 -3.102822 2 1
80
+ 72 72.2222 -2.950146 -3.225306 1 2
81
+ 73 72.6027 -2.901250 -3.382480 1 1
82
+ 74 72.9730 -2.445247 -2.523356 1 1
83
+ 75 73.3333 -1.389127 -2.162137 1 1
84
+ 76 73.6842 -1.785778 -3.861129 1 1
85
+ 77 72.7273 -2.869407 -3.211731 1 2
86
+ 78 73.0769 -1.074733 -1.593981 1 1
87
+ 79 72.1519 -2.173000 -2.315871 1 2
88
+ 80 72.5000 -2.152046 -3.567008 1 1
89
+ 81 71.6049 -3.335800 -3.542761 1 2
90
+ 82 70.7317 -3.339783 -3.456441 1 2
91
+ 83 69.8795 -6.070676 -6.150465 1 2
92
+ 84 70.2381 -2.455630 -2.367383 2 2
93
+ 85 70.5882 -4.788741 -5.365035 1 1
94
+ 86 69.7674 -5.869349 -6.232488 1 2
95
+ 87 68.9655 -1.336644 -1.112257 2 1
96
+ 88 68.1818 -4.670027 -4.327408 2 1
97
+ 89 68.5393 -2.765202 -2.798308 1 1
98
+ 90 68.8889 -6.160116 -4.249111 2 2
99
+ 91 68.1319 -4.592628 -4.346611 2 1
100
+ 92 68.4783 -4.695715 -3.497276 2 2
101
+ 93 68.8172 -2.384137 -1.784414 2 2
102
+ 94 69.1489 -4.187684 -3.244577 2 2
103
+ 95 69.4737 -5.265181 -6.098378 1 1
104
+ 96 69.7917 -1.870187 -2.880108 1 1
105
+ 97 70.1031 -4.583392 -4.858923 1 1
106
+ 98 70.4082 -4.715110 -3.541696 2 2
107
+ 99 69.6970 -2.834302 -2.206327 2 1
108
+ 100 70.0000 -6.763767 -6.304641 2 2
109
+ 101 69.3069 -4.677106 -4.761773 1 2
110
+ 102 69.6078 -3.416348 -4.547803 1 1
111
+ 103 68.9320 -2.317942 -2.908253 1 2
112
+ 104 69.2308 -8.115113 -8.009356 2 2
113
+ 105 69.5238 -1.218898 -2.229056 1 1
114
+ 106 69.8113 -2.700465 -2.294387 2 2
115
+ 107 70.0935 -3.384711 -2.409758 2 2
116
+ 108 70.3704 -3.855996 -2.780302 2 2
117
+ 109 69.7248 -5.662400 -5.787445 1 2
118
+ 110 70.0000 -4.371898 -3.301455 2 2
119
+ 111 70.2703 -4.286852 -3.717307 2 2
120
+ 112 69.6429 -3.547462 -3.151478 2 1
121
+ 113 69.0265 -4.429124 -4.885850 1 2
122
+ 114 69.2982 -3.699147 -5.004520 1 1
123
+ 115 69.5652 -1.615590 -2.414740 1 1
124
+ 116 69.8276 -3.781786 -2.644974 2 2
125
+ 117 69.2308 -4.297875 -4.641932 1 2
126
+ 118 69.4915 -0.738324 -2.285307 1 1
127
+ 119 69.7479 -2.604891 -2.925564 1 1
128
+ 120 69.1667 -2.622100 -2.510018 2 1
129
+ 121 69.4215 -1.009242 -0.630791 2 2
130
+ 122 69.6721 -6.500463 -4.949875 2 2
131
+ 123 69.9187 -3.051615 -3.422414 1 1
132
+ 124 70.1613 -2.509306 -2.042773 2 2
133
+ 125 70.4000 -3.720369 -2.814873 2 2
134
+ 126 69.8413 -6.509502 -6.813656 1 2
135
+ 127 70.0787 -1.976045 -1.562809 2 2
136
+ 128 70.3125 -1.835500 -2.098227 1 1
137
+ 129 70.5426 -1.707477 -1.301997 2 2
138
+ 130 70.7692 -5.416319 -4.929601 2 2
139
+ 131 70.2290 -3.525012 -3.699178 1 2
140
+ 132 70.4545 -2.721957 -2.408054 2 2
141
+ 133 70.6767 -1.446874 -3.869391 1 1
142
+ 134 70.8955 -5.418740 -4.218716 2 2
143
+ 135 71.1111 -4.182483 -5.105154 1 1
144
+ 136 71.3235 -5.929504 -4.480151 2 2
145
+ 137 70.8029 -6.427297 -6.423068 2 1
146
+ 138 70.2899 -3.667494 -4.185307 1 2
147
+ 139 69.7842 -4.675939 -4.509225 2 1
148
+ 140 70.0000 -1.357779 -1.053311 2 2
149
+ 141 70.2128 -4.127782 -3.364985 2 2
150
+ 142 70.4225 -2.245175 -2.213648 2 2
151
+ 143 70.6294 -1.896816 -5.641869 1 1
152
+ 144 70.1389 -2.017692 -1.933084 2 1
153
+ 145 69.6552 -1.556169 -13.213781 1 2
154
+ 146 69.8630 -4.313636 -4.043294 2 2
155
+ 147 70.0680 -4.786489 -4.678119 2 2
156
+ 148 70.2703 -1.789287 -2.418267 1 1
157
+ 149 70.4698 -4.378870 -3.593705 2 2
158
+ 150 70.6667 -3.902359 -3.031696 2 2
159
+ 151 70.1987 -2.088486 -1.991748 2 1
160
+ 152 69.7368 -3.684043 -3.998914 1 2
161
+ 153 69.9346 -3.488782 -3.760273 1 1
162
+ 154 70.1299 -3.408492 -3.485636 1 1
163
+ 155 69.6774 -2.272924 -2.423034 1 2
164
+ 156 69.2308 -2.477637 -2.372224 2 1
165
+ 157 69.4268 -2.828777 -4.209848 1 1
166
+ 158 69.6203 -4.205874 -3.702442 2 2
167
+ 159 69.8113 -3.634729 -5.301525 1 1
168
+ 160 70.0000 -4.118382 -3.333434 2 2
169
+ 161 70.1863 -2.490234 -2.629851 1 1
170
+ 162 69.7531 -0.852771 -15.167171 1 2
171
+ 163 69.9387 -1.998300 -2.037821 1 1
172
+ 164 70.1220 -4.926808 -4.609169 2 2
173
+ 165 70.3030 -4.407618 -4.327872 2 2
174
+ 166 70.4819 -4.118610 -3.525489 2 2
175
+ 167 70.0599 -4.990454 -4.570775 2 1
176
+ 168 69.6429 -4.865581 -5.190903 1 2
177
+ 169 69.2308 -3.334898 -3.822207 1 2
178
+ 170 68.8235 -4.455070 -3.762394 2 1
179
+ 171 69.0058 -3.142349 -4.413176 1 1
180
+ 172 69.1860 -4.632230 -5.898005 1 1
181
+ 173 69.3642 -3.252964 -2.844198 2 2
182
+ 174 69.5402 -6.592388 -6.138913 2 2
183
+ 175 69.1429 -4.102401 -4.047549 2 1
184
+ 176 69.3182 -1.693508 -3.399051 1 1
185
+ 177 69.4915 -5.937732 -5.452234 2 2
186
+ 178 69.1011 -2.501311 -2.281971 2 1
187
+ 179 69.2737 -2.553092 -2.690606 1 1
188
+ 180 69.4444 -6.045636 -5.471572 2 2
189
+ 181 69.6133 -3.821823 -2.608690 2 2
190
+ 182 69.7802 -3.861176 -3.396381 2 2
191
+ 183 69.9454 -5.933091 -5.651009 2 2
192
+ 184 70.1087 -1.817318 -2.998203 1 1
193
+ 185 69.7297 -5.941071 -5.979753 1 2
194
+ 186 69.8925 -2.885352 -3.425282 1 1
195
+ 187 70.0535 -4.780498 -4.442616 2 2
196
+ 188 69.6809 -3.199900 -2.959458 2 1
197
+ 189 69.3122 -5.091313 -5.560408 1 2
198
+ 190 69.4737 -5.137145 -4.919753 2 2
199
+ 191 69.6335 -3.957209 -3.997639 1 1
200
+ 192 69.7917 -5.859741 -4.251298 2 2
201
+ 193 69.9482 -4.020531 -3.879946 2 2
202
+ 194 70.1031 -1.921878 -1.291763 2 2
203
+ 195 69.7436 -4.258371 -4.026335 2 1
204
+ 196 69.8980 -3.610235 -2.617154 2 2
205
+ 197 70.0508 -0.517077 -2.726156 1 1
206
+ 198 70.2020 -5.602236 -6.153777 1 1
207
+ 199 70.3518 -4.686927 -3.899265 2 2
208
+ 200 70.5000 -3.327579 -2.952970 2 2
209
+ 201 70.6468 -2.751933 -3.426600 1 1
210
+ 202 70.7921 -7.587420 -7.239859 2 2
211
+ 203 70.9360 -3.884003 -3.640086 2 2
212
+ 204 71.0784 -3.047652 -3.394336 1 1
213
+ 205 71.2195 -3.378562 -1.985834 2 2
214
+ 206 70.8738 -5.438685 -5.644970 1 2
215
+ 207 71.0145 -2.683854 -2.981044 1 1
216
+ 208 71.1538 -4.463738 -2.911276 2 2
217
+ 209 71.2919 -5.140670 -4.653047 2 2
218
+ 210 70.9524 -8.099816 -8.861441 1 2
219
+ 211 70.6161 -5.558111 -5.247910 2 1
220
+ 212 70.7547 -4.918854 -3.633807 2 2
221
+ 213 70.4225 -5.109480 -5.653265 1 2
222
+ 214 70.5607 -2.843354 -3.002904 1 1
223
+ 215 70.6977 -5.265980 -5.017216 2 2
224
+ 216 70.3704 -3.680349 -3.393268 2 1
225
+ 217 70.5069 -0.936076 -1.468726 1 1
226
+ 218 70.6422 -2.294581 -2.262097 2 2
227
+ 219 70.7763 -2.650967 -2.956571 1 1
228
+ 220 70.9091 -3.438273 -3.264765 2 2
229
+ 221 71.0407 -2.779260 -8.496936 1 1
230
+ 222 70.7207 -0.366137 -7.575576 1 2
231
+ 223 70.8520 -4.640917 -4.463300 2 2
232
+ 224 70.5357 -5.657567 -6.126117 1 2
233
+ 225 70.6667 -4.079998 -3.029655 2 2
234
+ 226 70.7965 -2.972114 -3.431247 1 1
235
+ 227 70.4846 -3.960191 -3.609150 2 1
236
+ 228 70.6140 -1.425823 -3.059593 1 1
237
+ 229 70.7424 -4.105068 -3.935024 2 2
238
+ 230 70.8696 -3.053642 -3.348650 1 1
239
+ 231 70.9957 -1.988474 -2.423069 1 1
240
+ 232 71.1207 -2.377345 -1.040881 2 2
241
+ 233 71.2446 -1.676222 -2.033520 1 1
242
+ 234 71.3675 -4.062191 -3.148418 2 2
243
+ 235 71.4894 -3.402235 -2.832061 2 2
244
+ 236 71.1864 -2.243631 -1.889005 2 1
245
+ 237 71.3080 -1.352197 -1.667659 1 1
246
+ 238 71.4286 -3.187632 -2.739159 2 2
247
+ 239 71.5481 -2.768013 -2.457495 2 2
248
+ 240 71.2500 -7.423069 -7.854147 1 2
249
+ 241 71.3693 -4.262275 -3.553423 2 2
250
+ 242 71.4876 -6.022777 -5.735852 2 2
251
+ 243 71.1934 -4.928921 -4.721164 2 1
252
+ 244 71.3115 -2.039043 -3.379045 1 1
253
+ 245 71.4286 -3.890713 -5.441935 1 1
254
+ 246 71.5447 -2.387963 -2.818630 1 1
255
+ 247 71.2551 -4.555404 -5.068989 1 2
256
+ 248 71.3710 -3.154646 -2.853739 2 2
257
+ 249 71.4859 -3.418377 -2.161678 2 2
258
+ 250 71.6000 -3.543035 -4.156723 1 1
259
+ 251 71.7131 -0.567401 -1.017910 1 1
260
+ 252 71.8254 -6.527623 -6.235908 2 2
261
+ 253 71.9368 -5.523308 -5.695279 1 1
262
+ 254 72.0472 -4.070900 -2.935887 2 2
263
+ 255 72.1569 -3.096773 -4.054482 1 1
264
+ 256 72.2656 -3.063585 -3.952679 1 1
265
+ 257 71.9844 -2.580292 -1.906935 2 1
266
+ 258 71.7054 -3.322521 -3.634052 1 2
267
+ 259 71.8147 -4.270565 -4.429305 1 1
268
+ 260 71.9231 -4.780110 -5.565026 1 1
269
+ 261 72.0307 -3.805219 -3.658657 2 2
270
+ 262 72.1374 -5.359658 -5.258013 2 2
271
+ 263 72.2433 -4.765828 -6.170121 1 1
272
+ 264 72.3485 -4.258317 -3.930806 2 2
273
+ 265 72.4528 -1.776338 -2.467799 1 1
274
+ 266 72.5564 -5.956629 -7.978909 1 1
275
+ 267 72.6592 -3.484146 -3.571180 1 1
276
+ 268 72.3881 -2.924298 -2.729953 2 1
277
+ 269 72.4907 -3.262499 -3.114777 2 2
278
+ 270 72.5926 -2.171776 -3.753386 1 1
279
+ 271 72.6937 -3.402166 -3.138053 2 2
280
+ 272 72.7941 -4.382628 -3.825091 2 2
281
+ 273 72.8938 -3.835850 -3.531512 2 2
282
+ 274 72.9927 -2.790376 -2.179517 2 2
283
+ 275 73.0909 -1.598326 -2.412901 1 1
284
+ 276 72.8261 -2.262273 -2.125678 2 1
285
+ 277 72.9242 -3.026403 -3.118450 1 1
286
+ 278 72.6619 -2.679073 -2.578459 2 1
287
+ 279 72.7599 -4.697624 -2.937419 2 2
288
+ 280 72.8571 -4.130454 -3.893588 2 2
289
+ 281 72.9537 -2.354938 -2.009870 2 2
290
+ 282 73.0496 -4.037129 -3.716593 2 2
291
+ 283 73.1449 -3.375458 -3.272590 2 2
292
+ 284 72.8873 -2.778238 -2.584545 2 1
293
+ 285 72.9825 -5.340482 -5.638594 1 1
294
+ 286 73.0769 -3.546716 -3.695134 1 1
295
+ 287 73.1707 -2.009729 -4.059923 1 1
296
+ 288 73.2639 -3.713496 -2.526873 2 2
297
+ 289 73.3564 -2.121897 -2.575547 1 1
298
+ 290 73.4483 -5.504008 -6.206664 1 1
299
+ 291 73.5395 -1.666293 -3.177818 1 1
300
+ 292 73.2877 -1.750956 -1.879999 1 2
301
+ 293 73.3788 -3.562386 -2.918724 2 2
302
+ 294 73.1293 -4.389451 -3.084898 2 1
303
+ 295 73.2203 -1.557174 -2.151427 1 1
304
+ 296 73.3108 -4.706651 -4.556388 2 2
305
+ 297 73.0640 -0.743882 -0.925797 1 2
306
+ 298 72.8188 -5.402162 -6.669435 1 2
307
+ 299 72.9097 -2.154193 -1.801132 2 2
308
+ 300 73.0000 -3.006846 -3.794830 1 1
309
+ 301 73.0897 -4.409689 -5.574525 1 1
310
+ 302 72.8477 -4.276526 -4.361559 1 2
311
+ 303 72.6073 -8.689864 -7.535477 2 1
312
+ 304 72.6974 -3.867876 -3.537728 2 2
313
+ 305 72.4590 -2.740989 -2.859037 1 2
314
+ 306 72.5490 -2.580094 -2.323878 2 2
315
+ 307 72.6384 -2.769130 -3.005384 1 1
316
+ 308 72.4026 -4.777669 -4.988991 1 2
317
+ 309 72.4919 -2.570650 -3.218386 1 1
318
+ 310 72.2581 -4.433912 -4.508895 1 2
319
+ 311 72.3473 -1.084782 -4.546499 1 1
320
+ 312 72.1154 -5.136425 -4.056686 2 1
321
+ 313 71.8850 -4.336515 -4.909451 1 2
322
+ 314 71.9745 -6.121113 -5.348539 2 2
323
+ 315 72.0635 -2.111756 -3.578266 1 1
324
+ 316 72.1519 -6.573176 -5.363730 2 2
325
+ 317 71.9243 -4.954705 -4.570361 2 1
326
+ 318 72.0126 -5.282551 -4.487119 2 2
327
+ 319 72.1003 -1.443558 -2.286391 1 1
328
+ 320 72.1875 -4.295704 -5.291470 1 1
329
+ 321 71.9626 -3.012950 -3.029672 1 2
330
+ 322 71.7391 -2.068815 -2.147863 1 2
331
+ 323 71.8266 -2.588135 -2.987165 1 1
332
+ 324 71.6049 -3.936578 -3.904004 2 1
333
+ 325 71.6923 -4.948986 -4.380535 2 2
334
+ 326 71.7791 -2.857867 -1.906543 2 2
335
+ 327 71.8654 -2.257302 -3.026055 1 1
336
+ 328 71.9512 -3.923344 -4.376594 1 1
337
+ 329 72.0365 -3.414411 -9.358361 1 1
338
+ 330 72.1212 -0.977931 -1.941781 1 1
339
+ 331 71.9033 -5.572770 -3.775588 2 1
340
+ 332 71.9880 -1.971925 -2.690686 1 1
341
+ 333 72.0721 -2.781368 -1.745338 2 2
342
+ 334 72.1557 -2.003581 -5.115265 1 1
343
+ 335 71.9403 -4.582308 -4.347384 2 1
344
+ 336 71.7262 -3.530633 -4.091061 1 2
345
+ 337 71.8101 -1.801059 -1.624274 2 2
346
+ 338 71.5976 -4.147770 -3.299808 2 1
347
+ 339 71.6814 -5.363774 -4.761933 2 2
348
+ 340 71.7647 -5.264315 -5.601954 1 1
349
+ 341 71.8475 -2.410149 -4.184821 1 1
350
+ 342 71.9298 -6.446150 -6.312015 2 2
351
+ 343 72.0117 -3.971439 -4.440972 1 1
352
+ 344 72.0930 -2.482783 -2.771082 1 1
353
+ 345 72.1739 -1.007511 -2.972784 1 1
354
+ 346 71.9653 -4.461561 -7.060946 1 2
355
+ 347 72.0461 -4.118710 -3.570395 2 2
356
+ 348 72.1264 -0.829087 -2.526093 1 1
357
+ 349 71.9198 -3.561327 -5.605655 1 2
358
+ 350 71.7143 -2.008054 -3.653431 1 2
359
+ 351 71.7949 -2.330559 -3.396516 1 1
360
+ 352 71.8750 -3.492636 -2.593441 2 2
361
+ 353 71.6714 -2.023257 -1.816648 2 1
362
+ 354 71.7514 -5.076439 -3.675391 2 2
363
+ 355 71.8310 -2.467837 -2.096978 2 2
364
+ 356 71.9101 -3.357535 -3.220904 2 2
365
+ 357 71.9888 -3.686190 -3.269644 2 2
366
+ 358 72.0670 -2.265239 -2.065857 2 2
367
+ 359 72.1448 -6.701163 -4.106692 2 2
368
+ 360 71.9444 -2.418153 -2.830828 1 2
369
+ 361 72.0222 -2.668836 -1.275894 2 2
370
+ 362 71.8232 -4.864432 -4.971882 1 2
371
+ 363 71.6253 -2.561922 -2.478455 2 1
372
+ 364 71.7033 -5.498672 -6.170218 1 1
373
+ 365 71.7808 -5.656503 -6.367253 1 1
374
+ 366 71.5847 -3.510301 -9.078306 1 2
375
+ 367 71.6621 -5.514669 -6.287159 1 1
376
+ 368 71.7391 -2.821946 -1.283135 2 2
377
+ 369 71.5447 -3.768816 -3.836191 1 2
378
+ 370 71.6216 -5.034791 -5.132172 1 1
379
+ 371 71.6981 -4.082193 -4.273332 1 1
380
+ 372 71.5054 -2.723300 -2.357210 2 1
381
+ 373 71.3137 -5.292455 -5.260209 2 1
382
+ 374 71.3904 -1.529585 -2.664703 1 1
383
+ 375 71.4667 -3.274942 -2.516308 2 2
384
+ 376 71.5426 -6.014608 -4.508776 2 2
385
+ 377 71.3528 -3.619982 -3.812836 1 2
386
+ 378 71.1640 -3.327573 -3.468086 1 2
387
+ 379 70.9763 -0.755101 -1.674936 1 2
388
+ 380 70.7895 -8.225910 -7.512512 2 1
389
+ 381 70.8661 -11.229498 -8.512216 2 2
390
+ 382 70.6806 -2.216463 -2.326149 1 2
391
+ 383 70.7572 -2.049696 -0.646405 2 2
392
+ 384 70.5729 -3.506994 -2.623536 2 1
393
+ 385 70.3896 -1.151349 -0.999103 2 1
394
+ 386 70.4663 -1.801579 -9.227428 1 1
395
+ 387 70.5426 -1.803779 -2.034079 1 1
396
+ 388 70.3608 -5.235816 -6.097936 1 2
397
+ 389 70.4370 -2.573685 -1.669356 2 2
398
+ 390 70.5128 -2.712024 -3.852635 1 1
399
+ 391 70.3325 -3.607931 -5.255275 1 2
400
+ 392 70.1531 -5.404470 -3.285222 2 1
401
+ 393 70.2290 -7.139535 -6.402922 2 2
402
+ 394 70.3046 -1.653548 -4.452249 1 1
403
+ 395 70.1266 -3.632572 -3.769842 1 2
404
+ 396 69.9495 -5.069126 -4.772451 2 1
405
+ 397 69.7733 -7.492507 -7.103539 2 1
406
+ 398 69.8492 -3.965075 -7.376134 1 1
407
+ 399 69.9248 -4.450601 -4.458596 1 1
408
+ 400 70.0000 -2.542909 -5.480922 1 1
409
+ 401 70.0748 -2.599688 -2.897072 1 1
410
+ 402 69.9005 -3.968269 -4.407824 1 2
411
+ 403 69.9752 -6.944074 -6.918589 2 2
412
+ 404 70.0495 -3.948837 -2.838765 2 2
413
+ 405 70.1235 -2.953255 -4.141765 1 1
414
+ 406 70.1970 -8.051016 -9.319412 1 1
415
+ 407 70.2703 -5.633023 -5.553441 2 2
416
+ 408 70.3431 -3.199747 -3.158114 2 2
417
+ 409 70.4156 -3.961414 -4.203400 1 1
418
+ 410 70.4878 -4.979123 -5.548352 1 1
419
+ 411 70.5596 -4.682474 -5.315972 1 1
420
+ 412 70.6311 -3.936324 -2.715714 2 2
421
+ 413 70.7022 -3.744366 -2.761313 2 2
422
+ 414 70.5314 -4.587847 -4.771268 1 2
423
+ 415 70.6024 -2.344802 -2.093829 2 2
424
+ 416 70.6731 -4.238653 -2.293420 2 2
425
+ 417 70.7434 -4.663705 -5.690895 1 1
426
+ 418 70.8134 -5.466582 -6.971722 1 1
427
+ 419 70.6444 -4.498085 -5.049828 1 2
428
+ 420 70.4762 -3.310170 -3.468696 1 2
429
+ 421 70.5463 -4.283847 -3.691799 2 2
430
+ 422 70.3791 -2.642311 -2.340207 2 1
431
+ 423 70.2128 -3.318639 -3.201388 2 1
432
+ 424 70.2830 -5.050670 -5.661877 1 1
433
+ 425 70.3529 -3.945837 -3.113352 2 2
434
+ 426 70.1878 -1.510272 -1.969855 1 2
435
+ 427 70.2576 -3.769558 -5.168155 1 1
436
+ 428 70.0935 -4.222549 -3.261728 2 1
437
+ 429 70.1632 -3.686420 -3.678311 2 2
438
+ 430 70.0000 -5.826519 -4.891466 2 1
439
+ 431 70.0696 -4.670042 -6.013447 1 1
440
+ 432 69.9074 -4.191545 -4.705892 1 2
441
+ 433 69.9769 -1.745535 -1.508319 2 2
442
+ 434 70.0461 -7.221194 -7.723482 1 1
443
+ 435 70.1149 -3.270235 -3.724132 1 1
444
+ 436 70.1835 -0.588460 -2.041927 1 1
445
+ 437 70.2517 -3.451596 -3.345155 2 2
446
+ 438 70.3196 -3.539551 -3.520405 2 2
447
+ 439 70.1595 -3.364493 -2.651799 2 1
448
+ 440 70.2273 -4.098506 -4.849073 1 1
449
+ 441 70.2948 -3.437753 -4.977623 1 1
450
+ 442 70.3620 -5.254331 -5.678609 1 1
451
+ 443 70.4289 -1.515952 -3.491650 1 1
452
+ 444 70.2703 -5.282427 -6.629738 1 2
453
+ 445 70.3371 -1.360608 -2.550069 1 1
454
+ 446 70.4036 -4.614949 -3.700514 2 2
455
+ 447 70.2461 -1.862884 -1.755740 2 1
456
+ 448 70.3125 -3.885515 -3.280483 2 2
457
+ 449 70.1559 -2.555917 -2.582305 1 2
458
+ 450 70.2222 -4.248735 -3.205445 2 2
459
+ 451 70.2882 -4.114937 -3.358981 2 2
460
+ 452 70.3540 -4.959652 -5.206430 1 1
461
+ 453 70.4194 -4.974055 -4.727912 2 2
462
+ 454 70.4846 -3.018789 -3.459546 1 1
463
+ 455 70.3297 -1.663735 -7.867070 1 2
464
+ 456 70.3947 -4.390158 -4.120592 2 2
465
+ 457 70.2407 -10.745235 -9.647965 2 1
466
+ 458 70.3057 -4.004094 -3.464578 2 2
467
+ 459 70.3704 -4.482815 -4.292329 2 2
468
+ 460 70.4348 -0.280369 -6.843044 1 1
469
+ 461 70.4989 -2.612518 -3.708878 1 1
470
+ 462 70.3463 -5.370420 -5.104105 2 1
471
+ 463 70.4104 -1.000204 -1.143554 1 1
472
+ 464 70.2586 -2.775638 -2.676369 2 1
473
+ 465 70.3226 -2.143306 -1.938924 2 2
474
+ 466 70.3863 -5.418295 -4.911432 2 2
475
+ 467 70.4497 -5.145538 -5.557120 1 1
476
+ 468 70.5128 -5.098616 -3.880637 2 2
477
+ 469 70.3625 -1.727349 -1.316343 2 1
478
+ 470 70.4255 -1.491574 -1.297971 2 2
479
+ 471 70.4883 -2.210909 -2.260729 1 1
480
+ 472 70.5508 -4.212977 -3.602911 2 2
481
+ 473 70.6131 -4.161826 -2.568673 2 2
482
+ 474 70.6751 -4.390683 -4.965687 1 1
483
+ 475 70.7368 -1.316840 -1.148164 2 2
484
+ 476 70.7983 -5.030821 -4.786547 2 2
485
+ 477 70.6499 -4.565330 -5.302664 1 2
486
+ 478 70.5021 -4.314578 -3.702463 2 1
487
+ 479 70.5637 -3.041610 -3.249686 1 1
488
+ 480 70.6250 -5.453721 -5.404569 2 2
489
+ 481 70.6861 -2.160843 -2.517598 1 1
490
+ 482 70.5394 -4.518368 -5.343975 1 2
491
+ 483 70.3934 -5.377512 -4.404852 2 1
492
+ 484 70.2479 -2.486564 -1.573978 2 1
493
+ 485 70.3093 -1.508977 -2.571912 1 1
494
+ 486 70.3704 -2.499417 -3.172029 1 1
495
+ 487 70.4312 -2.711424 -2.097181 2 2
496
+ 488 70.4918 -3.517007 -2.507269 2 2
497
+ 489 70.5521 -1.908730 -2.232932 1 1
498
+ 490 70.6122 -3.407133 -3.925055 1 1
499
+ 491 70.6721 -5.371515 -4.278044 2 2
500
+ 492 70.7317 -5.528633 -5.036609 2 2
501
+ 493 70.7911 -0.984910 -2.371695 1 1
502
+ 494 70.8502 -4.034035 -3.912194 2 2
503
+ 495 70.9091 -4.030318 -4.202210 1 1
504
+ 496 70.7661 -3.542070 -3.545280 1 2
505
+ 497 70.6237 -5.339425 -4.757565 2 1
506
+ 498 70.4819 -1.377222 -2.034773 1 2
507
+ 499 70.5411 -2.648004 -3.293732 1 1
508
+ 500 70.6000 -5.273245 -3.964700 2 2
509
+ 501 70.6587 -0.624810 -1.493223 1 1
510
+ 502 70.7171 -1.036753 -1.623764 1 1
511
+ 503 70.7753 -2.607448 -2.572819 2 2
512
+ 504 70.8333 -2.524945 -2.672071 1 1
513
+ 505 70.6931 -8.127002 -8.782312 1 2
514
+ 506 70.5534 -5.488840 -5.023780 2 1
515
+ 507 70.6114 -4.411029 -4.598770 1 1
516
+ 508 70.6693 -6.367513 -3.298389 2 2
517
+ 509 70.7269 -2.864610 -7.973906 1 1
518
+ 510 70.5882 -3.867817 -3.330431 2 1
519
+ 511 70.6458 -5.062875 -3.762076 2 2
520
+ 512 70.7031 -3.546848 -4.135550 1 1
521
+ 513 70.7602 -3.655930 -3.588072 2 2
522
+ 514 70.6226 -5.050087 -4.415846 2 1
523
+ 515 70.6796 -6.641340 -6.992946 1 1
524
+ 516 70.7364 -4.247991 -4.304083 1 1
525
+ 517 70.5996 -1.622548 -1.520782 2 1
526
+ 518 70.6564 -2.712580 -1.751968 2 2
527
+ 519 70.7129 -9.361179 -8.974318 2 2
528
+ 520 70.7692 -2.429387 -1.856293 2 2
529
+ 521 70.8253 -0.955235 -1.093696 1 1
530
+ 522 70.8812 -4.033417 -3.334436 2 2
531
+ 523 70.7457 -3.936737 -4.170463 1 2
532
+ 524 70.8015 -6.938341 -6.973669 1 1
533
+ 525 70.8571 -5.216709 -2.109414 2 2
534
+ 526 70.9125 -0.251182 -16.985619 1 1
535
+ 527 70.9677 -6.415796 -4.293784 2 2
536
+ 528 71.0227 -6.410530 -5.891025 2 2
537
+ 529 71.0775 -5.082260 -6.061204 1 1
538
+ 530 71.1321 -5.369100 -3.774246 2 2
539
+ 531 70.9981 -3.658800 -3.417810 2 1
540
+ 532 71.0526 -2.279384 -2.976609 1 1
541
+ 533 71.1069 -3.154216 -2.680537 2 2
542
+ 534 71.1610 -2.736705 -3.436401 1 1
543
+ 535 71.0280 -3.430579 -3.613569 1 2
544
+ 536 70.8955 -2.348929 -2.196281 2 1
545
+ 537 70.9497 -5.932215 -6.490891 1 1
546
+ 538 71.0037 -2.303958 -3.289888 1 1
547
+ 539 71.0575 -4.224823 -2.559131 2 2
548
+ 540 70.9259 -2.244582 -2.560321 1 2
549
+ 541 70.9797 -2.450736 -2.734870 1 1
550
+ 542 70.8487 -3.658950 -4.420695 1 2
551
+ 543 70.9024 -5.244988 -4.546478 2 2
552
+ 544 70.9559 -2.364757 -4.063808 1 1
553
+ 545 70.8257 -1.394236 -4.520120 1 2
554
+ 546 70.8791 -2.461475 -3.296515 1 1
555
+ 547 70.9324 -6.094460 -4.579689 2 2
556
+ 548 70.9854 -4.104138 -4.572541 1 1
557
+ 549 70.8561 -5.419476 -5.281096 2 1
558
+ 550 70.7273 -3.384594 -3.455775 1 2
559
+ 551 70.5989 -1.046572 -8.186500 1 2
560
+ 552 70.6522 -2.233216 -2.737379 1 1
561
+ 553 70.7052 -5.943862 -5.052890 2 2
562
+ 554 70.5776 -4.785411 -5.257247 1 2
563
+ 555 70.6306 -2.586521 -1.765447 2 2
564
+ 556 70.6835 -5.826060 -4.535947 2 2
565
+ 557 70.7361 -4.557511 -5.035581 1 1
566
+ 558 70.7885 -1.404340 -3.131990 1 1
567
+ 559 70.6619 -2.405436 -2.410304 1 2
568
+ 560 70.5357 -2.451531 -1.986498 2 1
569
+ 561 70.5882 -1.901888 -2.380623 1 1
570
+ 562 70.6406 -0.550062 -1.333155 1 1
571
+ 563 70.5151 -1.410167 -2.589708 1 2
572
+ 564 70.5674 -2.907558 -3.441677 1 1
573
+ 565 70.6195 -3.351635 -3.244249 2 2
574
+ 566 70.6714 -3.538257 -4.859324 1 1
575
+ 567 70.7231 -2.725748 -2.606719 2 2
576
+ 568 70.7746 -6.832528 -4.403049 2 2
577
+ 569 70.6503 -5.783020 -5.405098 2 1
578
+ 570 70.7018 -4.723776 -6.611005 1 1
579
+ 571 70.7531 -4.398693 -2.972199 2 2
580
+ 572 70.8042 -2.700803 -2.616262 2 2
581
+ 573 70.6806 -4.037215 -4.591121 1 2
582
+ 574 70.7317 -3.416118 -2.601399 2 2
583
+ 575 70.7826 -4.962817 -4.551290 2 2
584
+ 576 70.8333 -2.503753 -3.205116 1 1
585
+ 577 70.8839 -4.255203 -4.051623 2 2
586
+ 578 70.9343 -3.274050 -3.047374 2 2
587
+ 579 70.8117 -5.171369 -5.179018 1 2
588
+ 580 70.8621 -1.886359 -3.348393 1 1
589
+ 581 70.9122 -5.905024 -8.262554 1 1
590
+ 582 70.7904 -5.448276 -5.651156 1 2
591
+ 583 70.8405 -4.386477 -4.158887 2 2
592
+ 584 70.7192 -4.705873 -4.589386 2 1
593
+ 585 70.7692 -0.440891 -1.021396 1 1
594
+ 586 70.8191 -3.580453 -3.968049 1 1
595
+ 587 70.6985 -4.604926 -4.353853 2 1
596
+ 588 70.7483 -4.413851 -2.040649 2 2
597
+ 589 70.6282 -3.321860 -3.889252 1 2
598
+ 590 70.6780 -4.026028 -3.519293 2 2
599
+ 591 70.5584 -3.614095 -3.887619 1 2
600
+ 592 70.6081 -3.397765 -4.191576 1 1
601
+ 593 70.6577 -4.552829 -4.414011 2 2
602
+ 594 70.5387 -4.896832 -4.193141 2 1
603
+ 595 70.5882 -1.791178 -1.768132 2 2
604
+ 596 70.6376 -5.304776 -4.980855 2 2
605
+ 597 70.6868 -2.191948 -2.251129 1 1
606
+ 598 70.7358 -3.092383 -3.366996 1 1
607
+ 599 70.7846 -3.042927 -4.309137 1 1
608
+ 600 70.8333 -3.787015 -3.713950 2 2
609
+ 601 70.8819 -3.121798 -2.604273 2 2
610
+ 602 70.9302 -2.378023 -4.737269 1 1
611
+ 603 70.9784 -6.551863 -4.821322 2 2
612
+ 604 70.8609 -2.937898 -3.024337 1 2
613
+ 605 70.7438 -2.853914 -1.916466 2 1
614
+ 606 70.7921 -3.194585 -3.101061 2 2
615
+ 607 70.6755 -2.869974 -2.477242 2 1
616
+ 608 70.7237 -2.629593 -3.185435 1 1
617
+ 609 70.6076 -7.591936 -7.326476 2 1
618
+ 610 70.6557 -2.609197 -2.886014 1 1
619
+ 611 70.5401 -4.960046 -4.872966 2 1
620
+ 612 70.5882 -5.000158 -6.625103 1 1
621
+ 613 70.6362 -3.159190 -2.181716 2 2
622
+ 614 70.6840 -5.131670 -4.647464 2 2
623
+ 615 70.7317 -2.885410 -2.618469 2 2
624
+ 616 70.6169 -5.730086 -5.878874 1 2
625
+ 617 70.6645 -10.231824 -5.565432 2 2
626
+ 618 70.5502 -3.675061 -3.949134 1 2
627
+ 619 70.5977 -3.916104 -3.473467 2 2
628
+ 620 70.4839 -5.190292 -4.245901 2 1
629
+ 621 70.5314 -3.286309 -2.100377 2 2
630
+ 622 70.5788 -3.412817 -3.753033 1 1
631
+ 623 70.6260 -2.773492 -2.832371 1 1
632
+ 624 70.6731 -4.800901 -3.858815 2 2
633
+ 625 70.7200 -3.745886 -6.067652 1 1
634
+ 626 70.6070 -1.487547 -0.801155 2 1
635
+ 627 70.6539 -3.371291 -4.181221 1 1
636
+ 628 70.7006 -2.801602 -2.272381 2 2
637
+ 629 70.7472 -2.014479 -2.553922 1 1
638
+ 630 70.7937 -0.978404 -1.575709 1 1
639
+ 631 70.8399 -2.342928 -3.553972 1 1
640
+ 632 70.8861 -1.720560 -1.412317 2 2
641
+ 633 70.9321 -1.536301 -1.410317 2 2
642
+ 634 70.9779 -2.283866 -3.384754 1 1
643
+ 635 70.8661 -4.919298 -5.018909 1 2
644
+ 636 70.9119 -0.800632 -2.648170 1 1
645
+ 637 70.9576 -3.061946 -5.132467 1 1
646
+ 638 71.0031 -5.129857 -5.608571 1 1
647
+ 639 71.0485 -1.318650 -1.742029 1 1
648
+ 640 71.0938 -1.530037 -1.446697 2 2
649
+ 641 70.9828 -6.926346 -5.710470 2 1
650
+ 642 71.0280 -4.482454 -3.306783 2 2
651
+ 643 71.0731 -1.408220 -0.804388 2 2
652
+ 644 71.1180 -3.287066 -4.062298 1 1
653
+ 645 71.1628 -1.038513 -3.182162 1 1
654
+ 646 71.0526 -3.902144 -3.729354 2 1
655
+ 647 70.9428 -3.611942 -2.237472 2 1
656
+ 648 70.9877 -3.243448 -3.066441 2 2
657
+ 649 71.0324 -3.436249 -3.802624 1 1
658
+ 650 71.0769 -3.288401 -3.829967 1 1
659
+ 651 70.9677 -5.827254 -6.024658 1 2
660
+ 652 71.0123 -5.460817 -4.977067 2 2
661
+ 653 71.0567 -5.121837 -3.764986 2 2
662
+ 654 70.9480 -3.408496 -3.868433 1 2
663
+ 655 70.8397 -2.298042 -2.408423 1 2
664
+ 656 70.8841 -3.895849 -2.574214 2 2
665
+ 657 70.7763 -5.319745 -5.576377 1 2
666
+ 658 70.8207 -3.871490 -1.644329 2 2
667
+ 659 70.8649 -2.220559 -1.655104 2 2
668
+ 660 70.7576 -3.088202 -3.889039 1 2
669
+ 661 70.8018 -3.229875 -3.380460 1 1
670
+ 662 70.8459 -3.143538 -3.939994 1 1
671
+ 663 70.8899 -6.587157 -6.398791 2 2
672
+ 664 70.7831 -3.507699 -3.408383 2 1
673
+ 665 70.8271 -3.098361 -3.092849 2 2
674
+ 666 70.8709 -3.696228 -4.729556 1 1
675
+ 667 70.9145 -8.349162 -6.475458 2 2
676
+ 668 70.9581 -1.966282 -2.048543 1 1
677
+ 669 71.0015 -2.639067 -3.160479 1 1
678
+ 670 70.8955 -3.455786 -2.994800 2 1
679
+ 671 70.9389 -3.126303 -4.605088 1 1
680
+ 672 70.9821 -2.645523 -2.348904 2 2
681
+ 673 71.0253 -1.729330 -1.729254 2 2
682
+ 674 70.9199 -5.504817 -5.119599 2 1
683
+ 675 70.9630 -3.442896 -3.510732 1 1
684
+ 676 70.8580 -2.333244 -2.784608 1 2
685
+ 677 70.9010 -3.815612 -4.196609 1 1
686
+ 678 70.7965 -5.588187 -5.854546 1 2
687
+ 679 70.8395 -5.613647 -4.657640 2 2
688
+ 680 70.8824 -3.732800 -4.604360 1 1
689
+ 681 70.9251 -3.036922 -3.168493 1 1
690
+ 682 70.9677 -2.161098 -2.098770 2 2
691
+ 683 71.0102 -1.989352 -3.062766 1 1
692
+ 684 71.0526 -5.657259 -7.450756 1 1
693
+ 685 71.0949 -4.648393 -2.297441 2 2
694
+ 686 71.1370 -5.803608 -5.279459 2 2
695
+ 687 71.1790 -3.727439 -2.698732 2 2
696
+ 688 71.2209 -8.148251 -8.174142 1 1
697
+ 689 71.2627 -2.700685 -2.232724 2 2
698
+ 690 71.1594 -2.654633 -2.672277 1 2
699
+ 691 71.0564 -2.514748 -2.001928 2 1
700
+ 692 71.0983 -1.719669 -2.665128 1 1
701
+ 693 71.1400 -4.573068 -4.972949 1 1
702
+ 694 71.1816 -3.430236 -3.358313 2 2
703
+ 695 71.2230 -3.854796 -4.815691 1 1
704
+ 696 71.2644 -4.930477 -5.481026 1 1
705
+ 697 71.3056 -2.199235 -2.650969 1 1
706
+ 698 71.3467 -4.181657 -3.686769 2 2
707
+ 699 71.3877 -3.410383 -4.500510 1 1
708
+ 700 71.4286 -4.310439 -3.686274 2 2
709
+ 701 71.3267 -2.429311 -2.166176 2 1
710
+ 702 71.2251 -4.135815 -5.054482 1 2
711
+ 703 71.2660 -3.676155 -3.805373 1 1
712
+ 704 71.3068 -1.313412 -1.668589 1 1
713
+ 705 71.2057 -3.105825 -3.461817 1 2
714
+ 706 71.2465 -3.265691 -3.139772 2 2
715
+ 707 71.1457 -4.502994 -4.396532 2 1
716
+ 708 71.1864 -3.969511 -3.150009 2 2
717
+ 709 71.2271 -2.142964 -1.788506 2 2
718
+ 710 71.2676 -3.625369 -4.385598 1 1
719
+ 711 71.1674 -1.664837 -2.085853 1 2
720
+ 712 71.0674 -2.465896 -3.753844 1 2
721
+ 713 71.1080 -3.205066 -3.171066 2 2
722
+ 714 71.1485 -3.065525 -3.009103 2 2
723
+ 715 71.1888 -4.391221 -3.549588 2 2
724
+ 716 71.0894 -4.373550 -4.422078 1 2
725
+ 717 71.1297 -1.905047 -0.183348 2 2
726
+ 718 71.0306 -4.632103 -4.484780 2 1
727
+ 719 70.9318 -4.927857 -4.706268 2 1
728
+ 720 70.8333 -2.109545 -4.657290 1 2
729
+ 721 70.7351 -5.032116 -5.943898 1 2
730
+ 722 70.6371 -2.276069 -2.052496 2 1
731
+ 723 70.6777 -2.242095 -2.597257 1 1
732
+ 724 70.7182 -4.125677 -4.042063 2 2
733
+ 725 70.6207 -2.189683 -2.743380 1 2
734
+ 726 70.6612 -4.288505 -4.134695 2 2
735
+ 727 70.7015 -2.723485 -2.262216 2 2
736
+ 728 70.6044 -3.350322 -3.180372 2 1
737
+ 729 70.6447 -1.194996 -2.285461 1 1
738
+ 730 70.6849 -3.980697 -4.407196 1 1
739
+ 731 70.7250 -4.566445 -4.611478 1 1
740
+ 732 70.7650 -2.910911 -2.165775 2 2
741
+ 733 70.8049 -6.672071 -6.293117 2 2
742
+ 734 70.8447 -1.264083 -11.823573 1 1
743
+ 735 70.7483 -3.491170 -2.654370 2 1
744
+ 736 70.7880 -1.766208 -2.548118 1 1
745
+ 737 70.8277 -4.562839 -5.044454 1 1
746
+ 738 70.7317 -4.197070 -4.711785 1 2
747
+ 739 70.7713 -0.817067 -7.068814 1 1
748
+ 740 70.8108 -4.660909 -3.337044 2 2
749
+ 741 70.7152 -4.717516 -3.933978 2 1
750
+ 742 70.7547 -2.455173 -2.129045 2 2
751
+ 743 70.7941 -4.095945 -4.071792 2 2
752
+ 744 70.8333 -4.797932 -4.028847 2 2
753
+ 745 70.8725 -2.894834 -3.636119 1 1
754
+ 746 70.7775 -7.205116 -7.181962 2 1
755
+ 747 70.8166 -5.112859 -7.046493 1 1
756
+ 748 70.8556 -4.062823 -3.276333 2 2
757
+ 749 70.8945 -3.850319 -4.886671 1 1
758
+ 750 70.9333 -4.461223 -8.447098 1 1
759
+
760
+ Final Winogrande score(750 tasks): 70.9333 +/- 1.6591
scores/Hammer2.1-7b-q4_k_m.arc ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 869 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 869 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 66.66666667
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 57.14285714
18
+ 8 62.50000000
19
+ 9 55.55555556
20
+ 10 60.00000000
21
+ 11 54.54545455
22
+ 12 58.33333333
23
+ 13 61.53846154
24
+ 14 64.28571429
25
+ 15 66.66666667
26
+ 16 68.75000000
27
+ 17 70.58823529
28
+ 18 66.66666667
29
+ 19 63.15789474
30
+ 20 60.00000000
31
+ 21 57.14285714
32
+ 22 59.09090909
33
+ 23 56.52173913
34
+ 24 58.33333333
35
+ 25 56.00000000
36
+ 26 53.84615385
37
+ 27 55.55555556
38
+ 28 53.57142857
39
+ 29 55.17241379
40
+ 30 53.33333333
41
+ 31 51.61290323
42
+ 32 50.00000000
43
+ 33 51.51515152
44
+ 34 50.00000000
45
+ 35 51.42857143
46
+ 36 50.00000000
47
+ 37 51.35135135
48
+ 38 52.63157895
49
+ 39 53.84615385
50
+ 40 52.50000000
51
+ 41 53.65853659
52
+ 42 54.76190476
53
+ 43 55.81395349
54
+ 44 54.54545455
55
+ 45 55.55555556
56
+ 46 56.52173913
57
+ 47 57.44680851
58
+ 48 56.25000000
59
+ 49 55.10204082
60
+ 50 56.00000000
61
+ 51 54.90196078
62
+ 52 55.76923077
63
+ 53 56.60377358
64
+ 54 55.55555556
65
+ 55 54.54545455
66
+ 56 55.35714286
67
+ 57 56.14035088
68
+ 58 56.89655172
69
+ 59 57.62711864
70
+ 60 58.33333333
71
+ 61 59.01639344
72
+ 62 58.06451613
73
+ 63 58.73015873
74
+ 64 59.37500000
75
+ 65 58.46153846
76
+ 66 59.09090909
77
+ 67 59.70149254
78
+ 68 60.29411765
79
+ 69 59.42028986
80
+ 70 58.57142857
81
+ 71 57.74647887
82
+ 72 56.94444444
83
+ 73 57.53424658
84
+ 74 58.10810811
85
+ 75 58.66666667
86
+ 76 59.21052632
87
+ 77 59.74025974
88
+ 78 60.25641026
89
+ 79 59.49367089
90
+ 80 60.00000000
91
+ 81 59.25925926
92
+ 82 59.75609756
93
+ 83 59.03614458
94
+ 84 58.33333333
95
+ 85 58.82352941
96
+ 86 59.30232558
97
+ 87 59.77011494
98
+ 88 60.22727273
99
+ 89 60.67415730
100
+ 90 60.00000000
101
+ 91 60.43956044
102
+ 92 60.86956522
103
+ 93 60.21505376
104
+ 94 59.57446809
105
+ 95 60.00000000
106
+ 96 60.41666667
107
+ 97 59.79381443
108
+ 98 59.18367347
109
+ 99 59.59595960
110
+ 100 60.00000000
111
+ 101 59.40594059
112
+ 102 59.80392157
113
+ 103 59.22330097
114
+ 104 58.65384615
115
+ 105 59.04761905
116
+ 106 58.49056604
117
+ 107 58.87850467
118
+ 108 58.33333333
119
+ 109 57.79816514
120
+ 110 58.18181818
121
+ 111 58.55855856
122
+ 112 58.92857143
123
+ 113 58.40707965
124
+ 114 58.77192982
125
+ 115 59.13043478
126
+ 116 58.62068966
127
+ 117 58.11965812
128
+ 118 58.47457627
129
+ 119 57.98319328
130
+ 120 58.33333333
131
+ 121 57.85123967
132
+ 122 58.19672131
133
+ 123 58.53658537
134
+ 124 58.06451613
135
+ 125 58.40000000
136
+ 126 58.73015873
137
+ 127 58.26771654
138
+ 128 58.59375000
139
+ 129 58.13953488
140
+ 130 57.69230769
141
+ 131 58.01526718
142
+ 132 58.33333333
143
+ 133 57.89473684
144
+ 134 57.46268657
145
+ 135 57.77777778
146
+ 136 57.35294118
147
+ 137 57.66423358
148
+ 138 57.97101449
149
+ 139 58.27338129
150
+ 140 57.85714286
151
+ 141 58.15602837
152
+ 142 57.74647887
153
+ 143 58.04195804
154
+ 144 57.63888889
155
+ 145 57.93103448
156
+ 146 57.53424658
157
+ 147 57.82312925
158
+ 148 58.10810811
159
+ 149 58.38926174
160
+ 150 58.66666667
161
+ 151 58.94039735
162
+ 152 59.21052632
163
+ 153 59.47712418
164
+ 154 59.74025974
165
+ 155 59.35483871
166
+ 156 59.61538462
167
+ 157 59.87261146
168
+ 158 60.12658228
169
+ 159 60.37735849
170
+ 160 60.00000000
171
+ 161 59.62732919
172
+ 162 59.87654321
173
+ 163 59.50920245
174
+ 164 59.14634146
175
+ 165 59.39393939
176
+ 166 59.63855422
177
+ 167 59.28143713
178
+ 168 59.52380952
179
+ 169 59.76331361
180
+ 170 60.00000000
181
+ 171 59.64912281
182
+ 172 59.88372093
183
+ 173 60.11560694
184
+ 174 60.34482759
185
+ 175 60.00000000
186
+ 176 59.65909091
187
+ 177 59.88700565
188
+ 178 59.55056180
189
+ 179 59.77653631
190
+ 180 59.44444444
191
+ 181 59.66850829
192
+ 182 59.89010989
193
+ 183 59.56284153
194
+ 184 59.23913043
195
+ 185 59.45945946
196
+ 186 59.67741935
197
+ 187 59.89304813
198
+ 188 60.10638298
199
+ 189 59.78835979
200
+ 190 59.47368421
201
+ 191 59.68586387
202
+ 192 59.37500000
203
+ 193 59.06735751
204
+ 194 59.27835052
205
+ 195 59.48717949
206
+ 196 59.18367347
207
+ 197 59.39086294
208
+ 198 59.09090909
209
+ 199 58.79396985
210
+ 200 59.00000000
211
+ 201 59.20398010
212
+ 202 59.40594059
213
+ 203 59.60591133
214
+ 204 59.80392157
215
+ 205 59.51219512
216
+ 206 59.22330097
217
+ 207 59.42028986
218
+ 208 59.61538462
219
+ 209 59.80861244
220
+ 210 60.00000000
221
+ 211 60.18957346
222
+ 212 59.90566038
223
+ 213 60.09389671
224
+ 214 60.28037383
225
+ 215 60.46511628
226
+ 216 60.18518519
227
+ 217 59.90783410
228
+ 218 59.63302752
229
+ 219 59.81735160
230
+ 220 59.54545455
231
+ 221 59.27601810
232
+ 222 59.45945946
233
+ 223 59.19282511
234
+ 224 59.37500000
235
+ 225 59.11111111
236
+ 226 58.84955752
237
+ 227 59.03083700
238
+ 228 59.21052632
239
+ 229 59.38864629
240
+ 230 59.56521739
241
+ 231 59.30735931
242
+ 232 59.05172414
243
+ 233 59.22746781
244
+ 234 59.40170940
245
+ 235 59.14893617
246
+ 236 59.32203390
247
+ 237 59.49367089
248
+ 238 59.66386555
249
+ 239 59.83263598
250
+ 240 60.00000000
251
+ 241 60.16597510
252
+ 242 59.91735537
253
+ 243 60.08230453
254
+ 244 59.83606557
255
+ 245 60.00000000
256
+ 246 60.16260163
257
+ 247 60.32388664
258
+ 248 60.48387097
259
+ 249 60.24096386
260
+ 250 60.40000000
261
+ 251 60.55776892
262
+ 252 60.71428571
263
+ 253 60.86956522
264
+ 254 60.62992126
265
+ 255 60.78431373
266
+ 256 60.93750000
267
+ 257 61.08949416
268
+ 258 61.24031008
269
+ 259 61.00386100
270
+ 260 61.15384615
271
+ 261 61.30268199
272
+ 262 61.45038168
273
+ 263 61.59695817
274
+ 264 61.74242424
275
+ 265 61.88679245
276
+ 266 61.65413534
277
+ 267 61.79775281
278
+ 268 61.56716418
279
+ 269 61.71003717
280
+ 270 61.85185185
281
+ 271 61.99261993
282
+ 272 61.76470588
283
+ 273 61.53846154
284
+ 274 61.31386861
285
+ 275 61.09090909
286
+ 276 61.23188406
287
+ 277 61.37184116
288
+ 278 61.15107914
289
+ 279 61.29032258
290
+ 280 61.42857143
291
+ 281 61.56583630
292
+ 282 61.70212766
293
+ 283 61.48409894
294
+ 284 61.61971831
295
+ 285 61.75438596
296
+ 286 61.88811189
297
+ 287 61.67247387
298
+ 288 61.80555556
299
+ 289 61.59169550
300
+ 290 61.72413793
301
+ 291 61.85567010
302
+ 292 61.64383562
303
+ 293 61.77474403
304
+ 294 61.56462585
305
+ 295 61.69491525
306
+ 296 61.82432432
307
+ 297 61.95286195
308
+ 298 62.08053691
309
+ 299 62.20735786
310
+ 300 62.00000000
311
+ 301 61.79401993
312
+ 302 61.92052980
313
+ 303 62.04620462
314
+ 304 62.17105263
315
+ 305 62.29508197
316
+ 306 62.09150327
317
+ 307 61.88925081
318
+ 308 62.01298701
319
+ 309 62.13592233
320
+ 310 61.93548387
321
+ 311 62.05787781
322
+ 312 61.85897436
323
+ 313 61.98083067
324
+ 314 61.78343949
325
+ 315 61.90476190
326
+ 316 62.02531646
327
+ 317 62.14511041
328
+ 318 61.94968553
329
+ 319 61.75548589
330
+ 320 61.56250000
331
+ 321 61.37071651
332
+ 322 61.49068323
333
+ 323 61.60990712
334
+ 324 61.41975309
335
+ 325 61.23076923
336
+ 326 61.34969325
337
+ 327 61.16207951
338
+ 328 61.28048780
339
+ 329 61.39817629
340
+ 330 61.51515152
341
+ 331 61.63141994
342
+ 332 61.74698795
343
+ 333 61.86186186
344
+ 334 61.67664671
345
+ 335 61.49253731
346
+ 336 61.30952381
347
+ 337 61.12759644
348
+ 338 60.94674556
349
+ 339 61.06194690
350
+ 340 61.17647059
351
+ 341 61.29032258
352
+ 342 61.40350877
353
+ 343 61.51603499
354
+ 344 61.62790698
355
+ 345 61.44927536
356
+ 346 61.56069364
357
+ 347 61.38328530
358
+ 348 61.20689655
359
+ 349 61.31805158
360
+ 350 61.14285714
361
+ 351 61.25356125
362
+ 352 61.36363636
363
+ 353 61.47308782
364
+ 354 61.29943503
365
+ 355 61.12676056
366
+ 356 60.95505618
367
+ 357 60.78431373
368
+ 358 60.89385475
369
+ 359 60.72423398
370
+ 360 60.83333333
371
+ 361 60.94182825
372
+ 362 61.04972376
373
+ 363 61.15702479
374
+ 364 60.98901099
375
+ 365 60.82191781
376
+ 366 60.65573770
377
+ 367 60.76294278
378
+ 368 60.86956522
379
+ 369 60.70460705
380
+ 370 60.54054054
381
+ 371 60.37735849
382
+ 372 60.21505376
383
+ 373 60.05361930
384
+ 374 60.16042781
385
+ 375 60.26666667
386
+ 376 60.10638298
387
+ 377 59.94694960
388
+ 378 60.05291005
389
+ 379 60.15831135
390
+ 380 60.26315789
391
+ 381 60.10498688
392
+ 382 59.94764398
393
+ 383 59.79112272
394
+ 384 59.89583333
395
+ 385 60.00000000
396
+ 386 60.10362694
397
+ 387 59.94832041
398
+ 388 59.79381443
399
+ 389 59.64010283
400
+ 390 59.48717949
401
+ 391 59.33503836
402
+ 392 59.43877551
403
+ 393 59.28753181
404
+ 394 59.39086294
405
+ 395 59.49367089
406
+ 396 59.34343434
407
+ 397 59.44584383
408
+ 398 59.29648241
409
+ 399 59.39849624
410
+ 400 59.50000000
411
+ 401 59.60099751
412
+ 402 59.70149254
413
+ 403 59.80148883
414
+ 404 59.90099010
415
+ 405 60.00000000
416
+ 406 60.09852217
417
+ 407 60.19656020
418
+ 408 60.29411765
419
+ 409 60.14669927
420
+ 410 60.24390244
421
+ 411 60.34063260
422
+ 412 60.43689320
423
+ 413 60.29055690
424
+ 414 60.38647343
425
+ 415 60.24096386
426
+ 416 60.09615385
427
+ 417 59.95203837
428
+ 418 60.04784689
429
+ 419 59.90453461
430
+ 420 59.76190476
431
+ 421 59.61995249
432
+ 422 59.71563981
433
+ 423 59.81087470
434
+ 424 59.66981132
435
+ 425 59.76470588
436
+ 426 59.62441315
437
+ 427 59.71896956
438
+ 428 59.57943925
439
+ 429 59.67365967
440
+ 430 59.53488372
441
+ 431 59.62877030
442
+ 432 59.72222222
443
+ 433 59.58429561
444
+ 434 59.67741935
445
+ 435 59.54022989
446
+ 436 59.63302752
447
+ 437 59.49656751
448
+ 438 59.36073059
449
+ 439 59.45330296
450
+ 440 59.31818182
451
+ 441 59.41043084
452
+ 442 59.27601810
453
+ 443 59.14221219
454
+ 444 59.23423423
455
+ 445 59.32584270
456
+ 446 59.41704036
457
+ 447 59.50782998
458
+ 448 59.37500000
459
+ 449 59.46547884
460
+ 450 59.55555556
461
+ 451 59.64523282
462
+ 452 59.73451327
463
+ 453 59.60264901
464
+ 454 59.47136564
465
+ 455 59.56043956
466
+ 456 59.42982456
467
+ 457 59.51859956
468
+ 458 59.60698690
469
+ 459 59.69498911
470
+ 460 59.56521739
471
+ 461 59.43600868
472
+ 462 59.52380952
473
+ 463 59.39524838
474
+ 464 59.26724138
475
+ 465 59.13978495
476
+ 466 59.22746781
477
+ 467 59.31477516
478
+ 468 59.40170940
479
+ 469 59.27505330
480
+ 470 59.36170213
481
+ 471 59.44798301
482
+ 472 59.53389831
483
+ 473 59.40803383
484
+ 474 59.49367089
485
+ 475 59.36842105
486
+ 476 59.24369748
487
+ 477 59.32914046
488
+ 478 59.41422594
489
+ 479 59.49895616
490
+ 480 59.58333333
491
+ 481 59.45945946
492
+ 482 59.54356846
493
+ 483 59.62732919
494
+ 484 59.50413223
495
+ 485 59.58762887
496
+ 486 59.67078189
497
+ 487 59.54825462
498
+ 488 59.63114754
499
+ 489 59.50920245
500
+ 490 59.38775510
501
+ 491 59.26680244
502
+ 492 59.34959350
503
+ 493 59.22920892
504
+ 494 59.31174089
505
+ 495 59.19191919
506
+ 496 59.27419355
507
+ 497 59.35613682
508
+ 498 59.43775100
509
+ 499 59.51903808
510
+ 500 59.60000000
511
+ 501 59.48103792
512
+ 502 59.56175299
513
+ 503 59.64214712
514
+ 504 59.52380952
515
+ 505 59.60396040
516
+ 506 59.48616601
517
+ 507 59.36883629
518
+ 508 59.25196850
519
+ 509 59.13555992
520
+ 510 59.21568627
521
+ 511 59.09980431
522
+ 512 59.17968750
523
+ 513 59.06432749
524
+ 514 59.14396887
525
+ 515 59.22330097
526
+ 516 59.10852713
527
+ 517 59.18762089
528
+ 518 59.26640927
529
+ 519 59.34489403
530
+ 520 59.23076923
531
+ 521 59.30902111
532
+ 522 59.19540230
533
+ 523 59.08221797
534
+ 524 58.96946565
535
+ 525 58.85714286
536
+ 526 58.74524715
537
+ 527 58.63377609
538
+ 528 58.52272727
539
+ 529 58.41209830
540
+ 530 58.49056604
541
+ 531 58.56873823
542
+ 532 58.64661654
543
+ 533 58.72420263
544
+ 534 58.61423221
545
+ 535 58.50467290
546
+ 536 58.58208955
547
+ 537 58.65921788
548
+ 538 58.55018587
549
+ 539 58.62708720
550
+ 540 58.51851852
551
+ 541 58.59519409
552
+ 542 58.67158672
553
+ 543 58.74769797
554
+ 544 58.63970588
555
+ 545 58.71559633
556
+ 546 58.60805861
557
+ 547 58.50091408
558
+ 548 58.57664234
559
+ 549 58.65209472
560
+ 550 58.72727273
561
+ 551 58.62068966
562
+ 552 58.51449275
563
+ 553 58.58951175
564
+ 554 58.66425993
565
+ 555 58.73873874
566
+ 556 58.63309353
567
+ 557 58.70736086
568
+ 558 58.60215054
569
+ 559 58.67620751
570
+ 560 58.75000000
571
+ 561 58.82352941
572
+ 562 58.89679715
573
+ 563 58.96980462
574
+ 564 58.86524823
575
+ 565 58.76106195
576
+ 566 58.83392226
577
+ 567 58.73015873
578
+ 568 58.80281690
579
+ 569 58.69947276
580
+ 570 58.59649123
581
+ 571 58.66900175
582
+ 572 58.56643357
583
+ 573 58.46422339
584
+ 574 58.53658537
585
+ 575 58.60869565
586
+ 576 58.68055556
587
+ 577 58.75216638
588
+ 578 58.82352941
589
+ 579 58.89464594
590
+ 580 58.79310345
591
+ 581 58.69191050
592
+ 582 58.76288660
593
+ 583 58.83361921
594
+ 584 58.90410959
595
+ 585 58.97435897
596
+ 586 59.04436860
597
+ 587 58.94378194
598
+ 588 58.84353741
599
+ 589 58.91341256
600
+ 590 58.98305085
601
+ 591 59.05245347
602
+ 592 58.95270270
603
+ 593 59.02192243
604
+ 594 59.09090909
605
+ 595 58.99159664
606
+ 596 58.89261745
607
+ 597 58.79396985
608
+ 598 58.69565217
609
+ 599 58.76460768
610
+ 600 58.83333333
611
+ 601 58.90183028
612
+ 602 58.80398671
613
+ 603 58.70646766
614
+ 604 58.60927152
615
+ 605 58.51239669
616
+ 606 58.58085809
617
+ 607 58.64909390
618
+ 608 58.71710526
619
+ 609 58.62068966
620
+ 610 58.68852459
621
+ 611 58.59247136
622
+ 612 58.49673203
623
+ 613 58.40130506
624
+ 614 58.30618893
625
+ 615 58.21138211
626
+ 616 58.11688312
627
+ 617 58.18476499
628
+ 618 58.09061489
629
+ 619 57.99676898
630
+ 620 58.06451613
631
+ 621 58.13204509
632
+ 622 58.03858521
633
+ 623 58.10593900
634
+ 624 58.01282051
635
+ 625 57.92000000
636
+ 626 57.82747604
637
+ 627 57.89473684
638
+ 628 57.80254777
639
+ 629 57.86963434
640
+ 630 57.93650794
641
+ 631 57.84469097
642
+ 632 57.75316456
643
+ 633 57.66192733
644
+ 634 57.72870662
645
+ 635 57.63779528
646
+ 636 57.70440252
647
+ 637 57.61381476
648
+ 638 57.68025078
649
+ 639 57.58998435
650
+ 640 57.50000000
651
+ 641 57.56630265
652
+ 642 57.47663551
653
+ 643 57.38724728
654
+ 644 57.45341615
655
+ 645 57.36434109
656
+ 646 57.43034056
657
+ 647 57.34157651
658
+ 648 57.25308642
659
+ 649 57.16486903
660
+ 650 57.07692308
661
+ 651 57.14285714
662
+ 652 57.05521472
663
+ 653 57.12098009
664
+ 654 57.18654434
665
+ 655 57.09923664
666
+ 656 57.16463415
667
+ 657 57.07762557
668
+ 658 56.99088146
669
+ 659 57.05614568
670
+ 660 57.12121212
671
+ 661 57.03479576
672
+ 662 57.09969789
673
+ 663 57.16440422
674
+ 664 57.22891566
675
+ 665 57.29323308
676
+ 666 57.20720721
677
+ 667 57.27136432
678
+ 668 57.18562874
679
+ 669 57.10014948
680
+ 670 57.16417910
681
+ 671 57.22801788
682
+ 672 57.14285714
683
+ 673 57.05794948
684
+ 674 57.12166172
685
+ 675 57.18518519
686
+ 676 57.10059172
687
+ 677 57.16395864
688
+ 678 57.22713864
689
+ 679 57.29013255
690
+ 680 57.35294118
691
+ 681 57.26872247
692
+ 682 57.18475073
693
+ 683 57.10102489
694
+ 684 57.16374269
695
+ 685 57.22627737
696
+ 686 57.14285714
697
+ 687 57.20524017
698
+ 688 57.26744186
699
+ 689 57.32946299
700
+ 690 57.39130435
701
+ 691 57.45296671
702
+ 692 57.51445087
703
+ 693 57.57575758
704
+ 694 57.63688761
705
+ 695 57.55395683
706
+ 696 57.47126437
707
+ 697 57.53228121
708
+ 698 57.44985673
709
+ 699 57.36766810
710
+ 700 57.42857143
711
+ 701 57.34664765
712
+ 702 57.40740741
713
+ 703 57.46799431
714
+ 704 57.52840909
715
+ 705 57.58865248
716
+ 706 57.64872521
717
+ 707 57.70862801
718
+ 708 57.76836158
719
+ 709 57.82792666
720
+ 710 57.88732394
721
+ 711 57.94655415
722
+ 712 58.00561798
723
+ 713 58.06451613
724
+ 714 57.98319328
725
+ 715 57.90209790
726
+ 716 57.96089385
727
+ 717 57.88005579
728
+ 718 57.79944290
729
+ 719 57.71905424
730
+ 720 57.77777778
731
+ 721 57.69764216
732
+ 722 57.75623269
733
+ 723 57.81466113
734
+ 724 57.87292818
735
+ 725 57.93103448
736
+ 726 57.85123967
737
+ 727 57.77166437
738
+ 728 57.82967033
739
+ 729 57.88751715
740
+ 730 57.94520548
741
+ 731 58.00273598
742
+ 732 58.06010929
743
+ 733 58.11732606
744
+ 734 58.17438692
745
+ 735 58.09523810
746
+ 736 58.15217391
747
+ 737 58.07327001
748
+ 738 57.99457995
749
+ 739 57.91610284
750
+ 740 57.83783784
751
+ 741 57.89473684
752
+ 742 57.95148248
753
+ 743 58.00807537
754
+ 744 57.93010753
755
+ 745 57.98657718
756
+ 746 57.90884718
757
+ 747 57.83132530
758
+ 748 57.75401070
759
+ 749 57.81041389
760
+ 750 57.86666667
761
+
762
+ Final result: 57.8667 +/- 1.8042
763
+ Random chance: 25.0083 +/- 1.5824
scores/Hammer2.1-7b-q4_k_m.hsw ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ hellaswag_score : loaded 10042 tasks from prompt.
6
+ ================================= is_spm = 0
7
+ hellaswag_score : selecting 750 randomized tasks.
8
+ hellaswag_score : calculating hellaswag score over selected tasks.
9
+
10
+ task acc_norm
11
+ 1 0.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 50.00000000
15
+ 5 60.00000000
16
+ 6 66.66666667
17
+ 7 71.42857143
18
+ 8 75.00000000
19
+ 9 77.77777778
20
+ 10 80.00000000
21
+ 11 81.81818182
22
+ 12 75.00000000
23
+ 13 76.92307692
24
+ 14 78.57142857
25
+ 15 80.00000000
26
+ 16 81.25000000
27
+ 17 76.47058824
28
+ 18 72.22222222
29
+ 19 68.42105263
30
+ 20 70.00000000
31
+ 21 71.42857143
32
+ 22 72.72727273
33
+ 23 73.91304348
34
+ 24 70.83333333
35
+ 25 68.00000000
36
+ 26 65.38461538
37
+ 27 62.96296296
38
+ 28 64.28571429
39
+ 29 65.51724138
40
+ 30 63.33333333
41
+ 31 64.51612903
42
+ 32 62.50000000
43
+ 33 63.63636364
44
+ 34 64.70588235
45
+ 35 65.71428571
46
+ 36 63.88888889
47
+ 37 64.86486486
48
+ 38 65.78947368
49
+ 39 66.66666667
50
+ 40 67.50000000
51
+ 41 68.29268293
52
+ 42 66.66666667
53
+ 43 65.11627907
54
+ 44 65.90909091
55
+ 45 66.66666667
56
+ 46 65.21739130
57
+ 47 63.82978723
58
+ 48 64.58333333
59
+ 49 65.30612245
60
+ 50 64.00000000
61
+ 51 64.70588235
62
+ 52 63.46153846
63
+ 53 62.26415094
64
+ 54 61.11111111
65
+ 55 61.81818182
66
+ 56 60.71428571
67
+ 57 61.40350877
68
+ 58 60.34482759
69
+ 59 61.01694915
70
+ 60 61.66666667
71
+ 61 60.65573770
72
+ 62 59.67741935
73
+ 63 60.31746032
74
+ 64 60.93750000
75
+ 65 60.00000000
76
+ 66 59.09090909
77
+ 67 59.70149254
78
+ 68 60.29411765
79
+ 69 60.86956522
80
+ 70 61.42857143
81
+ 71 60.56338028
82
+ 72 61.11111111
83
+ 73 61.64383562
84
+ 74 62.16216216
85
+ 75 61.33333333
86
+ 76 61.84210526
87
+ 77 62.33766234
88
+ 78 62.82051282
89
+ 79 63.29113924
90
+ 80 63.75000000
91
+ 81 62.96296296
92
+ 82 63.41463415
93
+ 83 63.85542169
94
+ 84 64.28571429
95
+ 85 64.70588235
96
+ 86 65.11627907
97
+ 87 65.51724138
98
+ 88 65.90909091
99
+ 89 65.16853933
100
+ 90 65.55555556
101
+ 91 65.93406593
102
+ 92 65.21739130
103
+ 93 65.59139785
104
+ 94 65.95744681
105
+ 95 66.31578947
106
+ 96 66.66666667
107
+ 97 67.01030928
108
+ 98 67.34693878
109
+ 99 67.67676768
110
+ 100 67.00000000
111
+ 101 67.32673267
112
+ 102 66.66666667
113
+ 103 66.01941748
114
+ 104 66.34615385
115
+ 105 66.66666667
116
+ 106 66.98113208
117
+ 107 67.28971963
118
+ 108 67.59259259
119
+ 109 66.97247706
120
+ 110 67.27272727
121
+ 111 67.56756757
122
+ 112 67.85714286
123
+ 113 68.14159292
124
+ 114 68.42105263
125
+ 115 68.69565217
126
+ 116 68.10344828
127
+ 117 67.52136752
128
+ 118 67.79661017
129
+ 119 68.06722689
130
+ 120 67.50000000
131
+ 121 66.94214876
132
+ 122 67.21311475
133
+ 123 67.47967480
134
+ 124 66.93548387
135
+ 125 67.20000000
136
+ 126 67.46031746
137
+ 127 67.71653543
138
+ 128 67.18750000
139
+ 129 67.44186047
140
+ 130 66.92307692
141
+ 131 67.17557252
142
+ 132 66.66666667
143
+ 133 66.91729323
144
+ 134 67.16417910
145
+ 135 67.40740741
146
+ 136 67.64705882
147
+ 137 67.88321168
148
+ 138 68.11594203
149
+ 139 68.34532374
150
+ 140 68.57142857
151
+ 141 68.79432624
152
+ 142 69.01408451
153
+ 143 69.23076923
154
+ 144 69.44444444
155
+ 145 69.65517241
156
+ 146 69.86301370
157
+ 147 70.06802721
158
+ 148 70.27027027
159
+ 149 70.46979866
160
+ 150 70.66666667
161
+ 151 70.19867550
162
+ 152 69.73684211
163
+ 153 69.93464052
164
+ 154 70.12987013
165
+ 155 70.32258065
166
+ 156 69.87179487
167
+ 157 69.42675159
168
+ 158 69.62025316
169
+ 159 69.18238994
170
+ 160 69.37500000
171
+ 161 69.56521739
172
+ 162 69.75308642
173
+ 163 69.93865031
174
+ 164 70.12195122
175
+ 165 70.30303030
176
+ 166 70.48192771
177
+ 167 70.65868263
178
+ 168 70.83333333
179
+ 169 70.41420118
180
+ 170 70.58823529
181
+ 171 70.76023392
182
+ 172 70.34883721
183
+ 173 69.94219653
184
+ 174 69.54022989
185
+ 175 69.71428571
186
+ 176 69.88636364
187
+ 177 70.05649718
188
+ 178 70.22471910
189
+ 179 69.83240223
190
+ 180 70.00000000
191
+ 181 69.61325967
192
+ 182 69.78021978
193
+ 183 69.94535519
194
+ 184 69.56521739
195
+ 185 69.72972973
196
+ 186 69.35483871
197
+ 187 68.98395722
198
+ 188 68.61702128
199
+ 189 68.25396825
200
+ 190 68.42105263
201
+ 191 68.58638743
202
+ 192 68.75000000
203
+ 193 68.39378238
204
+ 194 68.55670103
205
+ 195 68.20512821
206
+ 196 68.36734694
207
+ 197 68.02030457
208
+ 198 67.67676768
209
+ 199 67.83919598
210
+ 200 68.00000000
211
+ 201 68.15920398
212
+ 202 67.82178218
213
+ 203 67.48768473
214
+ 204 67.64705882
215
+ 205 67.31707317
216
+ 206 66.99029126
217
+ 207 67.14975845
218
+ 208 66.82692308
219
+ 209 66.98564593
220
+ 210 67.14285714
221
+ 211 67.29857820
222
+ 212 67.45283019
223
+ 213 67.60563380
224
+ 214 67.75700935
225
+ 215 67.44186047
226
+ 216 67.59259259
227
+ 217 67.28110599
228
+ 218 66.97247706
229
+ 219 67.12328767
230
+ 220 67.27272727
231
+ 221 66.96832579
232
+ 222 67.11711712
233
+ 223 67.26457399
234
+ 224 67.41071429
235
+ 225 67.55555556
236
+ 226 67.69911504
237
+ 227 67.40088106
238
+ 228 67.54385965
239
+ 229 67.68558952
240
+ 230 67.39130435
241
+ 231 67.09956710
242
+ 232 66.81034483
243
+ 233 66.52360515
244
+ 234 66.66666667
245
+ 235 66.38297872
246
+ 236 66.10169492
247
+ 237 66.24472574
248
+ 238 66.38655462
249
+ 239 66.52719665
250
+ 240 66.66666667
251
+ 241 66.80497925
252
+ 242 66.94214876
253
+ 243 67.07818930
254
+ 244 66.80327869
255
+ 245 66.53061224
256
+ 246 66.66666667
257
+ 247 66.39676113
258
+ 248 66.53225806
259
+ 249 66.66666667
260
+ 250 66.80000000
261
+ 251 66.93227092
262
+ 252 67.06349206
263
+ 253 66.79841897
264
+ 254 66.92913386
265
+ 255 67.05882353
266
+ 256 67.18750000
267
+ 257 66.92607004
268
+ 258 67.05426357
269
+ 259 67.18146718
270
+ 260 67.30769231
271
+ 261 67.43295019
272
+ 262 67.17557252
273
+ 263 66.92015209
274
+ 264 67.04545455
275
+ 265 67.16981132
276
+ 266 66.91729323
277
+ 267 67.04119850
278
+ 268 67.16417910
279
+ 269 67.28624535
280
+ 270 67.40740741
281
+ 271 67.15867159
282
+ 272 67.27941176
283
+ 273 67.03296703
284
+ 274 67.15328467
285
+ 275 66.90909091
286
+ 276 67.02898551
287
+ 277 67.14801444
288
+ 278 67.26618705
289
+ 279 67.38351254
290
+ 280 67.50000000
291
+ 281 67.61565836
292
+ 282 67.73049645
293
+ 283 67.49116608
294
+ 284 67.60563380
295
+ 285 67.36842105
296
+ 286 67.48251748
297
+ 287 67.59581882
298
+ 288 67.70833333
299
+ 289 67.82006920
300
+ 290 67.93103448
301
+ 291 68.04123711
302
+ 292 67.80821918
303
+ 293 67.91808874
304
+ 294 68.02721088
305
+ 295 68.13559322
306
+ 296 68.24324324
307
+ 297 68.35016835
308
+ 298 68.45637584
309
+ 299 68.22742475
310
+ 300 68.00000000
311
+ 301 67.77408638
312
+ 302 67.88079470
313
+ 303 67.65676568
314
+ 304 67.43421053
315
+ 305 67.54098361
316
+ 306 67.64705882
317
+ 307 67.42671010
318
+ 308 67.53246753
319
+ 309 67.63754045
320
+ 310 67.74193548
321
+ 311 67.84565916
322
+ 312 67.94871795
323
+ 313 68.05111821
324
+ 314 67.83439490
325
+ 315 67.93650794
326
+ 316 68.03797468
327
+ 317 68.13880126
328
+ 318 67.92452830
329
+ 319 67.71159875
330
+ 320 67.81250000
331
+ 321 67.60124611
332
+ 322 67.39130435
333
+ 323 67.49226006
334
+ 324 67.59259259
335
+ 325 67.69230769
336
+ 326 67.79141104
337
+ 327 67.58409786
338
+ 328 67.68292683
339
+ 329 67.78115502
340
+ 330 67.87878788
341
+ 331 67.97583082
342
+ 332 67.77108434
343
+ 333 67.86786787
344
+ 334 67.66467066
345
+ 335 67.76119403
346
+ 336 67.85714286
347
+ 337 67.65578635
348
+ 338 67.45562130
349
+ 339 67.55162242
350
+ 340 67.64705882
351
+ 341 67.74193548
352
+ 342 67.83625731
353
+ 343 67.93002915
354
+ 344 67.73255814
355
+ 345 67.82608696
356
+ 346 67.63005780
357
+ 347 67.72334294
358
+ 348 67.52873563
359
+ 349 67.33524355
360
+ 350 67.42857143
361
+ 351 67.52136752
362
+ 352 67.61363636
363
+ 353 67.42209632
364
+ 354 67.51412429
365
+ 355 67.60563380
366
+ 356 67.69662921
367
+ 357 67.50700280
368
+ 358 67.59776536
369
+ 359 67.68802228
370
+ 360 67.50000000
371
+ 361 67.59002770
372
+ 362 67.67955801
373
+ 363 67.76859504
374
+ 364 67.85714286
375
+ 365 67.94520548
376
+ 366 68.03278689
377
+ 367 68.11989101
378
+ 368 68.20652174
379
+ 369 68.29268293
380
+ 370 68.10810811
381
+ 371 68.19407008
382
+ 372 68.27956989
383
+ 373 68.09651475
384
+ 374 68.18181818
385
+ 375 68.26666667
386
+ 376 68.35106383
387
+ 377 68.16976127
388
+ 378 68.25396825
389
+ 379 68.33773087
390
+ 380 68.42105263
391
+ 381 68.50393701
392
+ 382 68.32460733
393
+ 383 68.40731070
394
+ 384 68.48958333
395
+ 385 68.57142857
396
+ 386 68.65284974
397
+ 387 68.47545220
398
+ 388 68.29896907
399
+ 389 68.38046272
400
+ 390 68.46153846
401
+ 391 68.28644501
402
+ 392 68.36734694
403
+ 393 68.44783715
404
+ 394 68.52791878
405
+ 395 68.60759494
406
+ 396 68.68686869
407
+ 397 68.76574307
408
+ 398 68.84422111
409
+ 399 68.92230576
410
+ 400 69.00000000
411
+ 401 69.07730673
412
+ 402 69.15422886
413
+ 403 68.98263027
414
+ 404 69.05940594
415
+ 405 68.88888889
416
+ 406 68.96551724
417
+ 407 69.04176904
418
+ 408 69.11764706
419
+ 409 69.19315403
420
+ 410 69.02439024
421
+ 411 68.85644769
422
+ 412 68.68932039
423
+ 413 68.76513317
424
+ 414 68.84057971
425
+ 415 68.67469880
426
+ 416 68.75000000
427
+ 417 68.82494005
428
+ 418 68.89952153
429
+ 419 68.97374702
430
+ 420 69.04761905
431
+ 421 68.88361045
432
+ 422 68.95734597
433
+ 423 69.03073286
434
+ 424 69.10377358
435
+ 425 69.17647059
436
+ 426 69.24882629
437
+ 427 69.08665105
438
+ 428 69.15887850
439
+ 429 68.99766900
440
+ 430 69.06976744
441
+ 431 69.14153132
442
+ 432 69.21296296
443
+ 433 69.28406467
444
+ 434 69.35483871
445
+ 435 69.19540230
446
+ 436 69.26605505
447
+ 437 69.10755149
448
+ 438 68.94977169
449
+ 439 69.02050114
450
+ 440 69.09090909
451
+ 441 69.16099773
452
+ 442 69.23076923
453
+ 443 69.30022573
454
+ 444 69.36936937
455
+ 445 69.43820225
456
+ 446 69.50672646
457
+ 447 69.57494407
458
+ 448 69.64285714
459
+ 449 69.71046771
460
+ 450 69.77777778
461
+ 451 69.62305987
462
+ 452 69.69026549
463
+ 453 69.53642384
464
+ 454 69.60352423
465
+ 455 69.45054945
466
+ 456 69.51754386
467
+ 457 69.58424508
468
+ 458 69.65065502
469
+ 459 69.71677560
470
+ 460 69.78260870
471
+ 461 69.84815618
472
+ 462 69.91341991
473
+ 463 69.97840173
474
+ 464 70.04310345
475
+ 465 69.89247312
476
+ 466 69.95708155
477
+ 467 70.02141328
478
+ 468 69.87179487
479
+ 469 69.72281450
480
+ 470 69.57446809
481
+ 471 69.42675159
482
+ 472 69.49152542
483
+ 473 69.55602537
484
+ 474 69.62025316
485
+ 475 69.68421053
486
+ 476 69.53781513
487
+ 477 69.39203354
488
+ 478 69.45606695
489
+ 479 69.51983299
490
+ 480 69.58333333
491
+ 481 69.64656965
492
+ 482 69.50207469
493
+ 483 69.56521739
494
+ 484 69.62809917
495
+ 485 69.48453608
496
+ 486 69.54732510
497
+ 487 69.60985626
498
+ 488 69.46721311
499
+ 489 69.52965235
500
+ 490 69.59183673
501
+ 491 69.65376782
502
+ 492 69.71544715
503
+ 493 69.77687627
504
+ 494 69.83805668
505
+ 495 69.89898990
506
+ 496 69.95967742
507
+ 497 70.02012072
508
+ 498 69.87951807
509
+ 499 69.73947896
510
+ 500 69.80000000
511
+ 501 69.86027944
512
+ 502 69.92031873
513
+ 503 69.98011928
514
+ 504 70.03968254
515
+ 505 70.09900990
516
+ 506 70.15810277
517
+ 507 70.21696252
518
+ 508 70.27559055
519
+ 509 70.33398821
520
+ 510 70.39215686
521
+ 511 70.45009785
522
+ 512 70.31250000
523
+ 513 70.17543860
524
+ 514 70.23346304
525
+ 515 70.29126214
526
+ 516 70.15503876
527
+ 517 70.01934236
528
+ 518 70.07722008
529
+ 519 69.94219653
530
+ 520 70.00000000
531
+ 521 69.86564299
532
+ 522 69.73180077
533
+ 523 69.78967495
534
+ 524 69.84732824
535
+ 525 69.90476190
536
+ 526 69.77186312
537
+ 527 69.63946869
538
+ 528 69.69696970
539
+ 529 69.75425331
540
+ 530 69.81132075
541
+ 531 69.86817326
542
+ 532 69.92481203
543
+ 533 69.79362101
544
+ 534 69.85018727
545
+ 535 69.90654206
546
+ 536 69.96268657
547
+ 537 70.01862197
548
+ 538 70.07434944
549
+ 539 69.94434137
550
+ 540 70.00000000
551
+ 541 70.05545287
552
+ 542 70.11070111
553
+ 543 70.16574586
554
+ 544 70.22058824
555
+ 545 70.27522936
556
+ 546 70.32967033
557
+ 547 70.38391225
558
+ 548 70.43795620
559
+ 549 70.49180328
560
+ 550 70.36363636
561
+ 551 70.41742287
562
+ 552 70.28985507
563
+ 553 70.34358047
564
+ 554 70.39711191
565
+ 555 70.27027027
566
+ 556 70.32374101
567
+ 557 70.19748654
568
+ 558 70.07168459
569
+ 559 69.94633274
570
+ 560 70.00000000
571
+ 561 70.05347594
572
+ 562 69.92882562
573
+ 563 69.98223801
574
+ 564 70.03546099
575
+ 565 69.91150442
576
+ 566 69.78798587
577
+ 567 69.84126984
578
+ 568 69.89436620
579
+ 569 69.77152900
580
+ 570 69.82456140
581
+ 571 69.87740806
582
+ 572 69.93006993
583
+ 573 69.98254799
584
+ 574 70.03484321
585
+ 575 70.08695652
586
+ 576 70.13888889
587
+ 577 70.01733102
588
+ 578 70.06920415
589
+ 579 70.12089810
590
+ 580 70.00000000
591
+ 581 69.87951807
592
+ 582 69.93127148
593
+ 583 69.98284734
594
+ 584 70.03424658
595
+ 585 70.08547009
596
+ 586 70.13651877
597
+ 587 70.18739353
598
+ 588 70.06802721
599
+ 589 69.94906621
600
+ 590 69.83050847
601
+ 591 69.88155668
602
+ 592 69.93243243
603
+ 593 69.81450253
604
+ 594 69.69696970
605
+ 595 69.74789916
606
+ 596 69.79865772
607
+ 597 69.84924623
608
+ 598 69.89966555
609
+ 599 69.78297162
610
+ 600 69.83333333
611
+ 601 69.88352745
612
+ 602 69.93355482
613
+ 603 69.81757877
614
+ 604 69.86754967
615
+ 605 69.91735537
616
+ 606 69.80198020
617
+ 607 69.85172982
618
+ 608 69.73684211
619
+ 609 69.78653530
620
+ 610 69.83606557
621
+ 611 69.88543372
622
+ 612 69.77124183
623
+ 613 69.82055465
624
+ 614 69.70684039
625
+ 615 69.75609756
626
+ 616 69.64285714
627
+ 617 69.52998379
628
+ 618 69.57928803
629
+ 619 69.62843296
630
+ 620 69.67741935
631
+ 621 69.72624799
632
+ 622 69.77491961
633
+ 623 69.82343499
634
+ 624 69.87179487
635
+ 625 69.92000000
636
+ 626 69.80830671
637
+ 627 69.85645933
638
+ 628 69.90445860
639
+ 629 69.79332273
640
+ 630 69.84126984
641
+ 631 69.88906498
642
+ 632 69.77848101
643
+ 633 69.82622433
644
+ 634 69.87381703
645
+ 635 69.92125984
646
+ 636 69.96855346
647
+ 637 70.01569859
648
+ 638 70.06269592
649
+ 639 70.10954617
650
+ 640 70.00000000
651
+ 641 70.04680187
652
+ 642 69.93769470
653
+ 643 69.98444790
654
+ 644 70.03105590
655
+ 645 69.92248062
656
+ 646 69.96904025
657
+ 647 70.01545595
658
+ 648 70.06172840
659
+ 649 69.95377504
660
+ 650 70.00000000
661
+ 651 70.04608295
662
+ 652 70.09202454
663
+ 653 70.13782542
664
+ 654 70.18348624
665
+ 655 70.22900763
666
+ 656 70.27439024
667
+ 657 70.31963470
668
+ 658 70.21276596
669
+ 659 70.10622155
670
+ 660 70.15151515
671
+ 661 70.19667171
672
+ 662 70.24169184
673
+ 663 70.28657617
674
+ 664 70.33132530
675
+ 665 70.37593985
676
+ 666 70.42042042
677
+ 667 70.46476762
678
+ 668 70.50898204
679
+ 669 70.55306428
680
+ 670 70.59701493
681
+ 671 70.64083458
682
+ 672 70.68452381
683
+ 673 70.72808321
684
+ 674 70.77151335
685
+ 675 70.81481481
686
+ 676 70.71005917
687
+ 677 70.75332349
688
+ 678 70.79646018
689
+ 679 70.83946981
690
+ 680 70.88235294
691
+ 681 70.77826725
692
+ 682 70.82111437
693
+ 683 70.86383602
694
+ 684 70.90643275
695
+ 685 70.80291971
696
+ 686 70.84548105
697
+ 687 70.88791849
698
+ 688 70.93023256
699
+ 689 70.97242380
700
+ 690 71.01449275
701
+ 691 71.05643994
702
+ 692 71.09826590
703
+ 693 70.99567100
704
+ 694 71.03746398
705
+ 695 71.07913669
706
+ 696 70.97701149
707
+ 697 71.01865136
708
+ 698 71.06017192
709
+ 699 71.10157368
710
+ 700 71.14285714
711
+ 701 71.18402282
712
+ 702 71.22507123
713
+ 703 71.26600284
714
+ 704 71.30681818
715
+ 705 71.34751773
716
+ 706 71.38810198
717
+ 707 71.42857143
718
+ 708 71.46892655
719
+ 709 71.50916784
720
+ 710 71.54929577
721
+ 711 71.44866385
722
+ 712 71.48876404
723
+ 713 71.52875175
724
+ 714 71.56862745
725
+ 715 71.46853147
726
+ 716 71.50837989
727
+ 717 71.54811715
728
+ 718 71.58774373
729
+ 719 71.48817803
730
+ 720 71.52777778
731
+ 721 71.42857143
732
+ 722 71.46814404
733
+ 723 71.50760719
734
+ 724 71.54696133
735
+ 725 71.58620690
736
+ 726 71.62534435
737
+ 727 71.66437414
738
+ 728 71.70329670
739
+ 729 71.74211248
740
+ 730 71.64383562
741
+ 731 71.68262654
742
+ 732 71.58469945
743
+ 733 71.62346521
744
+ 734 71.66212534
745
+ 735 71.70068027
746
+ 736 71.73913043
747
+ 737 71.64179104
748
+ 738 71.54471545
749
+ 739 71.44790257
750
+ 740 71.48648649
751
+ 741 71.52496626
752
+ 742 71.56334232
753
+ 743 71.60161507
754
+ 744 71.63978495
755
+ 745 71.54362416
756
+ 746 71.58176944
757
+ 747 71.48594378
758
+ 748 71.52406417
759
+ 749 71.56208278
760
+ 750 71.60000000
scores/Hammer2.1-7b-q4_k_m.mmlu ADDED
@@ -0,0 +1,763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
2
+ common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
3
+
4
+ system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
5
+ multiple_choice_score: there are 1548 tasks in prompt
6
+ multiple_choice_score: selecting 750 random tasks from 1548 tasks available
7
+ multiple_choice_score: preparing task data...done
8
+ multiple_choice_score : calculating TruthfulQA score over 750 tasks.
9
+
10
+ task acc_norm
11
+ 1 100.00000000
12
+ 2 50.00000000
13
+ 3 33.33333333
14
+ 4 50.00000000
15
+ 5 40.00000000
16
+ 6 33.33333333
17
+ 7 42.85714286
18
+ 8 50.00000000
19
+ 9 44.44444444
20
+ 10 40.00000000
21
+ 11 36.36363636
22
+ 12 41.66666667
23
+ 13 46.15384615
24
+ 14 42.85714286
25
+ 15 40.00000000
26
+ 16 43.75000000
27
+ 17 41.17647059
28
+ 18 38.88888889
29
+ 19 42.10526316
30
+ 20 45.00000000
31
+ 21 42.85714286
32
+ 22 40.90909091
33
+ 23 39.13043478
34
+ 24 37.50000000
35
+ 25 40.00000000
36
+ 26 38.46153846
37
+ 27 40.74074074
38
+ 28 39.28571429
39
+ 29 41.37931034
40
+ 30 43.33333333
41
+ 31 45.16129032
42
+ 32 46.87500000
43
+ 33 45.45454545
44
+ 34 47.05882353
45
+ 35 48.57142857
46
+ 36 47.22222222
47
+ 37 48.64864865
48
+ 38 50.00000000
49
+ 39 48.71794872
50
+ 40 47.50000000
51
+ 41 46.34146341
52
+ 42 45.23809524
53
+ 43 46.51162791
54
+ 44 45.45454545
55
+ 45 46.66666667
56
+ 46 45.65217391
57
+ 47 44.68085106
58
+ 48 45.83333333
59
+ 49 46.93877551
60
+ 50 46.00000000
61
+ 51 45.09803922
62
+ 52 44.23076923
63
+ 53 43.39622642
64
+ 54 42.59259259
65
+ 55 43.63636364
66
+ 56 44.64285714
67
+ 57 43.85964912
68
+ 58 44.82758621
69
+ 59 45.76271186
70
+ 60 45.00000000
71
+ 61 44.26229508
72
+ 62 43.54838710
73
+ 63 42.85714286
74
+ 64 42.18750000
75
+ 65 41.53846154
76
+ 66 42.42424242
77
+ 67 43.28358209
78
+ 68 42.64705882
79
+ 69 42.02898551
80
+ 70 42.85714286
81
+ 71 43.66197183
82
+ 72 43.05555556
83
+ 73 42.46575342
84
+ 74 43.24324324
85
+ 75 44.00000000
86
+ 76 44.73684211
87
+ 77 45.45454545
88
+ 78 46.15384615
89
+ 79 45.56962025
90
+ 80 46.25000000
91
+ 81 46.91358025
92
+ 82 46.34146341
93
+ 83 46.98795181
94
+ 84 47.61904762
95
+ 85 48.23529412
96
+ 86 47.67441860
97
+ 87 48.27586207
98
+ 88 48.86363636
99
+ 89 48.31460674
100
+ 90 47.77777778
101
+ 91 47.25274725
102
+ 92 46.73913043
103
+ 93 46.23655914
104
+ 94 46.80851064
105
+ 95 46.31578947
106
+ 96 46.87500000
107
+ 97 46.39175258
108
+ 98 45.91836735
109
+ 99 45.45454545
110
+ 100 45.00000000
111
+ 101 45.54455446
112
+ 102 45.09803922
113
+ 103 44.66019417
114
+ 104 45.19230769
115
+ 105 45.71428571
116
+ 106 46.22641509
117
+ 107 45.79439252
118
+ 108 46.29629630
119
+ 109 45.87155963
120
+ 110 45.45454545
121
+ 111 45.04504505
122
+ 112 45.53571429
123
+ 113 45.13274336
124
+ 114 45.61403509
125
+ 115 46.08695652
126
+ 116 45.68965517
127
+ 117 45.29914530
128
+ 118 44.91525424
129
+ 119 44.53781513
130
+ 120 44.16666667
131
+ 121 44.62809917
132
+ 122 45.08196721
133
+ 123 44.71544715
134
+ 124 44.35483871
135
+ 125 44.80000000
136
+ 126 44.44444444
137
+ 127 44.09448819
138
+ 128 43.75000000
139
+ 129 43.41085271
140
+ 130 43.07692308
141
+ 131 43.51145038
142
+ 132 43.18181818
143
+ 133 42.85714286
144
+ 134 42.53731343
145
+ 135 42.22222222
146
+ 136 42.64705882
147
+ 137 43.06569343
148
+ 138 42.75362319
149
+ 139 43.16546763
150
+ 140 43.57142857
151
+ 141 43.26241135
152
+ 142 43.66197183
153
+ 143 44.05594406
154
+ 144 43.75000000
155
+ 145 44.13793103
156
+ 146 44.52054795
157
+ 147 44.21768707
158
+ 148 44.59459459
159
+ 149 44.96644295
160
+ 150 44.66666667
161
+ 151 44.37086093
162
+ 152 44.73684211
163
+ 153 44.44444444
164
+ 154 44.15584416
165
+ 155 43.87096774
166
+ 156 43.58974359
167
+ 157 43.94904459
168
+ 158 44.30379747
169
+ 159 44.02515723
170
+ 160 44.37500000
171
+ 161 44.09937888
172
+ 162 43.82716049
173
+ 163 43.55828221
174
+ 164 43.29268293
175
+ 165 43.63636364
176
+ 166 43.37349398
177
+ 167 43.71257485
178
+ 168 43.45238095
179
+ 169 43.78698225
180
+ 170 43.52941176
181
+ 171 43.85964912
182
+ 172 43.60465116
183
+ 173 43.35260116
184
+ 174 43.10344828
185
+ 175 42.85714286
186
+ 176 42.61363636
187
+ 177 42.37288136
188
+ 178 42.69662921
189
+ 179 43.01675978
190
+ 180 42.77777778
191
+ 181 42.54143646
192
+ 182 42.30769231
193
+ 183 42.62295082
194
+ 184 42.39130435
195
+ 185 42.70270270
196
+ 186 43.01075269
197
+ 187 42.78074866
198
+ 188 42.55319149
199
+ 189 42.85714286
200
+ 190 42.63157895
201
+ 191 42.93193717
202
+ 192 42.70833333
203
+ 193 42.48704663
204
+ 194 42.26804124
205
+ 195 42.56410256
206
+ 196 42.34693878
207
+ 197 42.13197970
208
+ 198 41.91919192
209
+ 199 41.70854271
210
+ 200 41.50000000
211
+ 201 41.29353234
212
+ 202 41.08910891
213
+ 203 40.88669951
214
+ 204 40.68627451
215
+ 205 40.97560976
216
+ 206 40.77669903
217
+ 207 41.06280193
218
+ 208 40.86538462
219
+ 209 40.66985646
220
+ 210 40.47619048
221
+ 211 40.75829384
222
+ 212 40.56603774
223
+ 213 40.37558685
224
+ 214 40.18691589
225
+ 215 40.46511628
226
+ 216 40.74074074
227
+ 217 40.55299539
228
+ 218 40.82568807
229
+ 219 41.09589041
230
+ 220 41.36363636
231
+ 221 41.62895928
232
+ 222 41.44144144
233
+ 223 41.70403587
234
+ 224 41.96428571
235
+ 225 42.22222222
236
+ 226 42.47787611
237
+ 227 42.29074890
238
+ 228 42.54385965
239
+ 229 42.35807860
240
+ 230 42.17391304
241
+ 231 41.99134199
242
+ 232 42.24137931
243
+ 233 42.06008584
244
+ 234 42.30769231
245
+ 235 42.12765957
246
+ 236 41.94915254
247
+ 237 41.77215190
248
+ 238 41.59663866
249
+ 239 41.42259414
250
+ 240 41.66666667
251
+ 241 41.49377593
252
+ 242 41.73553719
253
+ 243 41.97530864
254
+ 244 41.80327869
255
+ 245 42.04081633
256
+ 246 41.86991870
257
+ 247 42.10526316
258
+ 248 42.33870968
259
+ 249 42.16867470
260
+ 250 42.00000000
261
+ 251 41.83266932
262
+ 252 41.66666667
263
+ 253 41.50197628
264
+ 254 41.33858268
265
+ 255 41.17647059
266
+ 256 41.40625000
267
+ 257 41.63424125
268
+ 258 41.47286822
269
+ 259 41.31274131
270
+ 260 41.15384615
271
+ 261 40.99616858
272
+ 262 40.83969466
273
+ 263 40.68441065
274
+ 264 40.90909091
275
+ 265 40.75471698
276
+ 266 40.97744361
277
+ 267 40.82397004
278
+ 268 40.67164179
279
+ 269 40.89219331
280
+ 270 41.11111111
281
+ 271 40.95940959
282
+ 272 41.17647059
283
+ 273 41.02564103
284
+ 274 40.87591241
285
+ 275 40.72727273
286
+ 276 40.94202899
287
+ 277 41.15523466
288
+ 278 41.36690647
289
+ 279 41.21863799
290
+ 280 41.07142857
291
+ 281 40.92526690
292
+ 282 40.78014184
293
+ 283 40.63604240
294
+ 284 40.49295775
295
+ 285 40.35087719
296
+ 286 40.20979021
297
+ 287 40.41811847
298
+ 288 40.27777778
299
+ 289 40.13840830
300
+ 290 40.34482759
301
+ 291 40.54982818
302
+ 292 40.75342466
303
+ 293 40.95563140
304
+ 294 40.81632653
305
+ 295 40.67796610
306
+ 296 40.54054054
307
+ 297 40.40404040
308
+ 298 40.60402685
309
+ 299 40.80267559
310
+ 300 41.00000000
311
+ 301 40.86378738
312
+ 302 40.72847682
313
+ 303 40.59405941
314
+ 304 40.46052632
315
+ 305 40.65573770
316
+ 306 40.52287582
317
+ 307 40.71661238
318
+ 308 40.58441558
319
+ 309 40.45307443
320
+ 310 40.64516129
321
+ 311 40.83601286
322
+ 312 41.02564103
323
+ 313 41.21405751
324
+ 314 41.40127389
325
+ 315 41.58730159
326
+ 316 41.45569620
327
+ 317 41.64037855
328
+ 318 41.50943396
329
+ 319 41.69278997
330
+ 320 41.56250000
331
+ 321 41.43302181
332
+ 322 41.30434783
333
+ 323 41.17647059
334
+ 324 41.04938272
335
+ 325 41.23076923
336
+ 326 41.10429448
337
+ 327 41.28440367
338
+ 328 41.46341463
339
+ 329 41.33738602
340
+ 330 41.21212121
341
+ 331 41.08761329
342
+ 332 41.26506024
343
+ 333 41.14114114
344
+ 334 41.01796407
345
+ 335 41.19402985
346
+ 336 41.36904762
347
+ 337 41.24629080
348
+ 338 41.12426036
349
+ 339 41.00294985
350
+ 340 41.17647059
351
+ 341 41.34897361
352
+ 342 41.52046784
353
+ 343 41.39941691
354
+ 344 41.27906977
355
+ 345 41.15942029
356
+ 346 41.32947977
357
+ 347 41.49855908
358
+ 348 41.37931034
359
+ 349 41.54727794
360
+ 350 41.71428571
361
+ 351 41.59544160
362
+ 352 41.47727273
363
+ 353 41.64305949
364
+ 354 41.52542373
365
+ 355 41.40845070
366
+ 356 41.57303371
367
+ 357 41.73669468
368
+ 358 41.89944134
369
+ 359 41.78272981
370
+ 360 41.94444444
371
+ 361 42.10526316
372
+ 362 42.26519337
373
+ 363 42.14876033
374
+ 364 42.03296703
375
+ 365 41.91780822
376
+ 366 42.07650273
377
+ 367 41.96185286
378
+ 368 41.84782609
379
+ 369 41.73441734
380
+ 370 41.89189189
381
+ 371 42.04851752
382
+ 372 41.93548387
383
+ 373 41.82305630
384
+ 374 41.97860963
385
+ 375 42.13333333
386
+ 376 42.02127660
387
+ 377 42.17506631
388
+ 378 42.32804233
389
+ 379 42.21635884
390
+ 380 42.10526316
391
+ 381 42.25721785
392
+ 382 42.40837696
393
+ 383 42.55874674
394
+ 384 42.44791667
395
+ 385 42.59740260
396
+ 386 42.48704663
397
+ 387 42.37726098
398
+ 388 42.26804124
399
+ 389 42.41645244
400
+ 390 42.56410256
401
+ 391 42.71099744
402
+ 392 42.85714286
403
+ 393 42.74809160
404
+ 394 42.63959391
405
+ 395 42.78481013
406
+ 396 42.92929293
407
+ 397 43.07304786
408
+ 398 42.96482412
409
+ 399 43.10776942
410
+ 400 43.00000000
411
+ 401 43.14214464
412
+ 402 43.03482587
413
+ 403 42.92803970
414
+ 404 42.82178218
415
+ 405 42.71604938
416
+ 406 42.61083744
417
+ 407 42.50614251
418
+ 408 42.40196078
419
+ 409 42.54278729
420
+ 410 42.43902439
421
+ 411 42.33576642
422
+ 412 42.23300971
423
+ 413 42.13075061
424
+ 414 42.27053140
425
+ 415 42.16867470
426
+ 416 42.30769231
427
+ 417 42.20623501
428
+ 418 42.10526316
429
+ 419 42.24343675
430
+ 420 42.38095238
431
+ 421 42.28028504
432
+ 422 42.18009479
433
+ 423 42.08037825
434
+ 424 42.21698113
435
+ 425 42.35294118
436
+ 426 42.48826291
437
+ 427 42.38875878
438
+ 428 42.28971963
439
+ 429 42.42424242
440
+ 430 42.32558140
441
+ 431 42.45939675
442
+ 432 42.59259259
443
+ 433 42.49422633
444
+ 434 42.39631336
445
+ 435 42.29885057
446
+ 436 42.20183486
447
+ 437 42.10526316
448
+ 438 42.23744292
449
+ 439 42.36902050
450
+ 440 42.27272727
451
+ 441 42.17687075
452
+ 442 42.08144796
453
+ 443 41.98645598
454
+ 444 41.89189189
455
+ 445 41.79775281
456
+ 446 41.70403587
457
+ 447 41.61073826
458
+ 448 41.51785714
459
+ 449 41.42538976
460
+ 450 41.55555556
461
+ 451 41.46341463
462
+ 452 41.59292035
463
+ 453 41.72185430
464
+ 454 41.62995595
465
+ 455 41.53846154
466
+ 456 41.66666667
467
+ 457 41.79431072
468
+ 458 41.70305677
469
+ 459 41.61220044
470
+ 460 41.73913043
471
+ 461 41.64859002
472
+ 462 41.55844156
473
+ 463 41.46868251
474
+ 464 41.59482759
475
+ 465 41.50537634
476
+ 466 41.41630901
477
+ 467 41.54175589
478
+ 468 41.45299145
479
+ 469 41.57782516
480
+ 470 41.48936170
481
+ 471 41.40127389
482
+ 472 41.31355932
483
+ 473 41.22621564
484
+ 474 41.13924051
485
+ 475 41.05263158
486
+ 476 41.17647059
487
+ 477 41.09014675
488
+ 478 41.21338912
489
+ 479 41.33611691
490
+ 480 41.45833333
491
+ 481 41.37214137
492
+ 482 41.28630705
493
+ 483 41.20082816
494
+ 484 41.11570248
495
+ 485 41.03092784
496
+ 486 40.94650206
497
+ 487 41.06776181
498
+ 488 40.98360656
499
+ 489 40.89979550
500
+ 490 41.02040816
501
+ 491 40.93686354
502
+ 492 40.85365854
503
+ 493 40.77079108
504
+ 494 40.89068826
505
+ 495 41.01010101
506
+ 496 41.12903226
507
+ 497 41.04627767
508
+ 498 40.96385542
509
+ 499 40.88176353
510
+ 500 40.80000000
511
+ 501 40.71856287
512
+ 502 40.63745020
513
+ 503 40.55666004
514
+ 504 40.67460317
515
+ 505 40.59405941
516
+ 506 40.71146245
517
+ 507 40.63116371
518
+ 508 40.74803150
519
+ 509 40.66797642
520
+ 510 40.78431373
521
+ 511 40.70450098
522
+ 512 40.62500000
523
+ 513 40.54580897
524
+ 514 40.46692607
525
+ 515 40.38834951
526
+ 516 40.50387597
527
+ 517 40.61895551
528
+ 518 40.73359073
529
+ 519 40.84778420
530
+ 520 40.96153846
531
+ 521 40.88291747
532
+ 522 40.80459770
533
+ 523 40.72657744
534
+ 524 40.83969466
535
+ 525 40.76190476
536
+ 526 40.68441065
537
+ 527 40.79696395
538
+ 528 40.71969697
539
+ 529 40.64272212
540
+ 530 40.75471698
541
+ 531 40.67796610
542
+ 532 40.60150376
543
+ 533 40.52532833
544
+ 534 40.44943820
545
+ 535 40.56074766
546
+ 536 40.48507463
547
+ 537 40.59590317
548
+ 538 40.70631970
549
+ 539 40.63079777
550
+ 540 40.55555556
551
+ 541 40.48059150
552
+ 542 40.59040590
553
+ 543 40.51565378
554
+ 544 40.44117647
555
+ 545 40.36697248
556
+ 546 40.29304029
557
+ 547 40.40219378
558
+ 548 40.51094891
559
+ 549 40.43715847
560
+ 550 40.36363636
561
+ 551 40.47186933
562
+ 552 40.39855072
563
+ 553 40.32549729
564
+ 554 40.43321300
565
+ 555 40.36036036
566
+ 556 40.46762590
567
+ 557 40.39497307
568
+ 558 40.50179211
569
+ 559 40.42933810
570
+ 560 40.35714286
571
+ 561 40.28520499
572
+ 562 40.39145907
573
+ 563 40.49733570
574
+ 564 40.42553191
575
+ 565 40.53097345
576
+ 566 40.45936396
577
+ 567 40.38800705
578
+ 568 40.31690141
579
+ 569 40.24604569
580
+ 570 40.35087719
581
+ 571 40.28021016
582
+ 572 40.38461538
583
+ 573 40.31413613
584
+ 574 40.24390244
585
+ 575 40.17391304
586
+ 576 40.27777778
587
+ 577 40.38128250
588
+ 578 40.31141869
589
+ 579 40.24179620
590
+ 580 40.34482759
591
+ 581 40.44750430
592
+ 582 40.37800687
593
+ 583 40.48027444
594
+ 584 40.58219178
595
+ 585 40.51282051
596
+ 586 40.61433447
597
+ 587 40.54514480
598
+ 588 40.47619048
599
+ 589 40.40747029
600
+ 590 40.33898305
601
+ 591 40.27072758
602
+ 592 40.20270270
603
+ 593 40.13490725
604
+ 594 40.23569024
605
+ 595 40.16806723
606
+ 596 40.26845638
607
+ 597 40.20100503
608
+ 598 40.13377926
609
+ 599 40.23372287
610
+ 600 40.16666667
611
+ 601 40.26622296
612
+ 602 40.19933555
613
+ 603 40.29850746
614
+ 604 40.23178808
615
+ 605 40.33057851
616
+ 606 40.26402640
617
+ 607 40.36243822
618
+ 608 40.29605263
619
+ 609 40.22988506
620
+ 610 40.16393443
621
+ 611 40.26186579
622
+ 612 40.19607843
623
+ 613 40.13050571
624
+ 614 40.22801303
625
+ 615 40.32520325
626
+ 616 40.42207792
627
+ 617 40.35656402
628
+ 618 40.45307443
629
+ 619 40.38772213
630
+ 620 40.32258065
631
+ 621 40.25764895
632
+ 622 40.35369775
633
+ 623 40.28892456
634
+ 624 40.22435897
635
+ 625 40.16000000
636
+ 626 40.25559105
637
+ 627 40.19138756
638
+ 628 40.12738854
639
+ 629 40.22257552
640
+ 630 40.15873016
641
+ 631 40.25356577
642
+ 632 40.34810127
643
+ 633 40.28436019
644
+ 634 40.22082019
645
+ 635 40.15748031
646
+ 636 40.09433962
647
+ 637 40.03139717
648
+ 638 39.96865204
649
+ 639 40.06259781
650
+ 640 40.15625000
651
+ 641 40.09360374
652
+ 642 40.18691589
653
+ 643 40.12441680
654
+ 644 40.21739130
655
+ 645 40.15503876
656
+ 646 40.09287926
657
+ 647 40.03091190
658
+ 648 39.96913580
659
+ 649 39.90755008
660
+ 650 39.84615385
661
+ 651 39.78494624
662
+ 652 39.87730061
663
+ 653 39.81623277
664
+ 654 39.90825688
665
+ 655 39.84732824
666
+ 656 39.93902439
667
+ 657 39.87823440
668
+ 658 39.96960486
669
+ 659 39.90895296
670
+ 660 39.84848485
671
+ 661 39.78819970
672
+ 662 39.72809668
673
+ 663 39.66817496
674
+ 664 39.75903614
675
+ 665 39.84962406
676
+ 666 39.78978979
677
+ 667 39.88005997
678
+ 668 39.97005988
679
+ 669 40.05979073
680
+ 670 40.00000000
681
+ 671 39.94038748
682
+ 672 39.88095238
683
+ 673 39.82169391
684
+ 674 39.76261128
685
+ 675 39.70370370
686
+ 676 39.64497041
687
+ 677 39.58641064
688
+ 678 39.52802360
689
+ 679 39.46980854
690
+ 680 39.55882353
691
+ 681 39.64757709
692
+ 682 39.58944282
693
+ 683 39.53147877
694
+ 684 39.47368421
695
+ 685 39.41605839
696
+ 686 39.35860058
697
+ 687 39.30131004
698
+ 688 39.24418605
699
+ 689 39.18722787
700
+ 690 39.13043478
701
+ 691 39.21852388
702
+ 692 39.16184971
703
+ 693 39.10533911
704
+ 694 39.04899135
705
+ 695 38.99280576
706
+ 696 38.93678161
707
+ 697 39.02439024
708
+ 698 38.96848138
709
+ 699 39.05579399
710
+ 700 39.00000000
711
+ 701 39.08701854
712
+ 702 39.17378917
713
+ 703 39.26031294
714
+ 704 39.34659091
715
+ 705 39.29078014
716
+ 706 39.23512748
717
+ 707 39.17963225
718
+ 708 39.12429379
719
+ 709 39.21015515
720
+ 710 39.29577465
721
+ 711 39.38115331
722
+ 712 39.32584270
723
+ 713 39.41093969
724
+ 714 39.49579832
725
+ 715 39.44055944
726
+ 716 39.38547486
727
+ 717 39.33054393
728
+ 718 39.27576602
729
+ 719 39.36022253
730
+ 720 39.30555556
731
+ 721 39.25104022
732
+ 722 39.33518006
733
+ 723 39.41908714
734
+ 724 39.36464088
735
+ 725 39.44827586
736
+ 726 39.39393939
737
+ 727 39.33975241
738
+ 728 39.28571429
739
+ 729 39.23182442
740
+ 730 39.17808219
741
+ 731 39.12448700
742
+ 732 39.20765027
743
+ 733 39.29058663
744
+ 734 39.37329700
745
+ 735 39.31972789
746
+ 736 39.26630435
747
+ 737 39.34871099
748
+ 738 39.29539295
749
+ 739 39.37753721
750
+ 740 39.32432432
751
+ 741 39.27125506
752
+ 742 39.35309973
753
+ 743 39.30013459
754
+ 744 39.38172043
755
+ 745 39.32885906
756
+ 746 39.27613941
757
+ 747 39.22356091
758
+ 748 39.17112299
759
+ 749 39.11882510
760
+ 750 39.20000000
761
+
762
+ Final result: 39.2000 +/- 1.7838
763
+ Random chance: 25.0000 +/- 1.5822
scores/Hammer2.1-7b-q4_k_m.ppx ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ====== Perplexity statistics ======
2
+ Mean PPL(Q) : 9.626296 ± 0.068786
3
+ Mean PPL(base) : 9.366577 ± 0.066397
4
+ Cor(ln(PPL(Q)), ln(PPL(base))): 99.41%
5
+ Mean ln(PPL(Q)/PPL(base)) : 0.027351 ± 0.000775
6
+ Mean PPL(Q)/PPL(base) : 1.027728 ± 0.000797
7
+ Mean PPL(Q)-PPL(base) : 0.259719 ± 0.007718
8
+
9
+ ====== KL divergence statistics ======
10
+ Mean KLD: 0.036492 ± 0.000124
11
+ Maximum KLD: 5.495077
12
+ 99.9% KLD: 0.394759
13
+ 99.0% KLD: 0.186226
14
+ 99.0% KLD: 0.186226
15
+ Median KLD: 0.026454
16
+ 10.0% KLD: 0.000503
17
+ 5.0% KLD: 0.000101
18
+ 1.0% KLD: 0.000007
19
+ Minimum KLD: -0.000047
20
+
21
+ ====== Token probability statistics ======
22
+ Mean Δp: -0.305 ± 0.013 %
23
+ Maximum Δp: 91.526%
24
+ 99.9% Δp: 25.028%
25
+ 99.0% Δp: 14.225%
26
+ 95.0% Δp: 7.335%
27
+ 90.0% Δp: 4.239%
28
+ 75.0% Δp: 0.692%
29
+ Median Δp: -0.005%
30
+ 25.0% Δp: -1.234%
31
+ 10.0% Δp: -5.184%
32
+ 5.0% Δp: -8.519%
33
+ 1.0% Δp: -16.353%
34
+ 0.1% Δp: -29.578%
35
+ Minimum Δp: -89.359%
36
+ RMS Δp : 4.987 ± 0.024 %
37
+ Same top p: 89.212 ± 0.080 %