File size: 7,823 Bytes
928d6cd 38ab69a 928d6cd d5d3198 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
---
base_model: h2oai/h2o-danube3-500m-base
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
---
# Uploaded model
base model pretrained on 2.5 epochs on no-prompt-oasst
hf (pretrained=appvoid/palmer-004-turbo-v1.2), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|---------------------------------------|------:|------|-----:|--------|---|-----:|---|-----:|
|arc_challenge | 1|none | 0|acc |↑ |0.3097|± |0.0135|
| | |none | 0|acc_norm|↑ |0.3464|± |0.0139|
|hellaswag | 1|none | 0|acc |↑ |0.4660|± |0.0050|
| | |none | 0|acc_norm|↑ |0.6130|± |0.0049|
|mmlu | 1|none | |acc |↑ |0.2728|± |0.0037|
| - humanities | 1|none | |acc |↑ |0.2593|± |0.0064|
| - formal_logic | 0|none | 0|acc |↑ |0.3175|± |0.0416|
| - high_school_european_history | 0|none | 0|acc |↑ |0.2970|± |0.0357|
| - high_school_us_history | 0|none | 0|acc |↑ |0.2941|± |0.0320|
| - high_school_world_history | 0|none | 0|acc |↑ |0.2489|± |0.0281|
| - international_law | 0|none | 0|acc |↑ |0.3306|± |0.0429|
| - jurisprudence | 0|none | 0|acc |↑ |0.2685|± |0.0428|
| - logical_fallacies | 0|none | 0|acc |↑ |0.2515|± |0.0341|
| - moral_disputes | 0|none | 0|acc |↑ |0.2486|± |0.0233|
| - moral_scenarios | 0|none | 0|acc |↑ |0.2380|± |0.0142|
| - philosophy | 0|none | 0|acc |↑ |0.2797|± |0.0255|
| - prehistory | 0|none | 0|acc |↑ |0.2593|± |0.0244|
| - professional_law | 0|none | 0|acc |↑ |0.2555|± |0.0111|
| - world_religions | 0|none | 0|acc |↑ |0.2339|± |0.0325|
| - other | 1|none | |acc |↑ |0.2813|± |0.0080|
| - business_ethics | 0|none | 0|acc |↑ |0.1700|± |0.0378|
| - clinical_knowledge | 0|none | 0|acc |↑ |0.3434|± |0.0292|
| - college_medicine | 0|none | 0|acc |↑ |0.3006|± |0.0350|
| - global_facts | 0|none | 0|acc |↑ |0.3600|± |0.0482|
| - human_aging | 0|none | 0|acc |↑ |0.1659|± |0.0250|
| - management | 0|none | 0|acc |↑ |0.3883|± |0.0483|
| - marketing | 0|none | 0|acc |↑ |0.2137|± |0.0269|
| - medical_genetics | 0|none | 0|acc |↑ |0.2500|± |0.0435|
| - miscellaneous | 0|none | 0|acc |↑ |0.2784|± |0.0160|
| - nutrition | 0|none | 0|acc |↑ |0.2386|± |0.0244|
| - professional_accounting | 0|none | 0|acc |↑ |0.2376|± |0.0254|
| - professional_medicine | 0|none | 0|acc |↑ |0.4265|± |0.0300|
| - virology | 0|none | 0|acc |↑ |0.3133|± |0.0361|
| - social sciences | 1|none | |acc |↑ |0.2847|± |0.0081|
| - econometrics | 0|none | 0|acc |↑ |0.2719|± |0.0419|
| - high_school_geography | 0|none | 0|acc |↑ |0.3788|± |0.0346|
| - high_school_government_and_politics| 0|none | 0|acc |↑ |0.2850|± |0.0326|
| - high_school_macroeconomics | 0|none | 0|acc |↑ |0.3359|± |0.0239|
| - high_school_microeconomics | 0|none | 0|acc |↑ |0.3361|± |0.0307|
| - high_school_psychology | 0|none | 0|acc |↑ |0.3064|± |0.0198|
| - human_sexuality | 0|none | 0|acc |↑ |0.2519|± |0.0381|
| - professional_psychology | 0|none | 0|acc |↑ |0.2190|± |0.0167|
| - public_relations | 0|none | 0|acc |↑ |0.3091|± |0.0443|
| - security_studies | 0|none | 0|acc |↑ |0.2449|± |0.0275|
| - sociology | 0|none | 0|acc |↑ |0.2388|± |0.0301|
| - us_foreign_policy | 0|none | 0|acc |↑ |0.2800|± |0.0451|
| - stem | 1|none | |acc |↑ |0.2731|± |0.0079|
| - abstract_algebra | 0|none | 0|acc |↑ |0.1900|± |0.0394|
| - anatomy | 0|none | 0|acc |↑ |0.1926|± |0.0341|
| - astronomy | 0|none | 0|acc |↑ |0.2829|± |0.0367|
| - college_biology | 0|none | 0|acc |↑ |0.2083|± |0.0340|
| - college_chemistry | 0|none | 0|acc |↑ |0.3400|± |0.0476|
| - college_computer_science | 0|none | 0|acc |↑ |0.3000|± |0.0461|
| - college_mathematics | 0|none | 0|acc |↑ |0.3100|± |0.0465|
| - college_physics | 0|none | 0|acc |↑ |0.2941|± |0.0453|
| - computer_security | 0|none | 0|acc |↑ |0.2400|± |0.0429|
| - conceptual_physics | 0|none | 0|acc |↑ |0.2553|± |0.0285|
| - electrical_engineering | 0|none | 0|acc |↑ |0.2828|± |0.0375|
| - elementary_mathematics | 0|none | 0|acc |↑ |0.2513|± |0.0223|
| - high_school_biology | 0|none | 0|acc |↑ |0.2935|± |0.0259|
| - high_school_chemistry | 0|none | 0|acc |↑ |0.2808|± |0.0316|
| - high_school_computer_science | 0|none | 0|acc |↑ |0.2400|± |0.0429|
| - high_school_mathematics | 0|none | 0|acc |↑ |0.2741|± |0.0272|
| - high_school_physics | 0|none | 0|acc |↑ |0.3775|± |0.0396|
| - high_school_statistics | 0|none | 0|acc |↑ |0.3194|± |0.0318|
| - machine_learning | 0|none | 0|acc |↑ |0.2321|± |0.0401|
|piqa | 1|none | 0|acc |↑ |0.7269|± |0.0104|
| | |none | 0|acc_norm|↑ |0.7242|± |0.0104|
|winogrande | 1|none | 0|acc |↑ |0.5848|± |0.0138|
| Groups |Version|Filter|n-shot|Metric| |Value | |Stderr|
|------------------|------:|------|------|------|---|-----:|---|-----:|
|mmlu | 1|none | |acc |↑ |0.2728|± |0.0037|
| - humanities | 1|none | |acc |↑ |0.2593|± |0.0064|
| - other | 1|none | |acc |↑ |0.2813|± |0.0080|
| - social sciences| 1|none | |acc |↑ |0.2847|± |0.0081|
| - stem | 1|none | |acc |↑ |0.2731|± |0.0079|
|