|
--- |
|
base_model: h2oai/h2o-danube3-500m-base |
|
language: |
|
- en |
|
license: apache-2.0 |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- unsloth |
|
- llama |
|
- trl |
|
- sft |
|
--- |
|
|
|
# Uploaded model |
|
|
|
base model pretrained on 2.5 epochs on no-prompt-oasst |
|
|
|
hf (pretrained=appvoid/palmer-004-turbo-v1.2), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8 |
|
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr| |
|
|---------------------------------------|------:|------|-----:|--------|---|-----:|---|-----:| |
|
|arc_challenge | 1|none | 0|acc |↑ |0.3097|± |0.0135| |
|
| | |none | 0|acc_norm|↑ |0.3464|± |0.0139| |
|
|hellaswag | 1|none | 0|acc |↑ |0.4660|± |0.0050| |
|
| | |none | 0|acc_norm|↑ |0.6130|± |0.0049| |
|
|mmlu | 1|none | |acc |↑ |0.2728|± |0.0037| |
|
| - humanities | 1|none | |acc |↑ |0.2593|± |0.0064| |
|
| - formal_logic | 0|none | 0|acc |↑ |0.3175|± |0.0416| |
|
| - high_school_european_history | 0|none | 0|acc |↑ |0.2970|± |0.0357| |
|
| - high_school_us_history | 0|none | 0|acc |↑ |0.2941|± |0.0320| |
|
| - high_school_world_history | 0|none | 0|acc |↑ |0.2489|± |0.0281| |
|
| - international_law | 0|none | 0|acc |↑ |0.3306|± |0.0429| |
|
| - jurisprudence | 0|none | 0|acc |↑ |0.2685|± |0.0428| |
|
| - logical_fallacies | 0|none | 0|acc |↑ |0.2515|± |0.0341| |
|
| - moral_disputes | 0|none | 0|acc |↑ |0.2486|± |0.0233| |
|
| - moral_scenarios | 0|none | 0|acc |↑ |0.2380|± |0.0142| |
|
| - philosophy | 0|none | 0|acc |↑ |0.2797|± |0.0255| |
|
| - prehistory | 0|none | 0|acc |↑ |0.2593|± |0.0244| |
|
| - professional_law | 0|none | 0|acc |↑ |0.2555|± |0.0111| |
|
| - world_religions | 0|none | 0|acc |↑ |0.2339|± |0.0325| |
|
| - other | 1|none | |acc |↑ |0.2813|± |0.0080| |
|
| - business_ethics | 0|none | 0|acc |↑ |0.1700|± |0.0378| |
|
| - clinical_knowledge | 0|none | 0|acc |↑ |0.3434|± |0.0292| |
|
| - college_medicine | 0|none | 0|acc |↑ |0.3006|± |0.0350| |
|
| - global_facts | 0|none | 0|acc |↑ |0.3600|± |0.0482| |
|
| - human_aging | 0|none | 0|acc |↑ |0.1659|± |0.0250| |
|
| - management | 0|none | 0|acc |↑ |0.3883|± |0.0483| |
|
| - marketing | 0|none | 0|acc |↑ |0.2137|± |0.0269| |
|
| - medical_genetics | 0|none | 0|acc |↑ |0.2500|± |0.0435| |
|
| - miscellaneous | 0|none | 0|acc |↑ |0.2784|± |0.0160| |
|
| - nutrition | 0|none | 0|acc |↑ |0.2386|± |0.0244| |
|
| - professional_accounting | 0|none | 0|acc |↑ |0.2376|± |0.0254| |
|
| - professional_medicine | 0|none | 0|acc |↑ |0.4265|± |0.0300| |
|
| - virology | 0|none | 0|acc |↑ |0.3133|± |0.0361| |
|
| - social sciences | 1|none | |acc |↑ |0.2847|± |0.0081| |
|
| - econometrics | 0|none | 0|acc |↑ |0.2719|± |0.0419| |
|
| - high_school_geography | 0|none | 0|acc |↑ |0.3788|± |0.0346| |
|
| - high_school_government_and_politics| 0|none | 0|acc |↑ |0.2850|± |0.0326| |
|
| - high_school_macroeconomics | 0|none | 0|acc |↑ |0.3359|± |0.0239| |
|
| - high_school_microeconomics | 0|none | 0|acc |↑ |0.3361|± |0.0307| |
|
| - high_school_psychology | 0|none | 0|acc |↑ |0.3064|± |0.0198| |
|
| - human_sexuality | 0|none | 0|acc |↑ |0.2519|± |0.0381| |
|
| - professional_psychology | 0|none | 0|acc |↑ |0.2190|± |0.0167| |
|
| - public_relations | 0|none | 0|acc |↑ |0.3091|± |0.0443| |
|
| - security_studies | 0|none | 0|acc |↑ |0.2449|± |0.0275| |
|
| - sociology | 0|none | 0|acc |↑ |0.2388|± |0.0301| |
|
| - us_foreign_policy | 0|none | 0|acc |↑ |0.2800|± |0.0451| |
|
| - stem | 1|none | |acc |↑ |0.2731|± |0.0079| |
|
| - abstract_algebra | 0|none | 0|acc |↑ |0.1900|± |0.0394| |
|
| - anatomy | 0|none | 0|acc |↑ |0.1926|± |0.0341| |
|
| - astronomy | 0|none | 0|acc |↑ |0.2829|± |0.0367| |
|
| - college_biology | 0|none | 0|acc |↑ |0.2083|± |0.0340| |
|
| - college_chemistry | 0|none | 0|acc |↑ |0.3400|± |0.0476| |
|
| - college_computer_science | 0|none | 0|acc |↑ |0.3000|± |0.0461| |
|
| - college_mathematics | 0|none | 0|acc |↑ |0.3100|± |0.0465| |
|
| - college_physics | 0|none | 0|acc |↑ |0.2941|± |0.0453| |
|
| - computer_security | 0|none | 0|acc |↑ |0.2400|± |0.0429| |
|
| - conceptual_physics | 0|none | 0|acc |↑ |0.2553|± |0.0285| |
|
| - electrical_engineering | 0|none | 0|acc |↑ |0.2828|± |0.0375| |
|
| - elementary_mathematics | 0|none | 0|acc |↑ |0.2513|± |0.0223| |
|
| - high_school_biology | 0|none | 0|acc |↑ |0.2935|± |0.0259| |
|
| - high_school_chemistry | 0|none | 0|acc |↑ |0.2808|± |0.0316| |
|
| - high_school_computer_science | 0|none | 0|acc |↑ |0.2400|± |0.0429| |
|
| - high_school_mathematics | 0|none | 0|acc |↑ |0.2741|± |0.0272| |
|
| - high_school_physics | 0|none | 0|acc |↑ |0.3775|± |0.0396| |
|
| - high_school_statistics | 0|none | 0|acc |↑ |0.3194|± |0.0318| |
|
| - machine_learning | 0|none | 0|acc |↑ |0.2321|± |0.0401| |
|
|piqa | 1|none | 0|acc |↑ |0.7269|± |0.0104| |
|
| | |none | 0|acc_norm|↑ |0.7242|± |0.0104| |
|
|winogrande | 1|none | 0|acc |↑ |0.5848|± |0.0138| |
|
|
|
| Groups |Version|Filter|n-shot|Metric| |Value | |Stderr| |
|
|------------------|------:|------|------|------|---|-----:|---|-----:| |
|
|mmlu | 1|none | |acc |↑ |0.2728|± |0.0037| |
|
| - humanities | 1|none | |acc |↑ |0.2593|± |0.0064| |
|
| - other | 1|none | |acc |↑ |0.2813|± |0.0080| |
|
| - social sciences| 1|none | |acc |↑ |0.2847|± |0.0081| |
|
| - stem | 1|none | |acc |↑ |0.2731|± |0.0079| |
|
|
