|
--- |
|
license: apache-2.0 |
|
--- |
|
### Evaluation |
|
|
|
| Tasks |Version|Filter|n-shot| Metric |Value | |Stderr| |
|
|-------------------------------|-------|------|-----:|--------|-----:|---|-----:| |
|
| - medmcqa |Yaml |none | 0|acc |0.5408|± |0.0077| |
|
| | |none | 0|acc_norm|0.5408|± |0.0077| |
|
| - medqa_4options |Yaml |none | 0|acc |0.5711|± |0.0139| |
|
| | |none | 0|acc_norm|0.5711|± |0.0139| |
|
| - anatomy (mmlu) | 0|none | 0|acc |0.6815|± |0.0402| |
|
| - clinical_knowledge (mmlu) | 0|none | 0|acc |0.7434|± |0.0269| |
|
| - college_biology (mmlu) | 0|none | 0|acc |0.8056|± |0.0331| |
|
| - college_medicine (mmlu) | 0|none | 0|acc |0.6647|± |0.0360| |
|
| - medical_genetics (mmlu) | 0|none | 0|acc |0.7300|± |0.0446| |
|
| - professional_medicine (mmlu)| 0|none | 0|acc |0.7353|± |0.0268| |
|
|stem |N/A |none | 0|acc_norm|0.5478|± |0.0067| |
|
| | |none | 0|acc |0.5909|± |0.0058| |
|
| - pubmedqa | 1|none | 0|acc |0.7620|± |0.0191| |
|
|
|
|Groups|Version|Filter|n-shot| Metric |Value | |Stderr| |
|
|------|-------|------|-----:|--------|-----:|---|-----:| |
|
|stem |N/A |none | 0|acc_norm|0.5478|± |0.0067| |
|
| | |none | 0|acc |0.5909|± |0.0058| |
|
|
|
 |
|
|
