Evaluation
Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
---|---|---|---|---|---|---|---|
- medmcqa | Yaml | none | 0 | acc | 0.5408 | ± | 0.0077 |
none | 0 | acc_norm | 0.5408 | ± | 0.0077 | ||
- medqa_4options | Yaml | none | 0 | acc | 0.5711 | ± | 0.0139 |
none | 0 | acc_norm | 0.5711 | ± | 0.0139 | ||
- anatomy (mmlu) | 0 | none | 0 | acc | 0.6815 | ± | 0.0402 |
- clinical_knowledge (mmlu) | 0 | none | 0 | acc | 0.7434 | ± | 0.0269 |
- college_biology (mmlu) | 0 | none | 0 | acc | 0.8056 | ± | 0.0331 |
- college_medicine (mmlu) | 0 | none | 0 | acc | 0.6647 | ± | 0.0360 |
- medical_genetics (mmlu) | 0 | none | 0 | acc | 0.7300 | ± | 0.0446 |
- professional_medicine (mmlu) | 0 | none | 0 | acc | 0.7353 | ± | 0.0268 |
stem | N/A | none | 0 | acc_norm | 0.5478 | ± | 0.0067 |
none | 0 | acc | 0.5909 | ± | 0.0058 | ||
- pubmedqa | 1 | none | 0 | acc | 0.7620 | ± | 0.0191 |
Groups | Version | Filter | n-shot | Metric | Value | Stderr | |
---|---|---|---|---|---|---|---|
stem | N/A | none | 0 | acc_norm | 0.5478 | ± | 0.0067 |
none | 0 | acc | 0.5909 | ± | 0.0058 |
- Downloads last month
- 179
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.