transformers safetensors llama text-generation medical mmlu medalpaca medmcqa conversational dataset:cais/mmlu dataset:medalpaca/medical_meadow_medqa dataset:medalpaca/medical_meadow_wikidoc dataset:openlifescienceai/medmcqa dataset:bigbio/med_qa dataset:GBaker/MedQA-USMLE-4-options dataset:medalpaca/medical_meadow_mmmlu dataset:medalpaca/medical_meadow_wikidoc_patient_information dataset:qiaojin/PubMedQA autotrain_compatible text-generation-inference endpoints_compatible 4-bit bitsandbytes region:us
---
datasets:
- cais/mmlu
- medalpaca/medical_meadow_medqa
- medalpaca/medical_meadow_wikidoc
- openlifescienceai/medmcqa
- bigbio/med_qa
- GBaker/MedQA-USMLE-4-options
- medalpaca/medical_meadow_mmmlu
- medalpaca/medical_meadow_wikidoc_patient_information
- qiaojin/PubMedQA
pipeline_tag: text-generation
tags:
- medical
- mmlu
- medalpaca
- medmcqa
---
### Evaluation results

| Dataset | GPT-3.5 | Tuned Llama 3 |
|:-------------:|:-----:|:----:|
| MMLU Clinical Knowledge   | 69.8| 74.34 |
| MMLU College Biology      | 72.2| 72.92 |
| MMLU College Medicine     | 61.3| 61.85 |
| MMLU Medical Genetics     | 70.0| 76.0  |
| MMLU Professional Medicine| 70.2| 72.43 |
| MMLU Anatomy              | 56.3| 61.48 |