gguf endpoints_compatible region:us conversational
---
{}
---
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


MedPhi-3-mini_v1 - GGUF
- Model creator: https://huggingface.co/ChenWeiLi/
- Original model: https://huggingface.co/ChenWeiLi/MedPhi-3-mini_v1/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [MedPhi-3-mini_v1.Q2_K.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q2_K.gguf) | Q2_K | 1.32GB |
| [MedPhi-3-mini_v1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q3_K_S.gguf) | Q3_K_S | 1.57GB |
| [MedPhi-3-mini_v1.Q3_K.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q3_K.gguf) | Q3_K | 1.82GB |
| [MedPhi-3-mini_v1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q3_K_M.gguf) | Q3_K_M | 1.82GB |
| [MedPhi-3-mini_v1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q3_K_L.gguf) | Q3_K_L | 1.94GB |
| [MedPhi-3-mini_v1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.IQ4_XS.gguf) | IQ4_XS | 1.93GB |
| [MedPhi-3-mini_v1.Q4_0.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q4_0.gguf) | Q4_0 | 2.03GB |
| [MedPhi-3-mini_v1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.IQ4_NL.gguf) | IQ4_NL | 2.04GB |
| [MedPhi-3-mini_v1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q4_K_S.gguf) | Q4_K_S | 2.04GB |
| [MedPhi-3-mini_v1.Q4_K.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q4_K.gguf) | Q4_K | 2.23GB |
| [MedPhi-3-mini_v1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q4_K_M.gguf) | Q4_K_M | 2.23GB |
| [MedPhi-3-mini_v1.Q4_1.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q4_1.gguf) | Q4_1 | 2.24GB |
| [MedPhi-3-mini_v1.Q5_0.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q5_0.gguf) | Q5_0 | 2.46GB |
| [MedPhi-3-mini_v1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q5_K_S.gguf) | Q5_K_S | 2.46GB |
| [MedPhi-3-mini_v1.Q5_K.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q5_K.gguf) | Q5_K | 2.62GB |
| [MedPhi-3-mini_v1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q5_K_M.gguf) | Q5_K_M | 2.62GB |
| [MedPhi-3-mini_v1.Q5_1.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q5_1.gguf) | Q5_1 | 2.68GB |
| [MedPhi-3-mini_v1.Q6_K.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q6_K.gguf) | Q6_K | 2.92GB |
| [MedPhi-3-mini_v1.Q8_0.gguf](https://huggingface.co/RichardErkhov/ChenWeiLi_-_MedPhi-3-mini_v1-gguf/blob/main/MedPhi-3-mini_v1.Q8_0.gguf) | Q8_0 | 3.78GB |




Original model description:
---
license: apache-2.0
---
### Evaluation

|             Tasks             |Version|Filter|n-shot| Metric |Value |   |Stderr|
|-------------------------------|-------|------|-----:|--------|-----:|---|-----:|
| - medmcqa                     |Yaml   |none  |     0|acc     |0.5408|±  |0.0077|
|                               |       |none  |     0|acc_norm|0.5408|±  |0.0077|
| - medqa_4options              |Yaml   |none  |     0|acc     |0.5711|±  |0.0139|
|                               |       |none  |     0|acc_norm|0.5711|±  |0.0139|
| - anatomy (mmlu)              |      0|none  |     0|acc     |0.6815|±  |0.0402|
| - clinical_knowledge (mmlu)   |      0|none  |     0|acc     |0.7434|±  |0.0269|
| - college_biology (mmlu)      |      0|none  |     0|acc     |0.8056|±  |0.0331|
| - college_medicine (mmlu)     |      0|none  |     0|acc     |0.6647|±  |0.0360|
| - medical_genetics (mmlu)     |      0|none  |     0|acc     |0.7300|±  |0.0446|
| - professional_medicine (mmlu)|      0|none  |     0|acc     |0.7353|±  |0.0268|
|stem                           |N/A    |none  |     0|acc_norm|0.5478|±  |0.0067|
|                               |       |none  |     0|acc     |0.5909|±  |0.0058|
| - pubmedqa                    |      1|none  |     0|acc     |0.7620|±  |0.0191|

|Groups|Version|Filter|n-shot| Metric |Value |   |Stderr|
|------|-------|------|-----:|--------|-----:|---|-----:|
|stem  |N/A    |none  |     0|acc_norm|0.5478|±  |0.0067|
|      |       |none  |     0|acc     |0.5909|±  |0.0058|

![Comparison Image](https://huggingface.co/ChenWeiLi/MedPhi-3-mini_v1/resolve/main/compare.png)


