exllamav3 quantizations of Mistral-Large-Instruct-2407
Sizes inspired by turboderp's exl3 quant of Mistral-Large-Instruct-2411 with one notable exception.
1.40 bpw/H4 20.820 GiB
1.60 bpw/H4 23.670 GiB
1.80 bpw/H5 26.541 GiB
2.00 bpw/H5 29.389 GiB
2.25 bpw/H5 32.927 GiB
2.50 bpw/H5 36.470 GiB
3.00 bpw/H6 43.616 GiB
3.50 bpw/H6 50.697 GiB
4.00 bpw/H6 57.795 GiB
5.00 bpw/H6 71.975 GiB
6.00 bpw/H6 86.155 GiB
8.00 bpw/H8 114.609 GiB
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
HF Inference deployability: The model authors have turned it off explicitly.
Model tree for MikeRoz/Mistral-Large-Instruct-2407-exl3
Base model
mistralai/Mistral-Large-Instruct-2407