EXL3 quantization of Mistral-Nemo-Instruct-2407, 4 bits per weight.

HumanEval (argmax)

Model Q4 Q6 Q8 FP16
Mistral-Nemo-Instruct-2407-exl3-4bpw (mistral) 74.4 72.6 73.2 72.0
Mistral-Nemo-Instruct-2407-exl3-4bpw (chatml) 70.1 72.0 71.3 72.6
Mistral-Nemo-Instruct-2407-exl3-6bpw (mistral) 70.7 69.5 69.5 68.9
Mistral-Nemo-Instruct-2407-exl3-6bpw (chatml) 68.3 70.1 69.5 68.9
Downloads last month
5
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for isogen/Mistral-Nemo-Instruct-2407-exl3-4bpw