Hugging Face's logo

gghfez
/
Mistral-Small-3.1-24B-Instruct-2503-novision-exl2-6bpw

Safetensors
mistral
6-bit
exl2
Model card Files Community

Text-Only EXL2 Quant of mistralai/Mistral-Small-3.1-24B-Instruct-2503

The following changes were made:

  1. Vision encoder removed
  2. Architecture changed to that of mistralai/Mistral-Small-24B-Instruct-2501
  3. Chat template in tokenizer_config.json was modified (see below).

I was having trouble with the timestamp at the beginning of the system prompt and removed it from tokenizer_config.json.

NOTE Tensor Parallel is not implemented in exllamav2 for both mistralai/Mistral-Small-3.1-24B-Instruct-2503 and mistralai/Mistral-Small-24B-Instruct-2501.

Downloads last month
2
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for gghfez/Mistral-Small-3.1-24B-Instruct-2503-novision-exl2-6bpw

Base model

mistralai/Mistral-Small-3.1-24B-Base-2503
Finetuned
mistralai/Mistral-Small-3.1-24B-Instruct-2503
Quantized
(52)
this model

Collection including gghfez/Mistral-Small-3.1-24B-Instruct-2503-novision-exl2-6bpw