Any plans for an update to 'mistralai/Mistral-Small-3.2-24B-Instruct-2506'?
#7
by
docgerbil
- opened
See title
Apologies, but each model we upload now requires an in-depth review process, so we recommend quantizing it yourself for now. If you encounter any issues, feel free to open an issue in the auto-round repo for assistance.
you could use auto_awq or auto_gptq in the format
auto-round-mllm \
--model mistralai/Mistral-Small-3.1-24B-Instruct-2503 \
--device 0 \
--bits 4 \
--format 'auto_round' \
--output_dir "./tmp_autoround"