Missing Tokenizer/Processor for use with Transformers
Traceback (most recent call last):
File "/home/mgoin/code/llm-compressor/examples/quantization_w4a4_fp4/magistral_example.py", line 13, in <module>
processor = AutoProcessor.from_pretrained(MODEL_ID)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mgoin/code/transformers/src/transformers/models/auto/processing_auto.py", line 408, in from_pretrained
raise ValueError(
ValueError: Unrecognized processing class in Magistral-Small-2506. Can't instantiate a processor, a tokenizer, an image processor or a feature extractor for this model. Make sure the repository contains the files of at least one of those processing classes.
Traceback (most recent call last): File "/home/mgoin/code/llm-compressor/examples/quantization_w4a4_fp4/magistral_example.py", line 13, in <module> processor = AutoProcessor.from_pretrained(MODEL_ID) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mgoin/code/transformers/src/transformers/models/auto/processing_auto.py", line 408, in from_pretrained raise ValueError( ValueError: Unrecognized processing class in Magistral-Small-2506. Can't instantiate a processor, a tokenizer, an image processor or a feature extractor for this model. Make sure the repository contains the files of at least one of those processing classes.
Hi Michael,
using the tokenizer from the base model worked fine for me to convert it to fp8:
https://huggingface.co/bullerwins/Magistral-Small-2506-fp8
It happened to me too on the Devstral release, I made a PR but didn't get merged
Note: I'm still testing the quality
Thanks for the heads up on what worked for you! I am working on experimental LLM Compressor support for the mistral-format, so you can find my results here https://huggingface.co/RedHatAI/Magistral-Small-2506-FP8
Hi, we don't plan on releasing a HF tokenizer and instead recommend using mistral-common.
What the hell is mistral-common?
It is a library that defines a set of tools to perform tokenization for our models and properly call tools. You can find the GitHub repo here: https://github.com/mistralai/mistral-common.
vLLM, which is our recommended framework to serve Magistral, supports mistral-common.