Quantization recipe?
#3
by
veden
- opened
Do you have any plans to add the recipe to the README or repo?
Much appreciated for generating these quantized models.
Currently it is an experimental flow in LLM Compressor, but you can follow the README here https://github.com/vllm-project/llm-compressor/tree/main/experimental/mistral
thank you
veden
changed discussion status to
closed