Quantized version of: Tesslate/UIGEN-T3-32B-Preview
FYI: Standard quants might not perform well https://www.reddit.com/r/LocalLLaMA/comments/1l808xc/comment/mx0yea2/
We found that standard quantization significantly degrades quality and can break the model's reasoning chains. For the best results, we highly recommend running it in BF16 or FP8.
'Make knowledge free for everyone'
Made with
- Downloads last month
- 78
Hardware compatibility
Log In
to view the estimation
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support