Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
eaddario 
posted an update Mar 4
Post
2073
Squeezing out tensor bits, part II

At post time, watt-ai/watt-tool-70B continues to top the Berkeley Function-Calling Leaderboard, with the 8B version occupying the 4th place. A remarkable achievement for a model of that size!

The "squeezed" version is now available at eaddario/Watt-Tool-8B-GGUF

(For context please see: https://huggingface.co/posts/eaddario/832567461491467)
UICO

Well done! Your technique is very impressiove! BTW，Could you provide quantization for QWQ-32B?

·
eaddario

Thank you @UICO , but at the moment rather than a technique, it's more of a mix of brutish-force, educated guesses, trial and error and the occasional luck, but will tackle QwQ 32B next as it will help me validate an idea (see my next post)

In this post