Script for fp8 scaling

#1
by jetjodh - opened

Hi @calcuis ,do you have a script for fp8 scaled weight calculation?

Owner

do you mean the conversion script? which is inside the tensor cutter

Yes, the one used for aura_flow_0.3_fp8_scaled.safetensors

Owner

ok, you have several ways to achieve this; you could use the tensor cutter inside the gguf-node; or you could install gguf-connector (pip install gguf-connector) then execute ggc q; or you could even get the q.py straight from connector (gguf node's parent) and run the script with safetensors in the same directory

Thanks, will try these out.

jetjodh changed discussion status to closed

Sign up or log in to comment