Script for fp8 scaling
#1
by
jetjodh
- opened
do you mean the conversion script? which is inside the tensor cutter
Yes, the one used for aura_flow_0.3_fp8_scaled.safetensors
ok, you have several ways to achieve this; you could use the tensor cutter inside the gguf-node; or you could install gguf-connector (pip install gguf-connector) then execute ggc q; or you could even get the q.py straight from connector (gguf node's parent) and run the script with safetensors in the same directory
Thanks, will try these out.
jetjodh
changed discussion status to
closed