This discussion has been hidden

#517
by yttria - opened
This comment has been hidden

The SOURCE is simply whatever was used as source for the quantisations, and it is provided only when it was not obvious - or somehow very difficult - to obtain, and thus, every such file has a story of its own. For example, it, might have required a special converter, some code fix, could be copied from somebody else (such as a leaked source) and so on. It always means some manual intervention of sorts has happened.

Most of the time, the internal source is simply whatever convert_hf_to_gguf.py (or another converter) outputs, but since that is considered obvious, it is not usually provided in such cases.

If you have questions, feel free to ask them - I might or might not remember the provenance of specific SOURCE files.

mradermacher changed discussion status to closed
This comment has been hidden

The reason the Qwen2.5 series of models comes with SOURCE GGUFs and way more quants than usual is because we used those models for months of extensive quant quality measurements and performance benchmarking. For the Qwen2.5 series of models and most other models you can easily generate your own source GGUF from the base model using llama.cpp using python convert_hf_to_gguf.py --outfile Qwen2.5-72B-Instruct.SOURCE.gguf /root/Qwen2.5-72B-Instruct where "/root/Qwen2.5-72B-Instruct" is the location of the base model.

yttria changed discussion title from SOURCE GGUF question to This discussion has been hidden

Sign up or log in to comment