nothing between 4.0 and 6.0?
#2
by
jackboot
- opened
4.0 is kind of too low.. 6.0 is too big. A 4.65 like 4km gguf would be just right.
we provided the measurement file for easy re-quanting: https://huggingface.co/anthracite-org/magnum-v2-72b-exl2/blob/main/measurement.json and a few people did already, maybe you can find one that fits on HF?
lucyknada
changed discussion status to
closed