Thanks for this and would love to see an int4 version

by CyberTimon - opened May 28

May 28

I really like this model but I really can't quantize this to int4, but it should be possible as this (https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-int4) is working.

Thank you very much!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

Thanks for this and would *love* to see an int4 version