Support for KoboldCpp?
Hi. Is this model supported by KoboldCpp? I tested the first version of the model online, and I was impressed with its answers when I compared to some other popular models. I'd like to use this on my computer through KoboldCpp. Would that be possible, please?
Hi. This model comes from chatglm.cpp and is not supported by KoboldCpp and I'm afraid the KoboldCpp community is not going to support this model in the near future.
However, an alternative is available: Xinference. You can start xinference with a single command and you are all set. A web UI will be available and models you need will be downloaded automatically.
Please star our project if you like it!
Hi. This model comes from chatglm.cpp and is not supported by KoboldCpp and I'm afraid the KoboldCpp community is not going to support this model in the near future.
However, an alternative is available: Xinference. You can start xinference with a single command and you are all set. A web UI will be available and models you need will be downloaded automatically.
Please star our project if you like it!
Thank you for a good tip, although I can't really use it, because one of the reasons I've using KoboldCpp is that it comes with a pre-built Windows executables which I can use without having to go through compiling process as I'm not much familiar with python. 😞 However, I left a star on the project, hoping it would get more attention from more talented people willing to collaborate and maybe one day someone will be so kind and create a simple to use package for Windows end-users. For now I'm adding it to my bookmarks and I'll be checking it periodically for news. 🤗
Also, I already asked in discussions at KoboldCpp project if they could add support for it, leaving links there to the project as well as the ggml model here, hopefully someone will pick it up, because it feels like this model opens a whole new world, sadly with only a few little holes in the walls as entrances I can't quite fit through, I need someone skilled to create the actual door for me. 😃
We are hosting a Hugging Face space at https://huggingface.co/spaces/Xorbits/xinference with the latest xinference!
Open the HF space in any browser and you can try all the supported edge-cutting models! Give it a try!