Update README.md
Browse files
README.md
CHANGED
@@ -58,7 +58,11 @@ To run GGUFs correctly, you need the most recent version of KoboldCPP, and to pa
|
|
58 |
|
59 |
Thanks to DaringDuck and tofumagnate for info how to apply this fix.
|
60 |
|
61 |
-
To run this model on vLLM, you'll need to build it from source from the git repo, GLM4 support
|
|
|
|
|
|
|
|
|
62 |
|
63 |
---
|
64 |
|
|
|
58 |
|
59 |
Thanks to DaringDuck and tofumagnate for info how to apply this fix.
|
60 |
|
61 |
+
To run this model on vLLM, you'll need to build it from source from the git repo, full GLM4 support hasn't reached release yet.
|
62 |
+
|
63 |
+
ExLLaMAv2 and v3 based backends, such as TabbyAPI should support the model out of the box.
|
64 |
+
|
65 |
+
Latest versions of llama.cpp server should also allow running GGUFs out-of-the-box.
|
66 |
|
67 |
---
|
68 |
|