LlamaCPP and Transformer implementation?
Hiya,
I wanted to inquire about plans for transformer and llama.cpp implementations?
I've seen that there are already two forks / branches that have implementations:
https://github.com/Noeda/llama.cpp/tree/dots1
https://github.com/redmoe-moutain/transformers/tree/dots.1
Are there any Timelines you have in mind as i haven't seen much movement recently, the current implementations as we use those two forks work fine with normal setups.
Do you have recommended sampler settings aswell?
Thank you very much!
@Luni Our current efforts are focused on supporting transformers, vllm, and sglang. We are actively working to merge these open PRs soon. @Noeda has contributed significantly to enabling llama.cpp support for dots1.
Additionally, for recommended sampler settings, please refer to the generation configuration available at this link.