How to use?
Hi there! How do you exactly use embeds? I've been using your version of VNTL's model. So far yours has been usually better at the translations. I'm using LM studio but I do not know how to use these kinds of embeds. Do I just load it up alongside the model or something?
Embeds are used for RAG. I find it helps with translations when feeding it chat pairs with the instruct format included.
However to run it, I've only had success with llama.cpp when it's converted to gguf format. Problem is that I have no idea how to build a new sentencepiece.bpe.model from the tokenizer to be able to convert it to gguf format, which means you've to run it via transformers which I've no experience in.
I see thanks! I've been mostly just running the model with LunaTranslator so unfortunately I'm a complete beginner regarding this.