QLoRA: Efficient Finetuning of Quantized LLMs
This is off-topic, but are there any communities or congregations (that aren't reddit) based around locally hosted LLMs? I'm asking because while I see a bunch of projects for exposing GGML/LLaMA to OpenAI compatible interfaces, some UIs, etc, I can't really find a good community or resources for the concept in general.<p>I'm working on a front-end for LLMs in general, having re-implemented a working version of OpenAI's code interpreter "plugin" already within the UI (and yes, I support file uploads), and support for the wealth of third-party OpenAI plugins that don't require auth (I've been testing with the first diagram plugin I found, it works well.) I'm planning to open source it once my breaking changes slow down.<p>This field moves very fast, I'm looking for feedback (and essentially testers/testing data) on what people want, and looking for prompts/chat logs/guidance templates (<a href="https://github.com/microsoft/guidance">https://github.com/microsoft/guidance</a>) for tasks they expect to "just work" with natural language.<p>Instead of being limited by the monetization for ChatGPT Plus (and limited number of messages every four hours) for extensibility within a chat interface, I want to open it and free it, with a Bring-Your-Own-(optionally local)-LLM/API key setup.