Some time ago, I built a predictive LLM router that routes chat requests between small and large LLM models based on prompt classification. It dynamically selects the most suitable model depending on the complexity of the user input, ensuring optimal performance while maintaining conversation context. I also fine-tuned a RoBERTa model to use with the package, but you can plug and play any classifier of your choice.
text
Some time ago, I built a predictive LLM router that routes chat requests between small and large LLM models based on prompt classification. It dynamically selects the most suitable model depending on the complexity of the user input, ensuring optimal performance while maintaining conversation context. I also fine-tuned a RoBERTa model to use with the package, but you can plug and play any classifier of your choice.
new_line
new_line
Project's homepage:
text
Project's homepage:
new_line
https://devquasar.com/llm-predictive-router/
https://devquasar.com/llm-predictive-router/
link
new_line
Pypi:
text
Pypi:
new_line
https://pypi.org/project/llm-predictive-router/
https://pypi.org/project/llm-predictive-router/
link
new_line
Model:
text
Model:
new_line
https://huggingface.co/DevQuasar/roberta-prompt_classifier-v0.1
DevQuasar/roberta-prompt_classifier-v0.1
model
resource
https://huggingface.co/DevQuasar/roberta-prompt_classifier-v0.1
new_line
Training data:
text
Training data:
new_line
https://huggingface.co/datasets/DevQuasar/llm_router_dataset-synth
DevQuasar/llm_router_dataset-synth
dataset
resource
https://huggingface.co/datasets/DevQuasar/llm_router_dataset-synth
new_line
Git:
text
Git:
new_line
https://github.com/csabakecskemeti/llm_predictive_router_package
https://github.com/csabakecskemeti/llm_predictive_router_package
link
new_line
new_line
Feel free to check it out, and/or contribute.
text
Feel free to check it out, and/or contribute.
Some time ago, I built a predictive LLM router that routes chat requests between small and large LLM models based on prompt classification. It dynamically selects the most suitable model depending on the complexity of the user input, ensuring optimal performance while maintaining conversation context. I also fine-tuned a RoBERTa model to use with the package, but you can plug and play any classifier of your choice.

Project's homepage:
https://devquasar.com/llm-predictive-router/
Pypi:
https://pypi.org/project/llm-predictive-router/
Model:
https://huggingface.co/DevQuasar/roberta-prompt_classifier-v0.1
Training data:
https://huggingface.co/datasets/DevQuasar/llm_router_dataset-synth
Git:
https://github.com/csabakecskemeti/llm_predictive_router_package

Feel free to check it out, and/or contribute.
👍
The-Rat-King lunarflu Csplk John6666 Sengil SugoiLoki tushifire