Arch-Router: Aligning LLM Routing with Human Preferences
With the rapid proliferation of large language models (LLMs) -- each
optimized for different strengths, style, or latency/cost profile -- routing
has become an essential technique to operationalize the use of different
models. However, existing LLM routing approaches are limited in two key ways:
they evaluate performance using benchmarks that often fail to capture human
preferences driven by subjective evaluation criteria, and they typically select
from a limited pool of models. In this work, we propose a preference-aligned
routing framework that guides model selection by matching queries to
user-defined domains (e.g., travel) or action types (e.g., image editing) --
offering a practical mechanism to encode preferences in routing decisions.
Specifically, we introduce Arch-Router, a compact 1.5B model that
learns to map queries to domain-action preferences for model routing decisions.
Our approach also supports seamlessly adding new models for routing without
requiring retraining or architectural modifications. Experiments on
conversational datasets demonstrate that our approach achieves state-of-the-art
(SOTA) results in matching queries with human preferences, outperforming top
proprietary models. Our approach captures subjective evaluation criteria and
makes routing decisions more transparent and flexible. Our model is available
at: https://huggingface.co/katanemo/Arch-Router-1.5B.
6858de6bc0c8e29df8ea3d03
Co Tran
6858de6bc0c8e29df8ea3d04
Salman Paracha
claimed_verified
2025-06-23T08:14:15.659Z
66b681906c8d3b36786b764c
/avatars/c04e97278b2275e2b34182229efa1c20.svg
Salman
user
parachas
6858de6bc0c8e29df8ea3d05
Adil Hafeez
6858de6bc0c8e29df8ea3d06
Shuguang Chen
claimed_verified
2025-06-24T08:09:48.654Z
622e9e56165ba2c1bcbc76da
https://cdn-avatars.huggingface.co/v1/production/uploads/1648869498279-622e9e56165ba2c1bcbc76da.jpeg
Shuguang Chen
user
nehcgs
Arch-Router: Aligning LLM Routing with Human Preferences
With the rapid proliferation of large language models (LLMs) -- each
optimized for different strengths, style, or latency/cost profile -- routing
has become an essential technique to operationalize the use of different
models. However, existing LLM routing approaches are limited in two key ways:
they evaluate performance using benchmarks that often fail to capture human
preferences driven by subjective evaluation criteria, and they typically select
from a limited pool of models. In this work, we propose a preference-aligned
routing framework that guides model selection by matching queries to
user-defined domains (e.g., travel) or action types (e.g., image editing) --
offering a practical mechanism to encode preferences in routing decisions.
Specifically, we introduce Arch-Router, a compact 1.5B model that
learns to map queries to domain-action preferences for model routing decisions.
Our approach also supports seamlessly adding new models for routing without
requiring retraining or architectural modifications. Experiments on
conversational datasets demonstrate that our approach achieves state-of-the-art
(SOTA) results in matching queries with human preferences, outperforming top
proprietary models. Our approach captures subjective evaluation criteria and
makes routing decisions more transparent and flexible. Our model is available
at: https://huggingface.co/katanemo/Arch-Router-1.5B.
A preference-aligned routing framework using a compact 1.5B model effectively matches queries to user-defined domains and action types, outperforming proprietary models in subjective evaluation criteria.
large language models LLM routing Arch-Router domain-action preferences