GraphRouter: A Graph-based Router for LLM Selections
Tao Feng Yanzhen Shen Jiaxuan You
The rapidly growing number and variety of Large Language Models (LLMs)
present significant challenges in efficiently selecting the appropriate LLM for
a given query, especially considering the trade-offs between performance and
computational cost. Current LLM selection methods often struggle to generalize
across new LLMs and different tasks because of their limited ability to leverage
contextual interactions among tasks, queries, and LLMs, as well as their depen-
dence on a transductive learning framework. To address these shortcomings, we
introduce a novel inductive graph framework, named as GraphRouter, which
fully utilizes the contextual information among tasks, queries, and LLMs to en-
hance the LLM selection process. GraphRouter constructs a heterogeneous
graph comprising task, query, and LLM nodes, with interactions represented as
edges, which efficiently captures the contextual information between the query’s
requirements and the LLM’s capabilities. Through an innovative edge prediction
mechanism, GraphRouter is able to predict attributes (the effect and cost of
LLM response) of potential edges, allowing for optimized recommendations that
adapt to both existing and newly introduced LLMs without requiring retraining.
Comprehensive experiments across three distinct effect-cost weight scenarios have
shown that GraphRouter substantially surpasses existing routers, delivering a
minimum performance improvement of 12.3%. In addition, it achieves enhanced
generalization across new LLMs settings and supports diverse tasks with at least a
9.5% boost in effect and a significant reduction in computational demands. This
work endeavors to apply a graph-based approach for the contextual and adaptive
selection of LLMs, offering insights for real-world applications.
LLM selection, Graph-based router, Contextual interactions, New LLM settings
We propose GraphRouter, a novel graph-based router that improves LLM selection for diverse tasks by leveraging an inductive graph framework to optimize performance-cost trade-offs and generalize across various models and scenarios.