HeurAgenix: Leveraging LLMs for Solving Complex Combinatorial
  Optimization Challenges
Heuristic algorithms play a vital role in solving combinatorial optimization
(CO) problems, yet traditional designs depend heavily on manual expertise and
struggle to generalize across diverse instances. We introduce
HeurAgenix, a two-stage hyper-heuristic framework powered by large
language models (LLMs) that first evolves heuristics and then selects among
them automatically. In the heuristic evolution phase, HeurAgenix leverages an
LLM to compare seed heuristic solutions with higher-quality solutions and
extract reusable evolution strategies. During problem solving, it dynamically
picks the most promising heuristic for each problem state, guided by the LLM's
perception ability. For flexibility, this selector can be either a
state-of-the-art LLM or a fine-tuned lightweight model with lower inference
cost. To mitigate the scarcity of reliable supervision caused by CO complexity,
we fine-tune the lightweight heuristic selector with a dual-reward mechanism
that jointly exploits singals from selection preferences and state perception,
enabling robust selection under noisy annotations. Extensive experiments on
canonical benchmarks show that HeurAgenix not only outperforms existing
LLM-based hyper-heuristics but also matches or exceeds specialized solvers.
Code is available at https://github.com/microsoft/HeurAgenix.
685d2b86696820ba1f28f3a8
Xianliang Yang
claimed_verified
2025-06-27T08:57:23.564Z
64d9a2439fef656cfd570232
/avatars/57e7282c21bb5eb716163ea4d679c158.svg
Xianliang Yang
user
VictorYXL
685d2b86696820ba1f28f3a9
Ling Zhang
685d2b86696820ba1f28f3aa
Haolong Qian
685d2b86696820ba1f28f3ab
Lei Song
685d2b86696820ba1f28f3ac
Jiang Bian
HeurAgenix: Leveraging LLMs for Solving Complex Combinatorial
  Optimization Challenges
Heuristic algorithms play a vital role in solving combinatorial optimization
(CO) problems, yet traditional designs depend heavily on manual expertise and
struggle to generalize across diverse instances. We introduce
HeurAgenix, a two-stage hyper-heuristic framework powered by large
language models (LLMs) that first evolves heuristics and then selects among
them automatically. In the heuristic evolution phase, HeurAgenix leverages an
LLM to compare seed heuristic solutions with higher-quality solutions and
extract reusable evolution strategies. During problem solving, it dynamically
picks the most promising heuristic for each problem state, guided by the LLM's
perception ability. For flexibility, this selector can be either a
state-of-the-art LLM or a fine-tuned lightweight model with lower inference
cost. To mitigate the scarcity of reliable supervision caused by CO complexity,
we fine-tune the lightweight heuristic selector with a dual-reward mechanism
that jointly exploits singals from selection preferences and state perception,
enabling robust selection under noisy annotations. Extensive experiments on
canonical benchmarks show that HeurAgenix not only outperforms existing
LLM-based hyper-heuristics but also matches or exceeds specialized solvers.
Code is available at https://github.com/microsoft/HeurAgenix.
HeurAgenix, a two-stage hyper-heuristic framework using large language models, evolves and selects heuristics dynamically for combinatorial optimization problems, achieving performance on par with specialized solvers.
hyper-heuristic framework large language models (LLMs) heuristic evolution selection preferences state perception dual-reward mechanism combinatorial optimization (CO)