Invoke Interfaces Only When Needed: Adaptive Invocation for Large
  Language Models in Question Answering
The collaborative paradigm of large and small language models (LMs)
effectively balances performance and cost, yet its pivotal challenge lies in
precisely pinpointing the moment of invocation when hallucinations arise in
small LMs. Previous optimization efforts primarily focused on post-processing
techniques, which were separate from the reasoning process of LMs, resulting in
high computational costs and limited effectiveness. In this paper, we propose a
practical invocation evaluation metric called AttenHScore, which calculates the
accumulation and propagation of hallucinations during the generation process of
small LMs, continuously amplifying potential reasoning errors. By dynamically
adjusting the detection threshold, we achieve more accurate real-time
invocation of large LMs. Additionally, considering the limited reasoning
capacity of small LMs, we leverage uncertainty-aware knowledge reorganization
to assist them better capture critical information from different text chunks.
Extensive experiments reveal that our AttenHScore outperforms most baseline in
enhancing real-time hallucination detection capabilities across multiple QA
datasets, especially when addressing complex queries. Moreover, our strategies
eliminate the need for additional model training and display flexibility in
adapting to various transformer-based LMs.
681a06459c99f4b5ce5f2838
Jihao Zhao
claimed_verified
2025-05-06T16:51:29.234Z
658e85bb5b7553ca5c29ba89
https://cdn-avatars.huggingface.co/v1/production/uploads/658e85bb5b7553ca5c29ba89/KK6UpS9agtrxevvBoup5N.jpeg
Jihao Zhao
user
Robot2050
681a06459c99f4b5ce5f2839
Chunlai Zhou
681a06459c99f4b5ce5f283a
Biao Qin
Invoke Interfaces Only When Needed: Adaptive Invocation for Large
  Language Models in Question Answering
The collaborative paradigm of large and small language models (LMs)
effectively balances performance and cost, yet its pivotal challenge lies in
precisely pinpointing the moment of invocation when hallucinations arise in
small LMs. Previous optimization efforts primarily focused on post-processing
techniques, which were separate from the reasoning process of LMs, resulting in
high computational costs and limited effectiveness. In this paper, we propose a
practical invocation evaluation metric called AttenHScore, which calculates the
accumulation and propagation of hallucinations during the generation process of
small LMs, continuously amplifying potential reasoning errors. By dynamically
adjusting the detection threshold, we achieve more accurate real-time
invocation of large LMs. Additionally, considering the limited reasoning
capacity of small LMs, we leverage uncertainty-aware knowledge reorganization
to assist them better capture critical information from different text chunks.
Extensive experiments reveal that our AttenHScore outperforms most baseline in
enhancing real-time hallucination detection capabilities across multiple QA
datasets, especially when addressing complex queries. Moreover, our strategies
eliminate the need for additional model training and display flexibility in
adapting to various transformer-based LMs.
A new metric called AttenHScore is proposed to dynamically detect hallucinations in small language models, enhancing real-time performance and reducing the need for additional training.
AttenHScore hallucinations large language models small language models reasoning process knowledge reorganization transformer-based LMs QA datasets