Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner
  Framework for Enhancing Autonomous Vehicle Interactions
Autonomous Vehicles (AVs) have entered the commercialization stage, but their
limited ability to interact and express intentions still poses challenges in
interactions with Human-driven Vehicles (HVs). Recent advances in large
language models (LLMs) enable bidirectional human-machine communication, but
the conflict between slow inference speed and the need for real-time
decision-making challenges practical deployment. To address these issues, this
paper introduces a parallel Actor-Reasoner framework designed to enable
explicit bidirectional AV-HV interactions across multiple scenarios. First, by
facilitating interactions between the LLM-driven Reasoner and heterogeneous
simulated HVs during training, an interaction memory database, referred to as
the Actor, is established. Then, by introducing the memory partition module and
the two-layer memory retrieval module, the Actor's ability to handle
heterogeneous HVs is significantly enhanced. Ablation studies and comparisons
with other decision-making methods demonstrate that the proposed Actor-Reasoner
framework significantly improves safety and efficiency. Finally, with the
combination of the external Human-Machine Interface (eHMI) information derived
from Reasoner's reasoning and the feasible action solutions retrieved from the
Actor, the effectiveness of the proposed Actor-Reasoner is confirmed in
multi-scenario field interactions. Our code is available at
https://github.com/FanGShiYuu/Actor-Reasoner.
67c8427047c2aa135346dced
Shiyu Fang
claimed_verified
2025-03-05T15:46:48.691Z
66d3290364c1e9b73208af82
/avatars/e0ea5f2b366927c7b146f248028a2e59.svg
Shiyu Fang
user
FanGShiYuu
67c8427047c2aa135346dcee
Jiaqi Liu
claimed_verified
2025-07-07T11:26:54.499Z
684ff37fa383bc5d6b0ff77f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0JPr-cd_rxQz3k6rmzBOF.png
JiaqiLiu
user
JiaaqiLiu
67c8427047c2aa135346dcef
Chengkai Xu
admin_assigned
2025-03-06T13:57:59.218Z
6319e98c02fb3220373762f6
/avatars/1456665d2ae6a284e2505744fec0cb1d.svg
Chengkai Xu
user
Shinghoi
67c8427047c2aa135346dcf0
Chen Lv
admin_assigned
2025-03-06T13:57:52.421Z
670bcbbcdb7d982eb6a7e027
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/2aSOLBxM_MgGtK0EFJAKL.png
Chen Lv
user
zafkiellc
67c8427047c2aa135346dcf1
Peng Hang
67c8427047c2aa135346dcf2
Jian Sun
admin_assigned
2025-03-06T14:06:29.074Z
6522245b54967a3a49b3b22b
/avatars/18d190808f91eaff4b81cec1e9690337.svg
Jian SUN
user
jiansun-china
Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner
  Framework for Enhancing Autonomous Vehicle Interactions
Autonomous Vehicles (AVs) have entered the commercialization stage, but their
limited ability to interact and express intentions still poses challenges in
interactions with Human-driven Vehicles (HVs). Recent advances in large
language models (LLMs) enable bidirectional human-machine communication, but
the conflict between slow inference speed and the need for real-time
decision-making challenges practical deployment. To address these issues, this
paper introduces a parallel Actor-Reasoner framework designed to enable
explicit bidirectional AV-HV interactions across multiple scenarios. First, by
facilitating interactions between the LLM-driven Reasoner and heterogeneous
simulated HVs during training, an interaction memory database, referred to as
the Actor, is established. Then, by introducing the memory partition module and
the two-layer memory retrieval module, the Actor's ability to handle
heterogeneous HVs is significantly enhanced. Ablation studies and comparisons
with other decision-making methods demonstrate that the proposed Actor-Reasoner
framework significantly improves safety and efficiency. Finally, with the
combination of the external Human-Machine Interface (eHMI) information derived
from Reasoner's reasoning and the feasible action solutions retrieved from the
Actor, the effectiveness of the proposed Actor-Reasoner is confirmed in
multi-scenario field interactions. Our code is available at
https://github.com/FanGShiYuu/Actor-Reasoner.
The parallel Actor-Reasoner framework improves AV-HV interactions through enhanced memory modules and bidirectional human-machine communication, enhancing safety and efficiency in real-time decision-making.
Actor-Reasoner framework interaction memory database memory partition module two-layer memory retrieval module heterogenous HVs large language models bidirectional communication real-time decision-making Human-Machine Interface (eHMI)