Imagine a car navigating a quiet suburban neighborhood.
Suddenly, a ball bounces onto the road. A human driver,
leveraging extensive experiential knowledge, would not
only perceive the immediate presence of the ball, but also
instinctively anticipate the possibility of a chasing child
and consequently decide to decelerate. In contrast, an au-
tonomous vehicle, devoid of such reasoning and experiential
anticipation, might continue driving until sensors detect the
child, only allowing for a narrower margin of safety. The
importance of human prior knowledge in driving systems
becomes clear: driving is not merely about reacting to the

*indicates equal contribution.

Figure 1. Comparison between (a) the conventional driving
system and (b) the proposed Agent-Driver. Our approach trans-
forms the conventional perception-prediction-planning framework
by introducing LLMs as an agent for autonomous driving.
ALanguageAgentforAutonomousDriving