Conventional perception-prediction-planning pipelines
leverage a series of neural networks as basic modules for
different tasks. However, these neural-network-based sys-
tems lack direct compatibility with human prior knowledge,
constraining their potential for leveraging such priors to
enhance driving performance. To handle this challenge,
we propose a novel framework that leverages text represen-
tations as a unified interface to connect neural networks
and human knowledge. The overall architecture of Agent-
Driver is shown in Figure 2. Our approach takes sensory
data as input and introduces neural modules for processing
these sensory data and extracting environmental information
about detection, prediction, occupancy, and map. On top
of the neural modules, we propose a tool library where
a set of functions are designed to further abstract the
neural outputs and return text-based messages. For each
driving scenario, an LLM selectively activates the required
ALanguageAgentforAutonomousDriving