databases [21, 40], where we encode the input query and
each record in the memory into embeddings and then retrieve
the top-K similar records via K-nearest neighbors (K-NN)
search in the embedding space. Since the driving scenarios
are quite diverse, the embedding-based search is inherently
limited by the encoding methods employed, resulting in
insufficient generalization capabilities. To overcome this
challenge, the second stage incorporates an LLM-based
fuzzy search, where the LLM is tasked to rank these records
according to their relevance to the query. This ranking is
based on the implicit similarity assessment by the LLM,
leveraging its capabilities in generalization and reasoning.
The cognitive memory equips our system with human
knowledge and past driving experiences. The retrieved
most similar experiences, together with commonsense and
environmental information, collectively form the inputs to
the reasoning engine. Text as a unified interface aligns the
ALanguageAgentforAutonomousDriving