  The Knowledge Base Question Answering (KBQA) task aims to answer natural
language questions based on a given knowledge base. As a kind of common method
for this task, semantic parsing-based ones first convert natural language
questions to logical forms (e.g., SPARQL queries) and then execute them on
knowledge bases to get answers. Recently, Large Language Models (LLMs) have
shown strong abilities in language understanding and may be adopted as semantic
parsers in such kinds of methods. However, in doing so, a great challenge for
LLMs is to understand the schema of knowledge bases. Therefore, in this paper,
we propose an In-Context Schema Understanding (ICSU) method for facilitating
LLMs to be used as a semantic parser in KBQA. Specifically, ICSU adopts the
In-context Learning mechanism to instruct LLMs to generate SPARQL queries with
examples. In order to retrieve appropriate examples from annotated
question-query pairs, which contain comprehensive schema information related to
questions, ICSU explores four different retrieval strategies. Experimental
results on the largest KBQA benchmark, KQA Pro, show that ICSU with all these
strategies outperforms that with a random retrieval strategy significantly
(from 12\% to 78.76\% in accuracy).

Computation and Language