  Large language models (LLMs) have a wealth of knowledge that allows them to
excel in various Natural Language Processing (NLP) tasks. Current research
focuses on enhancing their performance within their existing knowledge. Despite
their vast knowledge, LLMs are still limited by the amount of information they
can accommodate and comprehend. Therefore, the ability to understand their own
limitations on the unknows, referred to as self-knowledge, is of paramount
importance. This study aims to evaluate LLMs' self-knowledge by assessing their
ability to identify unanswerable or unknowable questions. We introduce an
automated methodology to detect uncertainty in the responses of these models,
providing a novel measure of their self-knowledge. We further introduce a
unique dataset, SelfAware, consisting of unanswerable questions from five
diverse categories and their answerable counterparts. Our extensive analysis,
involving 20 LLMs including GPT-3, InstructGPT, and LLaMA, discovering an
intrinsic capacity for self-knowledge within these models. Moreover, we
demonstrate that in-context learning and instruction tuning can further enhance
this self-knowledge. Despite this promising insight, our findings also
highlight a considerable gap between the capabilities of these models and human
proficiency in recognizing the limits of their knowledge.

Computation and Language