  More than one hundred benchmarks have been developed to test the commonsense
knowledge and commonsense reasoning abilities of artificial intelligence (AI)
systems. However, these benchmarks are often flawed and many aspects of common
sense remain untested. Consequently, we do not currently have any reliable way
of measuring to what extent existing AI systems have achieved these abilities.
This paper surveys the development and uses of AI commonsense benchmarks. We
discuss the nature of common sense; the role of common sense in AI; the goals
served by constructing commonsense benchmarks; and desirable features of
commonsense benchmarks. We analyze the common flaws in benchmarks, and we argue
that it is worthwhile to invest the work needed ensure that benchmark examples
are consistently high quality. We survey the various methods of constructing
commonsense benchmarks. We enumerate 139 commonsense benchmarks that have been
developed: 102 text-based, 18 image-based, 12 video based, and 7 simulated
physical environments. We discuss the gaps in the existing benchmarks and
aspects of commonsense reasoning that are not addressed in any existing
benchmark. We conclude with a number of recommendations for future development
of commonsense AI benchmarks.

Artificial Intelligence