answers to queries ( ‘questions’) via reasoning and inference [8,9]. The combined use of KGs and machine learning mod-
els can make AI systems more transparent and interpretable, as machine learning models are capable of extracting rela-
tions, features, and entities, as well as inferring new concepts. KGs can be used to answer questions, understand images,
and retrieve information, which are all relevant aspects of many types of research. Recently, the integration of AI sys-
tems into coherent and comprehensive KGs has emerged as an open challenge [10]. In this article, we conduct a systema-
tic review of how and where KGs are being used in existing XAI systems. To this end, we review scholarly publications
that examine several dimensions of KG-based XAI and categorise them based on how/where they leverage KGs in XAI.
Researchers and experts can also use this review to identify and highlight the areas where KGs have been used most and
Knowledge-graph-based explainable AI- A systematic review