machine learning models remains unanswered. Thus, we will explore this research gap in the present article.

Journal of Information Science, 2022, pp. 1–11 (cid:2) The Author(s), DOI: 10.1177/01655515221112844

Rajabi and Etminani

3

Figure 1. The systematic review methodology.

3.1. Research questions

This review examines the existing methodologies for leveraging KGs in explainable AI systems, including machine
learning algorithms. To achieve this goal, we seek to answer the following research question: ‘Can KGs be used to
achieve explainability and interpretability in AI-based systems?’ This general question is divided into the following sub-
questions:

• What kinds of XAI and machine learning models have used KGs for explainability?
• Where in the AI process were KGs utilised in these XAI models?
•

How did these XAI models use KGs to achieve explainability?

3.2. Eligibility criteria

Inclusion criteria:

•
•
Knowledge-graph-based explainable AI- A systematic review