Answers.Body: <p>As a very rough answer, we need to be able to confirm a model that we came up with. So that we need to convince some one that our model is working logically well. What I meant by &quot;logically well&quot; is that the decision-making process should be clear either for a single data or the general model.
  I would suggest taking a look at this page:
  <a href="https://towardsdatascience.com/decrypting-your-machine-learning-model-using-lime-5adc035109b5" rel="nofollow noreferrer">https://towardsdatascience.com/decrypting-your-machine-learning-model-using-lime-5adc035109b5</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-30T12:43:14.240Z
Answers.Text: Thanks for your answer. I was aware of this 'ambiguous' goal. My question is looking more for definitions (i.e. from literature) that define it, for example, in a multi-faceted way.
Answers.UserDisplayName: Robin van Hoorn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-30T16:12:36.760Z
Answers.Text: I would clarify one aspect in this answer: The difference between interpretable models (what you describe) and explainable models. A model is interpretable if its inner workings (intermediate results) are human understandable. A model is explainable if you can at least apply a post-hoc method that extends the models output/input with an explanation. While all interpretable models are also explainable, not all explainable models are interpretable, so explainability is a weaker property than interpretability.
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-29T15:19:40.827Z
Answers.Text: Im not sure if I would agree with that @Chillston. I always had the idea that interpretable is a subset of explainable; where 'interpretable' refers to whether you can make sense of the output of a model, and 'explainable' refers to whether you can also make sense of how the model gets to that output.
Answers.UserDisplayName: Robin van Hoorn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-10T15:41:27.360Z
Answers.Text: Yeah, I am sure there is a lot of room for debate, oftentimes interpretability and explainability are regarded as the same thing which it is certainly not. I agree with you that interpretability is a subset of explainability and thus a stronger statement about a model. Where I have a different opinion is that to 'make sense' of an output, you have to make sense of how the model gets to that output first. Without that, how would you make sense of anything, given only an output without simply 'trusting' the model? Therefore, your description of 'interpretability' wouldn't work for me like that.
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-27T16:53:15.033Z
Answers.LastActivityDate: 2023-10-27T16:53:15.033Z
Answers.UserDisplayName: M SA
Body: <p>I understand the need for <a href="https://ai.stackexchange.com/questions/14224/why-do-we-need-explainable-ai">Explainability in AI</a>. However, I am uncertain of what is meant by 'making AI explainable'.</p>
  <p>What needs to be explainable? Is it the output of a model? Does it refer to the model itself? Does it refer to the user interface of the tool that the AI is a part of? Is it all of the above? If so, what is not included in Explainable AI?</p>
  <p>What do we strive for when making AI 'explainable'? Are there commonly referred to definitions of explainability of AI, which go beyond <a href="https://positivethinking.tech/insights/navigating-the-eu-ai-act-how-explainable-ai-simplifies-regulatory-compliance/" rel="nofollow noreferrer">'to understand how a decision was made'</a>?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-24T16:45:11.800Z
Comments.Text: depends on the case, but usually is "why is the model acting like that"... so for example, you would like to know that your resnet is recognizing a cat because the picture has some whiskers and ears, more than some other features
Comments.UserDisplayName: Alberto
Tags: <definitions><explainable-ai>
Title: What is Explainable AI and what does it strive for?
UserDisplayName: Robin van Hoorn
ThreadText: # What is Explainable AI and what does it strive for?
  
  (Asked by: username_0 on 2023-10-24)
  
  I understand the need for [Explainability in AI](https://ai.stackexchange.com/questions/14224/why-do-we-need-explainable-ai). However, I am uncertain of what is meant by 'making AI explainable'.
  
  What needs to be explainable? Is it the output of a model? Does it refer to the model itself? Does it refer to the user interface of the tool that the AI is a part of? Is it all of the above? If so, what is not included in Explainable AI?
  
  What do we strive for when making AI 'explainable'? Are there commonly referred to definitions of explainability of AI, which go beyond ['to understand how a decision was made'](https://positivethinking.tech/insights/navigating-the-eu-ai-act-how-explainable-ai-simplifies-regulatory-compliance/)?
  
  **Tags:** definitions, explainable-ai
  
  > **Comments:**
  > **username_1**: depends on the case, but usually is "why is the model acting like that"... so for example, you would like to know that your resnet is recognizing a cat because the picture has some whiskers and ears, more than some other features  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-10-27. Score: 1
  
  As a very rough answer, we need to be able to confirm a model that we came up with. So that we need to convince some one that our model is working logically well. What I meant by "logically well" is that the decision-making process should be clear either for a single data or the general model.
  I would suggest taking a look at this page:
  <https://towardsdatascience.com/decrypting-your-machine-learning-model-using-lime-5adc035109b5>
  
  > **Comments:**
  > **username_0**: Thanks for your answer. I was aware of this 'ambiguous' goal. My question is looking more for definitions (i.e. from literature) that define it, for example, in a multi-faceted way.  
  > **username_3**: I would clarify one aspect in this answer: The difference between interpretable models (what you describe) and explainable models. A model is interpretable if its inner workings (intermediate results) are human understandable. A model is explainable if you can at least apply a post-hoc method that extends the models output/input with an explanation. While all interpretable models are also explainable, not all explainable models are interpretable, so explainability is a weaker property than interpretability.  
  > **username_0**: Im not sure if I would agree with that @username_3. I always had the idea that interpretable is a subset of explainable; where 'interpretable' refers to whether you can make sense of the output of a model, and 'explainable' refers to whether you can also make sense of how the model gets to that output.  
  > **username_3**: Yeah, I am sure there is a lot of room for debate, oftentimes interpretability and explainability are regarded as the same thing which it is certainly not. I agree with you that interpretability is a subset of explainability and thus a stronger statement about a model. Where I have a different opinion is that to 'make sense' of an output, you have to make sense of how the model gets to that output first. Without that, how would you make sense of anything, given only an output without simply 'trusting' the model? Therefore, your description of 'interpretability' wouldn't work for me like that.  
  
  ---
  
  