Body: <p>The marvelous book <a href="https://www.newinchess.com/game-changer" rel="nofollow noreferrer">Game Changer: AlphaZero's Groundbreaking Chess Strategies and the Promise of AI</a> gave rise to this question. It is - in my opinion - a perfect example of rationalizing a piece of AI that makes it understandable and explainable. I favour the term &quot;rationalizable&quot; instead of &quot;<a href="https://de.wikipedia.org/wiki/Explainable_Artificial_Intelligence" rel="nofollow noreferrer">explainable</a>&quot; or &quot;<a href="https://www.techopedia.com/explainable-ai-isnt-enough-we-need-understandable-ai/2/34671" rel="nofollow noreferrer">understandable</a>&quot; because it can naturally be applied to humans and animals as well (see <a href="https://oxford.universitypressscholarship.com/view/10.1093/acprof:oso/9780198528272.001.0001/acprof-9780198528272" rel="nofollow noreferrer">rational animals</a>), and because it captures better what it is (or should be) about: guessing the <a href="https://en.wikipedia.org/wiki/Rationale" rel="nofollow noreferrer">rationale</a> behind some observed behaviour. A rationale lets an observed behaviour seem rational.</p>
  <p>Candidates for rationizable AI are systems that show some sufficiently complex behaviour (not just giving classifying answers) that also humans may show. For example playing behaviour which is typically described in terms of strategies.</p>
  <p>To give some examples how the authors of Game Changer rationalize AlphaZero (p. 98):</p>
  <ul>
  <li><p>AlphaZero likes to target the opponent's king.</p>
  </li>
  <li><p>AlphaZero likes to keep its own king out of danger.</p>
  </li>
  <li><p>AlphaZero makes sure the central situation is stable before it weakens its own kingside structure to open lines against the opponent's king.</p>
  </li>
  <li><p>AlphaZero defends by creating confusion and introducing tactics into the game.</p>
  </li>
  </ul>
  <p>The authors even reconstruct AlphaZero's &quot;thought processes&quot; (see p. 77: &quot;An in-depth illustration of one move&quot;)</p>
  <p>To make such rationalizations you need a lot of expert knowledge. And AlphaZero most surely is not capable of giving such rationalizations in human-understandable form by itself.</p>
  <p>My main question is for other <strong>specific examples of rationalizing (some piece of) AI</strong>.</p>
  <p>Side questions are:</p>
  <ul>
  <li><p>Do you find this kind of rationalization sound, and is it what one
  means when talking about explainable AI?</p>
  </li>
  <li><p>What has this concept of rationalizability to do with
  the game-theoretic concept of <a href="https://en.wikipedia.org/wiki/Rationalizability" rel="nofollow noreferrer">rationalizability</a>?</p>
  </li>
  </ul>
  <hr />
  <p><strong>Edit:</strong> Only today I found some references that explicitly mention &quot;rationalization&quot; or &quot;rationality&quot; in the context of AI:</p>
  <ul>
  <li><p>U. Ehsan et al (2017): <a href="https://Rationalization:%20A%20Neural%20Machine%20Translation%20Approach%20to%20Generating%20Natural%20Language%20Explanations" rel="nofollow noreferrer">Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations</a></p>
  </li>
  <li><p>T. Marwala et al (2021): <a href="https://www.sciencedirect.com/book/9780128206768/rational-machines-and-artificial-intelligence" rel="nofollow noreferrer">Rational Machines and Artificial Intelligence</a></p>
  </li>
  </ul>
  <p>This paper doesn't mention rationality explicitly but stresses on human-understandability. And it goes even one step further into the direction of self-explainability:</p>
  <ul>
  <li>D.C. Elton (2020): <a href="https://arxiv.org/pdf/2002.05149.pdf" rel="nofollow noreferrer">Self-explaining AI as an alternative to interpretable AI</a></li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-23T11:56:43.010Z
Comments.Text: You're asking multiple questions here. I recommend that you edit your post to leave just one question or, at least, specify what is your main question (maybe you the other questions are just subquestions, but that doesn't seem to be the case).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-23T12:28:27.827Z
Comments.Text: @nbro: Is it better now?
Comments.UserDisplayName: Hans-Peter Stricker
Tags: <reference-request><game-theory><alphazero><chess><explainable-ai>
Title: Examples of rationalizable AI
UserDisplayName: Hans-Peter Stricker
ThreadText: # Examples of rationalizable AI
  
  (Asked by: username_0 on 2022-02-23)
  
  The marvelous book [Game Changer: AlphaZero's Groundbreaking Chess Strategies and the Promise of AI](https://www.newinchess.com/game-changer) gave rise to this question. It is - in my opinion - a perfect example of rationalizing a piece of AI that makes it understandable and explainable. I favour the term "rationalizable" instead of "[explainable](https://de.wikipedia.org/wiki/Explainable_Artificial_Intelligence)" or "[understandable](https://www.techopedia.com/explainable-ai-isnt-enough-we-need-understandable-ai/2/34671)" because it can naturally be applied to humans and animals as well (see [rational animals](https://oxford.universitypressscholarship.com/view/10.1093/acprof:oso/9780198528272.001.0001/acprof-9780198528272)), and because it captures better what it is (or should be) about: guessing the [rationale](https://en.wikipedia.org/wiki/Rationale) behind some observed behaviour. A rationale lets an observed behaviour seem rational.
  
  Candidates for rationizable AI are systems that show some sufficiently complex behaviour (not just giving classifying answers) that also humans may show. For example playing behaviour which is typically described in terms of strategies.
  
  To give some examples how the authors of Game Changer rationalize AlphaZero (p. 98):
  
  * AlphaZero likes to target the opponent's king.
  * AlphaZero likes to keep its own king out of danger.
  * AlphaZero makes sure the central situation is stable before it weakens its own kingside structure to open lines against the opponent's king.
  * AlphaZero defends by creating confusion and introducing tactics into the game.
  
  The authors even reconstruct AlphaZero's "thought processes" (see p. 77: "An in-depth illustration of one move")
  
  To make such rationalizations you need a lot of expert knowledge. And AlphaZero most surely is not capable of giving such rationalizations in human-understandable form by itself.
  
  My main question is for other **specific examples of rationalizing (some piece of) AI**.
  
  Side questions are:
  
  * Do you find this kind of rationalization sound, and is it what one
    means when talking about explainable AI?
  * What has this concept of rationalizability to do with
    the game-theoretic concept of [rationalizability](https://en.wikipedia.org/wiki/Rationalizability)?
  
  ---
  
  **Edit:** Only today I found some references that explicitly mention "rationalization" or "rationality" in the context of AI:
  
  * U. Ehsan et al (2017): [Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://Rationalization:%20A%20Neural%20Machine%20Translation%20Approach%20to%20Generating%20Natural%20Language%20Explanations)
  * T. Marwala et al (2021): [Rational Machines and Artificial Intelligence](https://www.sciencedirect.com/book/9780128206768/rational-machines-and-artificial-intelligence)
  
  This paper doesn't mention rationality explicitly but stresses on human-understandability. And it goes even one step further into the direction of self-explainability:
  
  * D.C. Elton (2020): [Self-explaining AI as an alternative to interpretable AI](https://arxiv.org/pdf/2002.05149.pdf)
  
  **Tags:** reference-request, game-theory, alphazero, chess, explainable-ai
  
  > **Comments:**
  > **username_1**: You're asking multiple questions here. I recommend that you edit your post to leave just one question or, at least, specify what is your main question (maybe you the other questions are just subquestions, but that doesn't seem to be the case).  
  > **username_0**: @username_1: Is it better now?  
  
  ---
  
  ## 0 answers
  
  