Answers.Body: <p>There are three cases in which it is easily possible to distinguish strong AI play from the strong human play:</p>
  <ol>
  <li>The AI is playing at superhuman skill level</li>
  </ol>
  <p>This seems obvious, but I want to mention it for the sake of completeness. The current skill ceiling of top-level chess is well known and an opponent playing way above this skill ceiling must either be an AI or a chess guru who hid in a cave for the last centuries. Applying Occam's razor I would go with the AI. So to mimic human play the AI must make sure to stay at a human ELO level.</p>
  <ol start="2">
  <li>The AI plays an obvious weak move in a losing or strange situation</li>
  </ol>
  <p>When the AI is in a bad situation and still tries to win the game without any reasonable move available, it might be tempted to play an obviously bad move, because the decision engine assigns it the highest probability of success. Such moves are considered rude at high-level play and a human player would not play them but resign. The most famous example of this is move 101 in the fourth game between <a href="http://gokifu.com/s/2iq8-gokifu-20160313-Alphago(9p)-Lee_Sedol(9p).html" rel="nofollow noreferrer">Lee Sedol and AlphaGo</a>. AlphaGo just lost a big group but still played T9 to extend it even further. This is a move a rookie player stops making after his first 10 games because it is that obvious that it does not work. A professional go player would never do this under normal circumstances. AlphaGo was out of options and played the rude move instead of resigning. This doesn't mean an AI cannot be programmed to avoid such behavior, but dire or simply very strange situations might induce moves like this, giving the nature of the AI away.</p>
  <ol start="3">
  <li>Other factors than the actual moves are considered</li>
  </ol>
  <p>Depending on the test scenario, the AI might give away its nature through other means than the actual moves. Things like moving too fast or playing without breaks for hours. But I assume the question was just focused on the actual play and not the surrounding factors.</p>
  <p>Those are the obvious examples where an AI might give away his true nature. I highly doubt that a strong AI, playing at the strong human level, can really be identified because of his &quot;inhuman&quot; style. I think it is more of a psychological pitfall to consider innovative moves from an AI &quot;inhuman&quot; and the same move from a pro player as &quot;brilliant&quot; if you know who is who. I am an amateur 3dan Go player - far from the top but with several years of experience under the belt - and at least I wasn't able to see anything &quot;inhuman&quot; in AlphaGos play so far. Ingo, it is well known that there are different styles of play, and after reviewing enough pro games one gets a sense of which player might have been educated in Korea, China, or Japan because certain patterns of strategies are more or less common in those countries. The lack of such an individual nuance might be an indication, that the player is actually a strong AI, but I am confident, that even the strongest pro wouldn't have a high success rate guessing who is human and who is the AI.</p>
  <p>In chess, there are significantly fewer options at every move, many fixed patterns especially in the opening, and therefore less room for an individual style. Therefore I consider the same challenge even more difficult than in go. So if an AI has reason to &quot;play like a human&quot;, this should be possible with the current level of AI technology.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-08-23T22:28:21.283Z
Answers.LastActivityDate: 2021-04-19T22:40:45.630Z
Answers.LastEditDate: 2021-04-19T22:40:45.630Z
Answers.UserDisplayName: Demento
Answers.Body: <p>Yes - in chess the term "computer move" is used to denote a move found by a chess engine that a human player would never find (often because they make some slight improvement that a human would not be able to calculate). Humans use pattern recognition and some calculation in order to understand the chess position they are in while computers are able to calculate (~30 - 120+) moves ahead, a feat that a human player cannot match. It is for this reason that one can distinguish a human move from a computer move at some times.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-24T12:09:11.813Z
Answers.Text: Are you sure the are able to calculate ~30 to 120+ moves ahead? That seems very much to me.
Answers.UserDisplayName: HelloGoodbye
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-24T18:22:42.533Z
Answers.Text: But couldn't that "bewildering to humans" move be perceived as a random choice, potentially played to disrupt?  And if it leads to a win, couldn't that be perceived as the disruption tactic having worked?  (I guess it partly depends on the expertise of the human competitor, as the Imitation Game is subjective...)  PS Welcome to AI!
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-24T03:51:12.160Z
Answers.LastActivityDate: 2017-08-24T03:51:12.160Z
Answers.UserDisplayName: styg
Answers.Body: <p>I recall a friend saying that yes, it is somewhat obvious if you are playing against an AI.</p>
  
  <p>From what he said, against normal players, there is a certain rhythm and structure that "makes sense". But AI play doesn't have this quality, "it doesn't make sense, but it just works". This seems to echo what you mentioned about the aesthetic quality of Go</p>
  
  <p>Apparently because of this, chess/go AI can be detected during online play.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-23T20:35:58.230Z
Answers.LastActivityDate: 2017-08-23T20:35:58.230Z
Answers.UserDisplayName: k.c. sayz 'k.c sayz'
Answers.Body: <p>AI programs that exist in today's world fall into the category of Narrow Intelligence. Narrow Intelligence
  are easy to distinguish when compared to General Intelligence (ones that resemble more like humans).<br>
  Highly advanced AI can often resemble to act like humans thought. I will like to talk about <strong>Deep Blue</strong> here.<br><br>
  Garry Kasparov, in a <a href="http://time.com/3705316/deep-blue-kasparov/" rel="nofollow noreferrer">series of matches</a> during Feb 1996, even though
  won three matches but, had a <strong>" shattering experience"</strong>. Deep Blue
  flummoxed him in that first game by making a move with no immediate
  material advantage; nudging a pawn into a position where it could be
  easily captured. He said, "<em>It was a wonderful and <strong>extremely human</strong>
  move</em>" and this threw him for a loop. He further added that he had
  played a lot of computers but had never experienced anything like
  this. He added, "<strong>I could feel, I could smell, a new kind of intelligence
  across the table.</strong>"<br>
  Next year he lost to an improved version of Deep Blue. Here also he
  lost mostly because of psychological reason. He later said that he
  was riled by a move the computer made that was so surprising and
  so <strong>un-machine-like</strong> that he was sure that IBM team had cheated.
  It was later found out that it was a glitch in Deep Blue. The program
  came across too many options and had no clear preference and thus
  played a random move.<br>
  It was this random move that shifted the game in favour of Deep Blue.
  <br><br>
  One advantage that Deep Blue (or other programs) have is the ability to perform far <strong>more computations</strong> in a given time. Humans, on the other hand, can extend their decision tree only to <em>very limited depth</em>. On having no clear decision to take, we take a random decision and try to learn from the experience, unlike most Narrow Intelligence programs present today. We need a <strong>fair amount of
  randomness and learning from experience</strong> to be added to the program in order
  to make it more like a human.</p>
  
  <p>The answer to this question also depends on your position on <strong>"<a href="https://en.wikipedia.org/wiki/Chinese_room" rel="nofollow noreferrer">Chinese Room Argument</a>"</strong>.
  The argument holds that a program can't give a computer understanding
  or consciousness regardless of how intelligently or human like
  the program may make the computer behave. Chinese Room Argument focuses on something very important when it comes to <strong>"Consciousness v/s Simulation of Consciousness"</strong>. John Searle argued there that it is possible for a machine (or human) to follow a huge number of predefined rules (algorithm), in order to complete the task, without thinking or possessing the mind.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-24T00:36:04.737Z
Answers.Text: This answer is *very* insightful.  As absurd as it may sound to many, my project involves making "dumb" in addition to "smart" AI because humans playing the game have different levels of skill, and losing every game is no kind of fun for the vast majority of players.  Your point about the mistakes re: Kasparov is highly useful! (We're planning on using "glacially slow" reinforcement learning to tune individual automata to their owners, and raising or lowering tactical capability through modifying depth of lookahead.)
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-24T00:40:22.957Z
Answers.Text: I really liked the idea about making dumb plus smart AI. If we are trying to make something like humans, then we need to remember that humans also have IQ of 70 or 80 and not just 130+. @DukeZhou
Answers.UserDisplayName: Ugnes
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-24T00:42:30.607Z
Answers.Text: Although the basic non-trivial form of the game is currently unsolved, my six year-old niece, without having a full grasp of the game mechanics, can beat our dumbest automata "Boopsie", which makes purely random moves.  (Even our currently strongest, weak AI presents a challenge to the average adult player, but generally doesn't win by a margin that leaves the human feeling discouraged or humiliated.) In this model, Player satisfaction trumps AI strength.
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-24T00:27:47.037Z
Answers.LastActivityDate: 2017-08-24T00:27:47.037Z
Answers.UserDisplayName: Ugnes
Body: <p>I don't play nearly enough Chess to be able to answer.  For context, AlphaGo is stronger than the current strongest human player, but AlphaGo's game play has been cast as "inhuman" in the sense that it doesn't resemble human play. <em>(In Go, this can involve aesthetic qualities.)</em></p>
  
  <p>Really I'm wondering about "narrow" application of the Imitation Game/Turing Test, where one might design an automata to play more like a human, so that human players would be unable to determine if their opponent was a human or an automata.</p>
  
Tags: <turing-test><chess><go>
Title: Is the play of strong Chess AI easily distinguishable from human play?
UserDisplayName: DukeZhou
ThreadText: # Is the play of strong Chess AI easily distinguishable from human play?
  
  (Asked by: username_0 on 2017-08-23)
  
  I don't play nearly enough Chess to be able to answer. For context, AlphaGo is stronger than the current strongest human player, but AlphaGo's game play has been cast as "inhuman" in the sense that it doesn't resemble human play. *(In Go, this can involve aesthetic qualities.)*
  
  Really I'm wondering about "narrow" application of the Imitation Game/Turing Test, where one might design an automata to play more like a human, so that human players would be unable to determine if their opponent was a human or an automata.
  
  **Tags:** turing-test, chess, go
  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_5 on 2017-08-24. Score: 4 | Accepted answer
  
  AI programs that exist in today's world fall into the category of Narrow Intelligence. Narrow Intelligence
  are easy to distinguish when compared to General Intelligence (ones that resemble more like humans).  
  Highly advanced AI can often resemble to act like humans thought. I will like to talk about **Deep Blue** here.  
    
  Garry Kasparov, in a [series of matches](http://time.com/3705316/deep-blue-kasparov/) during Feb 1996, even though
  won three matches but, had a **" shattering experience"**. Deep Blue
  flummoxed him in that first game by making a move with no immediate
  material advantage; nudging a pawn into a position where it could be
  easily captured. He said, "*It was a wonderful and **extremely human**
  move*" and this threw him for a loop. He further added that he had
  played a lot of computers but had never experienced anything like
  this. He added, "**I could feel, I could smell, a new kind of intelligence
  across the table.**"  
  Next year he lost to an improved version of Deep Blue. Here also he
  lost mostly because of psychological reason. He later said that he
  was riled by a move the computer made that was so surprising and
  so **un-machine-like** that he was sure that IBM team had cheated.
  It was later found out that it was a glitch in Deep Blue. The program
  came across too many options and had no clear preference and thus
  played a random move.  
  It was this random move that shifted the game in favour of Deep Blue.
    
    
  One advantage that Deep Blue (or other programs) have is the ability to perform far **more computations** in a given time. Humans, on the other hand, can extend their decision tree only to *very limited depth*. On having no clear decision to take, we take a random decision and try to learn from the experience, unlike most Narrow Intelligence programs present today. We need a **fair amount of
  randomness and learning from experience** to be added to the program in order
  to make it more like a human.
  
  The answer to this question also depends on your position on **"[Chinese Room Argument](https://en.wikipedia.org/wiki/Chinese_room)"**.
  The argument holds that a program can't give a computer understanding
  or consciousness regardless of how intelligently or human like
  the program may make the computer behave. Chinese Room Argument focuses on something very important when it comes to **"Consciousness v/s Simulation of Consciousness"**. John Searle argued there that it is possible for a machine (or human) to follow a huge number of predefined rules (algorithm), in order to complete the task, without thinking or possessing the mind.
  
  > **Comments:**
  > **username_0**: This answer is *very* insightful.  As absurd as it may sound to many, my project involves making "dumb" in addition to "smart" AI because humans playing the game have different levels of skill, and losing every game is no kind of fun for the vast majority of players.  Your point about the mistakes re: Kasparov is highly useful! (We're planning on using "glacially slow" reinforcement learning to tune individual automata to their owners, and raising or lowering tactical capability through modifying depth of lookahead.)  
  > **username_5**: I really liked the idea about making dumb plus smart AI. If we are trying to make something like humans, then we need to remember that humans also have IQ of 70 or 80 and not just 130+. @username_0  
  > **username_0**: Although the basic non-trivial form of the game is currently unsolved, my six year-old niece, without having a full grasp of the game mechanics, can beat our dumbest automata "Boopsie", which makes purely random moves.  (Even our currently strongest, weak AI presents a challenge to the average adult player, but generally doesn't win by a margin that leaves the human feeling discouraged or humiliated.) In this model, Player satisfaction trumps AI strength.  
  
  ---
  
  ### Answer by username_1 on 2017-08-23. Score: 4
  
  There are three cases in which it is easily possible to distinguish strong AI play from the strong human play:
  
  1. The AI is playing at superhuman skill level
  
  This seems obvious, but I want to mention it for the sake of completeness. The current skill ceiling of top-level chess is well known and an opponent playing way above this skill ceiling must either be an AI or a chess guru who hid in a cave for the last centuries. Applying Occam's razor I would go with the AI. So to mimic human play the AI must make sure to stay at a human ELO level.
  
  2. The AI plays an obvious weak move in a losing or strange situation
  
  When the AI is in a bad situation and still tries to win the game without any reasonable move available, it might be tempted to play an obviously bad move, because the decision engine assigns it the highest probability of success. Such moves are considered rude at high-level play and a human player would not play them but resign. The most famous example of this is move 101 in the fourth game between [Lee Sedol and AlphaGo](http://gokifu.com/s/2iq8-gokifu-20160313-Alphago(9p)-Lee_Sedol(9p).html). AlphaGo just lost a big group but still played T9 to extend it even further. This is a move a rookie player stops making after his first 10 games because it is that obvious that it does not work. A professional go player would never do this under normal circumstances. AlphaGo was out of options and played the rude move instead of resigning. This doesn't mean an AI cannot be programmed to avoid such behavior, but dire or simply very strange situations might induce moves like this, giving the nature of the AI away.
  
  3. Other factors than the actual moves are considered
  
  Depending on the test scenario, the AI might give away its nature through other means than the actual moves. Things like moving too fast or playing without breaks for hours. But I assume the question was just focused on the actual play and not the surrounding factors.
  
  Those are the obvious examples where an AI might give away his true nature. I highly doubt that a strong AI, playing at the strong human level, can really be identified because of his "inhuman" style. I think it is more of a psychological pitfall to consider innovative moves from an AI "inhuman" and the same move from a pro player as "brilliant" if you know who is who. I am an amateur 3dan Go player - far from the top but with several years of experience under the belt - and at least I wasn't able to see anything "inhuman" in AlphaGos play so far. Ingo, it is well known that there are different styles of play, and after reviewing enough pro games one gets a sense of which player might have been educated in Korea, China, or Japan because certain patterns of strategies are more or less common in those countries. The lack of such an individual nuance might be an indication, that the player is actually a strong AI, but I am confident, that even the strongest pro wouldn't have a high success rate guessing who is human and who is the AI.
  
  In chess, there are significantly fewer options at every move, many fixed patterns especially in the opening, and therefore less room for an individual style. Therefore I consider the same challenge even more difficult than in go. So if an AI has reason to "play like a human", this should be possible with the current level of AI technology.
  
  
  ---
  
  ### Answer by username_2 on 2017-08-24. Score: 1
  
  Yes - in chess the term "computer move" is used to denote a move found by a chess engine that a human player would never find (often because they make some slight improvement that a human would not be able to calculate). Humans use pattern recognition and some calculation in order to understand the chess position they are in while computers are able to calculate (~30 - 120+) moves ahead, a feat that a human player cannot match. It is for this reason that one can distinguish a human move from a computer move at some times.
  
  > **Comments:**
  > **username_3**: Are you sure the are able to calculate ~30 to 120+ moves ahead? That seems very much to me.  
  > **username_0**: But couldn't that "bewildering to humans" move be perceived as a random choice, potentially played to disrupt?  And if it leads to a win, couldn't that be perceived as the disruption tactic having worked?  (I guess it partly depends on the expertise of the human competitor, as the Imitation Game is subjective...)  PS Welcome to AI!  
  
  ---
  
  ### Answer by username_4 on 2017-08-23. Score: 1
  
  I recall a friend saying that yes, it is somewhat obvious if you are playing against an AI.
  
  From what he said, against normal players, there is a certain rhythm and structure that "makes sense". But AI play doesn't have this quality, "it doesn't make sense, but it just works". This seems to echo what you mentioned about the aesthetic quality of Go
  
  Apparently because of this, chess/go AI can be detected during online play.
  
  
  ---
  
  