Answers.Body: <blockquote>
    <p>Is it fair to compare AlphaGo with a Human player?</p>
  </blockquote>
  
  <p>Depends on the purpose of the comparison.</p>
  
  <p>If we are comparing ability to win a game of Go, then yes.</p>
  
  <p>If we are comparing learning ability, then <em>maybe</em>. It depends on the task. AlphaGo and systems like it are capable of learning only in well-described limited domains. There may be an analogy with sensory learning (it might even be possible in theory to take a small piece of brain tissue and run an algorithm similar to AlphaGo's learning process on it). </p>
  
  <p>In general, the approach used by AlphaGo and other reinforcement learning successes is "trial-and-error plus function approximation". It seems analogous to perception and motor skills, such as object recognition or riding a bike, as opposed to reasoning skills and games as humans play them, which goes through many more cognitive and conscious layers that have no real analog in a RL system like AlphaGo.</p>
  
  <blockquote>
    <p>A human player plays limited games compared to a system that undergoes millions of iterations</p>
  </blockquote>
  
  <p>This is an advantage of a machine to learn this kind of task. It would equally apply in other simulated environments with simple rules. If your goal is to have the most skilled and optimal navigation of such a domain, the implication now is that you would not train a human expert through years of study, but to write the simulator and train an AlphaGo-like machine. </p>
  
  <p>This is no different a comparison than deciding cars and roads are better solutions to long distance travel for the general population than walking or horses and carts. It doesn't matter what underlies the advantage of one over the other, the assessment is cost/benefit, which resolves to a single comparable number. </p>
  
  <p>It would, however, be wrong to assess AlphaGo as a better general-purpose learning engine than a human. The fact that humans do not have to work fully through millions of simulations in full detail is important. It means that something about how humans learn is still not covered by learning machines. <a href="https://arxiv.org/abs/1604.00289" rel="nofollow noreferrer">Some of these things are understood and being discussed</a> - such as the ability to focus intuitively on important aspects of what to learn, the ability to reason about the environment, learning analogously or transfer learning from other domains.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T09:48:34.213Z
Answers.LastActivityDate: 2018-01-28T11:28:12.230Z
Answers.LastEditDate: 2018-01-28T11:28:12.230Z
Answers.UserDisplayName: Neil Slater
Answers.Body: <p>Yes it is. If we ever compare computers to humans we should take into acount the fact that computers can work 24 hours a day every day and faster than humans. That is bigest advantage of computers over humans.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T09:38:51.687Z
Answers.LastActivityDate: 2018-01-28T09:38:51.687Z
Answers.UserDisplayName: Tone Škoda
Answers.Body: <p>If you read through the abstracts of Chess AI papers, it is often pointed out that humans "search" the Chess game tree much more efficiently than computers, which was why it was so hard to beat the top humans in Chess for so many years.  (The human efficiency may have to do with intuition and judgement, which are difficult to replicate. "Confidence levels" for AI evaluations is one method of addressing these issues, as is "<a href="https://en.wikipedia.org/wiki/Monte_Carlo_tree_search" rel="nofollow noreferrer">monte carlo</a>".  But it's also important to note that humans are far more limited in the depth and breadth of their "searches", which is why, now that we have the right algorithms, humans can no longer win.)  </p>
  
  <blockquote>
    <p>Is it fair?  </p>
  </blockquote>
  
  <p>Perhaps the more salient question is:</p>
  
  <blockquote>
    <p><strong>Is it useful to compare AlphaGo to a human player?</strong></p>
  </blockquote>
  
  <p>It most certainly is, because it tells us that we have is sometimes termed a "strong-narrow AI" that can outperform a human in a single task. </p>
  
  <p>Why AlphaGo beating <a href="https://en.wikipedia.org/wiki/Lee_Sedol#Match_against_AlphaGo" rel="nofollow noreferrer">Lee Sedol</a> was a big deal is the complexity of Go, the <a href="https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability" rel="nofollow noreferrer">intractability</a> of the Go game tree, and the fact that computers were previously ineffective against high-level human Go players.</p>
  
  <p>This human vs. AI evaluation doesn't strictly fall under the "Turing Test" (<a href="https://en.wikipedia.org/wiki/Turing_test" rel="nofollow noreferrer">Imitation Game</a>), it does fall squarely under the maxim of <a href="https://en.wikipedia.org/wiki/Protagoras#Relativism" rel="nofollow noreferrer">Protagoras</a> that "Man is the measure of all things."</p>
  
  <p>This is critical because intelligence is a spectrum, and gauging strength of intelligence, in the context of intractable problems <em>(problems that cannot be fully solved due to their size)</em> is a function of relative strength of two agents, whether human or AI.</p>
  
  <p>This relative assessment is all we have, and all we may ever have for certain sets of problems.</p>
  
  <p>The problem with humans is not that we're not clever, but that our minds have cognitive limitations.  So to tackle certain problems, intelligent machines are useful.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-31T00:08:14.873Z
Answers.LastActivityDate: 2018-02-01T19:24:03.093Z
Answers.LastEditDate: 2018-02-01T19:24:03.093Z
Answers.UserDisplayName: DukeZhou
Answers.Body: <p>As a chess player and a AI/ML engineer, I can say yes, why not. I'm not sure why it isn't fair to compare anything if you give each side it's just due and do a 'fair comparison'. Obviously, what that encompasses is extremely subjective, but there are philosophical and logical measures of fairness.</p>
  
  <p>Now speaking on the comparison, AlphaZero and a Human's learning styles are much more similar than that of a Human's and Stockfish. This is mainly due to the fact that human's in some capacity use <a href="https://www.princeton.edu/~yael/Publications/Niv2009.pdf" rel="nofollow noreferrer">RL</a>, mainly in the dopaminergic neural pathways. While human behavior can certainly be modeled as an alpha/beta tree-search, it is not anything like the way we make decisions.  </p>
  
  <p>As far as the top humans, who cares? We we've been worse than computers for years.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-14T08:23:57.840Z
Answers.LastActivityDate: 2018-04-14T08:23:57.840Z
Answers.UserDisplayName: hisairnessag3
Answers.Body: <p>There is no such thing as fairness when comparing. You define a measure for performance and then compare the values of the measure.</p>
  
  <p>One sensible measure for playing the game of GO is the 'Number of games won', regardless of any investment in the development of the system, computational or sample efficiency. AlphaGo is currently at the top by this measure.</p>
  
  <p>Another sensible measure could be 'Number of games won under a restriction on sample efficiency during training'. As others pointed out, such a measure could be much more favorable for humans.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-01T13:46:03.393Z
Answers.LastActivityDate: 2018-02-02T08:27:22.170Z
Answers.LastEditDate: 2018-02-02T08:27:22.170Z
Answers.UserDisplayName: user1365490
Body: <p>A human player plays limited games compared to a system that undergoes millions of iterations. Is it really fair to compare AlphaGo with the world #1 player when we know experience increases with the increase in number of games played?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-03T15:51:41.317Z
Comments.Text: Define fair. :)
Comments.UserDisplayName: Jim
Tags: <philosophy><game-ai><alphago>
Title: Is it fair to compare AlphaGo with a Human player?
UserDisplayName: Sai
ThreadText: # Is it fair to compare AlphaGo with a Human player?
  
  (Asked by: username_0 on 2018-01-28)
  
  A human player plays limited games compared to a system that undergoes millions of iterations. Is it really fair to compare AlphaGo with the world #1 player when we know experience increases with the increase in number of games played?
  
  **Tags:** philosophy, game-ai, alphago
  
  > **Comments:**
  > **username_1**: Define fair. :)  
  
  ---
  
  ## 5 answers
  
  ### Answer by username_2 on 2018-01-28. Score: 2
  
  > Is it fair to compare AlphaGo with a Human player?
  
  Depends on the purpose of the comparison.
  
  If we are comparing ability to win a game of Go, then yes.
  
  If we are comparing learning ability, then *maybe*. It depends on the task. AlphaGo and systems like it are capable of learning only in well-described limited domains. There may be an analogy with sensory learning (it might even be possible in theory to take a small piece of brain tissue and run an algorithm similar to AlphaGo's learning process on it).
  
  In general, the approach used by AlphaGo and other reinforcement learning successes is "trial-and-error plus function approximation". It seems analogous to perception and motor skills, such as object recognition or riding a bike, as opposed to reasoning skills and games as humans play them, which goes through many more cognitive and conscious layers that have no real analog in a RL system like AlphaGo.
  
  > A human player plays limited games compared to a system that undergoes millions of iterations
  
  This is an advantage of a machine to learn this kind of task. It would equally apply in other simulated environments with simple rules. If your goal is to have the most skilled and optimal navigation of such a domain, the implication now is that you would not train a human expert through years of study, but to write the simulator and train an AlphaGo-like machine.
  
  This is no different a comparison than deciding cars and roads are better solutions to long distance travel for the general population than walking or horses and carts. It doesn't matter what underlies the advantage of one over the other, the assessment is cost/benefit, which resolves to a single comparable number.
  
  It would, however, be wrong to assess AlphaGo as a better general-purpose learning engine than a human. The fact that humans do not have to work fully through millions of simulations in full detail is important. It means that something about how humans learn is still not covered by learning machines. [Some of these things are understood and being discussed](https://arxiv.org/abs/1604.00289) - such as the ability to focus intuitively on important aspects of what to learn, the ability to reason about the environment, learning analogously or transfer learning from other domains.
  
  
  ---
  
  ### Answer by username_3 on 2018-01-28. Score: 0
  
  Yes it is. If we ever compare computers to humans we should take into acount the fact that computers can work 24 hours a day every day and faster than humans. That is bigest advantage of computers over humans.
  
  
  ---
  
  ### Answer by username_4 on 2018-01-31. Score: 0
  
  If you read through the abstracts of Chess AI papers, it is often pointed out that humans "search" the Chess game tree much more efficiently than computers, which was why it was so hard to beat the top humans in Chess for so many years. (The human efficiency may have to do with intuition and judgement, which are difficult to replicate. "Confidence levels" for AI evaluations is one method of addressing these issues, as is "[monte carlo](https://en.wikipedia.org/wiki/Monte_Carlo_tree_search)". But it's also important to note that humans are far more limited in the depth and breadth of their "searches", which is why, now that we have the right algorithms, humans can no longer win.)
  
  > Is it fair?
  
  Perhaps the more salient question is:
  
  > **Is it useful to compare AlphaGo to a human player?**
  
  It most certainly is, because it tells us that we have is sometimes termed a "strong-narrow AI" that can outperform a human in a single task.
  
  Why AlphaGo beating [Lee Sedol](https://en.wikipedia.org/wiki/Lee_Sedol#Match_against_AlphaGo) was a big deal is the complexity of Go, the [intractability](https://en.wikipedia.org/wiki/Computational_complexity_theory#Intractability) of the Go game tree, and the fact that computers were previously ineffective against high-level human Go players.
  
  This human vs. AI evaluation doesn't strictly fall under the "Turing Test" ([Imitation Game](https://en.wikipedia.org/wiki/Turing_test)), it does fall squarely under the maxim of [Protagoras](https://en.wikipedia.org/wiki/Protagoras#Relativism) that "Man is the measure of all things."
  
  This is critical because intelligence is a spectrum, and gauging strength of intelligence, in the context of intractable problems *(problems that cannot be fully solved due to their size)* is a function of relative strength of two agents, whether human or AI.
  
  This relative assessment is all we have, and all we may ever have for certain sets of problems.
  
  The problem with humans is not that we're not clever, but that our minds have cognitive limitations. So to tackle certain problems, intelligent machines are useful.
  
  
  ---
  
  ### Answer by username_5 on 2018-04-14. Score: 0
  
  As a chess player and a AI/ML engineer, I can say yes, why not. I'm not sure why it isn't fair to compare anything if you give each side it's just due and do a 'fair comparison'. Obviously, what that encompasses is extremely subjective, but there are philosophical and logical measures of fairness.
  
  Now speaking on the comparison, AlphaZero and a Human's learning styles are much more similar than that of a Human's and Stockfish. This is mainly due to the fact that human's in some capacity use [RL](https://www.princeton.edu/~yael/Publications/Niv2009.pdf), mainly in the dopaminergic neural pathways. While human behavior can certainly be modeled as an alpha/beta tree-search, it is not anything like the way we make decisions.
  
  As far as the top humans, who cares? We we've been worse than computers for years.
  
  
  ---
  
  ### Answer by username_6 on 2018-02-01. Score: 0
  
  There is no such thing as fairness when comparing. You define a measure for performance and then compare the values of the measure.
  
  One sensible measure for playing the game of GO is the 'Number of games won', regardless of any investment in the development of the system, computational or sample efficiency. AlphaGo is currently at the top by this measure.
  
  Another sensible measure could be 'Number of games won under a restriction on sample efficiency during training'. As others pointed out, such a measure could be much more favorable for humans.
  
  
  ---
  
  