Answers.Body: <p>Yes it's created something important. Until Alpha(Go) Zero all (or almost all) of <a href="https://en.wikipedia.org/wiki/Deep_learning" rel="nofollow noreferrer">Deep Learning</a> approach to <a href="https://en.wikipedia.org/wiki/Reinforcement_learning" rel="nofollow noreferrer">Reinforcement Learning</a> was based on <a href="https://en.wikipedia.org/wiki/Temporal_difference_learning" rel="nofollow noreferrer">Time Difference</a> loss function. The weakness of Time Difference loss function was that it was essentially training on itself, that is data produced by the same method was used as part of <a href="https://en.wikipedia.org/wiki/Nonlinear_regression" rel="nofollow noreferrer">regression</a> target. That was producing the problem of "extrapolation error" - solution would blow up, or oscillate wildly. There was attempts to mitigate that problem (n-steps algorithms), but they weren't helping much. Alpha Zero combined Deep Network with tree search instead (<a href="https://en.wikipedia.org/wiki/Monte_Carlo_tree_search" rel="nofollow noreferrer">Monte Carlo Tree Search</a>). Tree search algorithm produced wide and long fields of high precision data (<a href="https://en.wikipedia.org/wiki/Reinforcement_learning#Value_function" rel="nofollow noreferrer">value function</a>), with network influence on value much diminished. That way network was training mostly not on itself, but on data produced by tree, and tree search itself was accelerated greatly by network (using it as heuristics). The whole happens to be much more then sum of parts.</p>
  
  <p>This approach is not limited to broad game or RL theory. It may work for any problem for which high-precision simulator could be built. Essentially if problem allow Monte Carlo Tree Search, or other tree search which could be augmented with heuristic, Alpha Zero approach would probably work on it. Of cause Alpha Zero approach is computationally expensive, so it's not always efficient to apply it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-10T07:04:46.153Z
Answers.LastActivityDate: 2019-04-10T07:04:46.153Z
Answers.UserDisplayName: mirror2image
Body: <p>How can we use the ability of AlphaGo Zero computer, to do something in any other life important related field? Is it possible to make something important besides having created something so smart that can play mind games way better than humans?</p>
  
Tags: <machine-learning><gaming><alphago-zero>
Title: How do the achievements met in the gaming field (ex. AlphaGo Zero) impact other fields of application?
UserDisplayName: Jay Critch
ThreadText: # How do the achievements met in the gaming field (ex. AlphaGo Zero) impact other fields of application?
  
  (Asked by: username_0 on 2019-04-09)
  
  How can we use the ability of AlphaGo Zero computer, to do something in any other life important related field? Is it possible to make something important besides having created something so smart that can play mind games way better than humans?
  
  **Tags:** machine-learning, gaming, alphago-zero
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-04-10. Score: 1
  
  Yes it's created something important. Until Alpha(Go) Zero all (or almost all) of [Deep Learning](https://en.wikipedia.org/wiki/Deep_learning) approach to [Reinforcement Learning](https://en.wikipedia.org/wiki/Reinforcement_learning) was based on [Time Difference](https://en.wikipedia.org/wiki/Temporal_difference_learning) loss function. The weakness of Time Difference loss function was that it was essentially training on itself, that is data produced by the same method was used as part of [regression](https://en.wikipedia.org/wiki/Nonlinear_regression) target. That was producing the problem of "extrapolation error" - solution would blow up, or oscillate wildly. There was attempts to mitigate that problem (n-steps algorithms), but they weren't helping much. Alpha Zero combined Deep Network with tree search instead ([Monte Carlo Tree Search](https://en.wikipedia.org/wiki/Monte_Carlo_tree_search)). Tree search algorithm produced wide and long fields of high precision data ([value function](https://en.wikipedia.org/wiki/Reinforcement_learning#Value_function)), with network influence on value much diminished. That way network was training mostly not on itself, but on data produced by tree, and tree search itself was accelerated greatly by network (using it as heuristics). The whole happens to be much more then sum of parts.
  
  This approach is not limited to broad game or RL theory. It may work for any problem for which high-precision simulator could be built. Essentially if problem allow Monte Carlo Tree Search, or other tree search which could be augmented with heuristic, Alpha Zero approach would probably work on it. Of cause Alpha Zero approach is computationally expensive, so it's not always efficient to apply it.
  
  
  ---
  
  