Answers.Body: <p>I see three ways of tackling this:</p>
  
  <ul>
  <li>You can add a confidence value to you position evaluation function. And then use this to improve your tree search. This sounds amenable to mathematical analysis, you'd have to adapt min-max search to handle a trade-off between confidence and evaluation.</li>
  <li>You can output a distribution of evaluations instead of a single numerical value. And again use this information to improve tree search. That's similar to the one above.</li>
  <li>You directly output a "how much compute should be spent on this position"-value.</li>
  </ul>
  
  <p>The second thing has been done in some of Deepmind's video game reinforcement learning projects. So you might want to take a look at that. </p>
  
  <p>I haven't seen anything like this for turn-based board games. It certainly sounds like a sensible idea. </p>
  
  <p>If you want to do the third thing you need to figure out how to capture and backpropagate "how much compute should be spent on this position". That doesn't sound trivial and is not guaranteed to work, but it would be pretty neat and it would tilt your work more towards Deep Learning kinda stuff. </p>
  
  <p>No, you should not use genetic algorithms. It's almost always better to optimise your function non-randomly. In your case backpropagation is highly likely to be much more efficient. </p>
  
  <p>One pitfall is that even backpropagation might not be efficient enough. If you train your networks from scratch by self-play on your laptop, it is possible that the learning never really takes off. Self-play doesn't provide enough information for efficient learning, which is why AlphaZero and co use humongous amounts of compute.</p>
  
  <p>So it might be more sensible to plan your training in such a way that it can use existing games. Then evaluation and confidence can be learned without doing any tree-search yourself. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-11T13:54:56.213Z
Answers.Text: Thanks! one last question... Would you recommend setting a target ELO for this project ? If so, what?
Answers.UserDisplayName: S. Dauncey
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-11T14:45:02.877Z
Answers.Text: I wouldn't set a target Elo. Instead I would compare the engine to an engine version without these ideas. I.e. you would implement a tree search and then run it with a simple evaluation function like only material, and compare that to your improved version with neural network evaluation and confidence etc. This way you can show that your ideas actually do something.
Answers.UserDisplayName: BlindKungFuMaster
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-08T10:52:53.943Z
Answers.LastActivityDate: 2018-10-08T10:52:53.943Z
Answers.UserDisplayName: BlindKungFuMaster
Body: <p>for my A-level computing project I was going to do an investigation into a type of engine design which incorporates Neural Networks that I haven't seen elsewhere; as part of this project I need to base my specification around the recipients of the outcomes for investigative projects (hopefully you guys).</p>
  
  <p>My current design idea is this:</p>
  
  <ul>
  <li><p>For a given position, A neural network will evaluate the position and spit out two values: how good it is for white/black, and a value representing how much will be gained from analysing this position further.</p></li>
  <li><p>All of this information (along with the depth) could then be combined to determine whether the engine should analyse to a layer further down in the decision tree or return the first value (how good the position is for white) so that it could then be used as a "leaf" in the decision tree that could then be analysed by mini-max (along with suitable pruning algorithms) to give an overall analysis of a given move.</p></li>
  <li><p>To train the neural network: I was going to play two networks against each other and analyse their evaluations of the certain moments in the game post-game to tell them (using back propagation) where they should look deeper or evaluate a certain position as better/worse based on who won the match. I think this method is similar to policy gradients. Another approach I was looking into was a genetic algorithm however I was sceptical of whether I have nearly enough processing power available to me seen as I'm doing this all on my laptop :(</p></li>
  </ul>
  
  <p>I would be very grateful if you could answer these questions for me:</p>
  
  <ol>
  <li><p>Have you seen any similar projects like this before?</p></li>
  <li><p>What are any pitfalls that you could see with this method?</p></li>
  <li><p>would you consider using a similar method in your engines? Why/ why not?</p></li>
  <li><p>how would you alter this investigative project so that it could benefit you more personally?</p></li>
  <li><p>Would you use the genetic algorithm approach?</p></li>
  </ol>
  
  <p>Thanks a lot in advance for any replies!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-09T00:36:50.657Z
Comments.Text: This appears to be a similar project, although admittedly I've not looked into it in depth, it's based loosely on Alpha Zero, Google's neural network for chess (and other games): https://github.com/Zeta36/chess-alpha-zero
Comments.UserDisplayName: Jon Story
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-06T18:43:25.383Z
Comments.Text: how can you know how much more can be gained by analyzing further unless  you actually analysed further
Comments.UserDisplayName: edwina oliver
Tags: <engines><alphazero>
Title: Guidance on a new idea for an engine design?
ThreadText: # Guidance on a new idea for an engine design?
  
  (Asked by: username_0 on 2018-10-07)
  
  for my A-level computing project I was going to do an investigation into a type of engine design which incorporates Neural Networks that I haven't seen elsewhere; as part of this project I need to base my specification around the recipients of the outcomes for investigative projects (hopefully you guys).
  
  My current design idea is this:
  
  * For a given position, A neural network will evaluate the position and spit out two values: how good it is for white/black, and a value representing how much will be gained from analysing this position further.
  * All of this information (along with the depth) could then be combined to determine whether the engine should analyse to a layer further down in the decision tree or return the first value (how good the position is for white) so that it could then be used as a "leaf" in the decision tree that could then be analysed by mini-max (along with suitable pruning algorithms) to give an overall analysis of a given move.
  * To train the neural network: I was going to play two networks against each other and analyse their evaluations of the certain moments in the game post-game to tell them (using back propagation) where they should look deeper or evaluate a certain position as better/worse based on who won the match. I think this method is similar to policy gradients. Another approach I was looking into was a genetic algorithm however I was sceptical of whether I have nearly enough processing power available to me seen as I'm doing this all on my laptop :(
  
  I would be very grateful if you could answer these questions for me:
  
  1. Have you seen any similar projects like this before?
  2. What are any pitfalls that you could see with this method?
  3. would you consider using a similar method in your engines? Why/ why not?
  4. how would you alter this investigative project so that it could benefit you more personally?
  5. Would you use the genetic algorithm approach?
  
  Thanks a lot in advance for any replies!
  
  **Tags:** engines, alphazero
  
  > **Comments:**
  > **username_1**: This appears to be a similar project, although admittedly I've not looked into it in depth, it's based loosely on Alpha Zero, Google's neural network for chess (and other games): https://github.com/Zeta36/chess-alpha-zero  
  > **username_2**: how can you know how much more can be gained by analyzing further unless  you actually analysed further  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2018-10-08. Score: 1
  
  I see three ways of tackling this:
  
  * You can add a confidence value to you position evaluation function. And then use this to improve your tree search. This sounds amenable to mathematical analysis, you'd have to adapt min-max search to handle a trade-off between confidence and evaluation.
  * You can output a distribution of evaluations instead of a single numerical value. And again use this information to improve tree search. That's similar to the one above.
  * You directly output a "how much compute should be spent on this position"-value.
  
  The second thing has been done in some of Deepmind's video game reinforcement learning projects. So you might want to take a look at that.
  
  I haven't seen anything like this for turn-based board games. It certainly sounds like a sensible idea.
  
  If you want to do the third thing you need to figure out how to capture and backpropagate "how much compute should be spent on this position". That doesn't sound trivial and is not guaranteed to work, but it would be pretty neat and it would tilt your work more towards Deep Learning kinda stuff.
  
  No, you should not use genetic algorithms. It's almost always better to optimise your function non-randomly. In your case backpropagation is highly likely to be much more efficient.
  
  One pitfall is that even backpropagation might not be efficient enough. If you train your networks from scratch by self-play on your laptop, it is possible that the learning never really takes off. Self-play doesn't provide enough information for efficient learning, which is why AlphaZero and co use humongous amounts of compute.
  
  So it might be more sensible to plan your training in such a way that it can use existing games. Then evaluation and confidence can be learned without doing any tree-search yourself.
  
  > **Comments:**
  > **username_0**: Thanks! one last question... Would you recommend setting a target ELO for this project ? If so, what?  
  > **username_3**: I wouldn't set a target Elo. Instead I would compare the engine to an engine version without these ideas. I.e. you would implement a tree search and then run it with a simple evaluation function like only material, and compare that to your improved version with neural network evaluation and confidence etc. This way you can show that your ideas actually do something.  
  
  ---
  
  