Answers.Body: <p>As far as I understand, you don't give the snake rewards for eating fast.  In fact, "Each step, 1 point." means that the slower it finds food, the better. So why would it do turns, considering it is dangerous.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-27T00:02:15.977Z
Answers.Text: At first I don't added the "1 point per step" method and actually made the reward being higher if the food was eaten fast and the pattern was the same. I ended up with this after a thousand tests. Made it aprox like this: http://jsfiddle.net/gunsct5r/5/ check line `296` now. That's the reason `stepsWithoutPoints` is reset to `1` instead of `0`. To never divide by `0`. Actually is a good point but removed it and tested other things as it also don't worked.
Answers.UserDisplayName: Jorge Fuentes González
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-26T22:23:16.720Z
Answers.LastActivityDate: 2017-12-26T22:23:16.720Z
Answers.UserDisplayName: user31264
Answers.Body: <p>I observed the scoring system you used and compared it to the field size and starving rate. The worms either died or did not try enough hard with the prize of 10 of eating, so I added the amount to 1000.</p>
  
  <p>This did still not introduce the 180 degrees turn yet. I had to adjust these settings on Neat:</p>
  
  <pre><code>  mutationRate: 0.7,
    mutationAmount: 6,
  </code></pre>
  
  <p>which introduced <img src="https://i.sstatic.net/OT1iN.png" alt="the effect on this screenshot"> where the worms optimize their path as diagonal and when forced to change 90 direction, actually a tiny 180 happens on screen.</p>
  
  <p>On same mutationRate and mutationAmount, but 10 points I once observed a horizontal 180, when the food appeared back and diagonal movement was not invented. Also I observed with this last setup a motion of constant 180s flow where consecutive vertical or horizontal 180s formed a two pixel thick line on screen. </p>
  
  <p>These observations do not 100% do the same as your animation, but do instead introduce various ways that are quite near, or can be in certain conditions an initiative to such phenomenom. </p>
  
  <p><img src="https://i.sstatic.net/ZSKbn.png" alt="This picture shows also the two pixel line. "></p>
  
  <p>White worm introduces two pixel line, red 90 degree turn with miniature 180.</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>I changed the nice curve ratio like this:</p>
  
  <pre><code>     var ratio = 0.75 + ((mindir / maxdir) * 0.25* difstepsratio);
  </code></pre>
  
  <p>and around generation 200 there lived one worm to be able to move to all directions and make 180 if needed.</p>
  
  <p>Unfortunately it died before I found screen shot buttons from my phone and (s)he happened to become second in the race and thus was left as a unicorn who existed only once.</p>
  
  <p>Still remains mystery whether that mutation could survive and become the main species some day. </p>
  
  <p><strong>Further remarks:</strong></p>
  
  <p>I noticed that when mutation takes place in snake code the same pattern is replicated many times. If snake gets idea of consecutive turn lefts or turn rights when it is long, a suicide with tail is quite often taking place.</p>
  
  <p>I tried to taggle this with a higher score on snake collision than starvation and around generation #180 the dna of suicidal 360 loop was born and remained some generations.</p>
  
  <p>However, this generation did not evolve to the desired intelligent 180 worm I previously had seen. Instead, the suicide was so rewarding that it killed that branch quite soon away.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-26T19:24:57.727Z
Answers.Text: Thank you for your answer. I really appreciate it. I think that I've not explained myself very well. In your photos the path is still upwards (or sidewards). They do not turn 180 in both directions. They do left-right but not up-down, or they do up-down but not left-right. I want them to learn that if they pass near a piece of food, they can turn around and get it instead of continuing to the edge and appearing from the other side. If I added a scoring system to penalize the ones that do not turn to get the food, I don't know why I never get a mutation where they do.
Answers.UserDisplayName: Jorge Fuentes González
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-26T19:25:54.210Z
Answers.Text: I think that I added all the necessary inputs for the network recognize it (food position, wall positions and penalize linear movements), but I'm sure that I've done something wrong.
Answers.UserDisplayName: Jorge Fuentes González
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-26T18:58:06.657Z
Answers.LastActivityDate: 2018-01-01T09:21:22.497Z
Answers.LastEditDate: 2018-01-01T09:21:22.497Z
Answers.UserDisplayName: mico
Body: <p>I'm really new to neural networks. I'm trying to make a neural network with genetics algorithms which will make a snake learn to look for the food and avoid hitting his tail.</p>
  
  <p>The thing is that I think that I've done it, but as there's no walls the snake learns to go one direction only without making a 180 turn <a href="https://i.imgur.com/cdVmixk.gifv" rel="nofollow noreferrer">[GIF HERE]</a>.</p>
  
  <p>I've tried to incentivate mutations that make them turn by decreasing the score of the snakes that always takes the same directions, but it don't works. I've only made them dumber, needing more breeds to reach another "smart" linear snake.</p>
  
  <p>I've made a network with 5 inputs:</p>
  
  <ul>
  <li>Food position relative to my position and direction (2 inputs. x and y)</li>
  <li>Nearest wall (my tail) if I turn left</li>
  <li>Nearest wall (my tail) if I do not turn</li>
  <li>Nearest wall (my tail) if I turn right</li>
  </ul>
  
  <p>The output are 3, being the first one turning left, the second do not turn and the third going right. I make the snake go the highest one of the 3 outputs.</p>
  
  <p>I've added 1 hidden layer of 8 neurons (inputs + outputs premise).</p>
  
  <p>The way I calculate the score is:</p>
  
  <ul>
  <li>Each step, 1 point.</li>
  <li>Each food eaten, 10 points.</li>
  <li>If the snake lasts too much time without eating food, dies.</li>
  <li>If hits his tail, dies.</li>
  </ul>
  
  <p>Then I save each time the direction this snake has gone (up, right, down, left) and increment them by one. When the snake dies, I weight the final score by the difference between the lowest and highest values. If the difference is high, they receive a big penalty (down to 0.25 of their score). This way if a snake is pretty much linear gets a high penalty and if a snake does a cool pattern, gets a low penalty.</p>
  
  <p>Also, I keep a record of each time a direction change happens compared to the last direction change, so if a snake keeps going in circles don't gets a high score because of "cool pattern method" for going all 4 directions.</p>
  
  <p>With all this, I don't understand why my best snakes are always the linear ones :-/</p>
  
  <p>I spawn 20 snakes and get the best 4 of each generation when everyone dies.</p>
  
  <p>For generations I use neataptics.js and for neural networks I use synaptics.js. I have a Fiddle here: <a href="http://jsfiddle.net/Llorx/gunsct5r/" rel="nofollow noreferrer">http://jsfiddle.net/Llorx/gunsct5r/</a></p>
  
  <p>At line <code>10</code> you can see the network definition. At <code>211</code> you can see the snake "view" (food position and walls. Where it gets the inputs) and at <code>164</code> you can see the score weight calculation depending on steps taken that I mentioned before.</p>
  
  <p>All inputs are normalized from 0 to 1.</p>
  
  <p>I'm sure that I'm doing, not one, but a lot of things pretty bad, as I'm a newbie on this, but some light on this will be really cool.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-26T13:39:57.847Z
Comments.Text: And that downvote??? Hate those ninja downvoters without explanation, really. That way I'm never going to learn what's the problem and how to improve the question. That's your way to improve this awesome webpage?
Comments.UserDisplayName: Jorge Fuentes González
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-26T21:58:51.197Z
Comments.Text: After looking at the code in fiddle I started to think how the snake can see backward distance? It would be quite big leap to find out only by guessing to turn left or right when the numbers tells to go around in front.
Comments.UserDisplayName: mico
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-26T22:04:42.707Z
Comments.Text: @mico Well, I add as an input the position of the food in reference of the current position and direction of the snake. What it receives as "left, right, front" view only are the walls, so she don't knows when has a wall behind, but actually knows the exact position of the food relative to itself.
Comments.UserDisplayName: Jorge Fuentes González
Tags: <neural-networks><genetic-algorithms>
Title: Trying to make snake learn different directions
UserDisplayName: Jorge Fuentes González
ThreadText: # Trying to make snake learn different directions
  
  (Asked by: username_0 on 2017-12-22)
  
  I'm really new to neural networks. I'm trying to make a neural network with genetics algorithms which will make a snake learn to look for the food and avoid hitting his tail.
  
  The thing is that I think that I've done it, but as there's no walls the snake learns to go one direction only without making a 180 turn [[GIF HERE]](https://i.imgur.com/cdVmixk.gifv).
  
  I've tried to incentivate mutations that make them turn by decreasing the score of the snakes that always takes the same directions, but it don't works. I've only made them dumber, needing more breeds to reach another "smart" linear snake.
  
  I've made a network with 5 inputs:
  
  * Food position relative to my position and direction (2 inputs. x and y)
  * Nearest wall (my tail) if I turn left
  * Nearest wall (my tail) if I do not turn
  * Nearest wall (my tail) if I turn right
  
  The output are 3, being the first one turning left, the second do not turn and the third going right. I make the snake go the highest one of the 3 outputs.
  
  I've added 1 hidden layer of 8 neurons (inputs + outputs premise).
  
  The way I calculate the score is:
  
  * Each step, 1 point.
  * Each food eaten, 10 points.
  * If the snake lasts too much time without eating food, dies.
  * If hits his tail, dies.
  
  Then I save each time the direction this snake has gone (up, right, down, left) and increment them by one. When the snake dies, I weight the final score by the difference between the lowest and highest values. If the difference is high, they receive a big penalty (down to 0.25 of their score). This way if a snake is pretty much linear gets a high penalty and if a snake does a cool pattern, gets a low penalty.
  
  Also, I keep a record of each time a direction change happens compared to the last direction change, so if a snake keeps going in circles don't gets a high score because of "cool pattern method" for going all 4 directions.
  
  With all this, I don't understand why my best snakes are always the linear ones :-/
  
  I spawn 20 snakes and get the best 4 of each generation when everyone dies.
  
  For generations I use neataptics.js and for neural networks I use synaptics.js. I have a Fiddle here: <http://jsfiddle.net/Llorx/gunsct5r/>
  
  At line `10` you can see the network definition. At `211` you can see the snake "view" (food position and walls. Where it gets the inputs) and at `164` you can see the score weight calculation depending on steps taken that I mentioned before.
  
  All inputs are normalized from 0 to 1.
  
  I'm sure that I'm doing, not one, but a lot of things pretty bad, as I'm a newbie on this, but some light on this will be really cool.
  
  **Tags:** neural-networks, genetic-algorithms
  > **Comments:**
  > **username_0**: And that downvote??? Hate those ninja downvoters without explanation, really. That way I'm never going to learn what's the problem and how to improve the question. That's your way to improve this awesome webpage?  
  > **username_1**: After looking at the code in fiddle I started to think how the snake can see backward distance? It would be quite big leap to find out only by guessing to turn left or right when the numbers tells to go around in front.  
  > **username_0**: @mico Well, I add as an input the position of the food in reference of the current position and direction of the snake. What it receives as "left, right, front" view only are the walls, so she don't knows when has a wall behind, but actually knows the exact position of the food relative to itself.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2017-12-26. Score: 1
  
  As far as I understand, you don't give the snake rewards for eating fast. In fact, "Each step, 1 point." means that the slower it finds food, the better. So why would it do turns, considering it is dangerous.
  
  > **Comments:**
  > **username_0**: At first I don't added the "1 point per step" method and actually made the reward being higher if the food was eaten fast and the pattern was the same. I ended up with this after a thousand tests. Made it aprox like this: http://jsfiddle.net/gunsct5r/5/ check line `296` now. That's the reason `stepsWithoutPoints` is reset to `1` instead of `0`. To never divide by `0`. Actually is a good point but removed it and tested other things as it also don't worked.  
  
  ---
  
  ### Answer by username_1 on 2017-12-26. Score: 1
  
  I observed the scoring system you used and compared it to the field size and starving rate. The worms either died or did not try enough hard with the prize of 10 of eating, so I added the amount to 1000.
  This did still not introduce the 180 degrees turn yet. I had to adjust these settings on Neat:
    mutationRate: 0.7,
    mutationAmount: 6,
  
  which introduced  where the worms optimize their path as diagonal and when forced to change 90 direction, actually a tiny 180 happens on screen.
  On same mutationRate and mutationAmount, but 10 points I once observed a horizontal 180, when the food appeared back and diagonal movement was not invented. Also I observed with this last setup a motion of constant 180s flow where consecutive vertical or horizontal 180s formed a two pixel thick line on screen. 
  These observations do not 100% do the same as your animation, but do instead introduce various ways that are quite near, or can be in certain conditions an initiative to such phenomenom. 
  
  White worm introduces two pixel line, red 90 degree turn with miniature 180.
  EDIT
  I changed the nice curve ratio like this:
       var ratio = 0.75 + ((mindir / maxdir) * 0.25* difstepsratio);
  
  and around generation 200 there lived one worm to be able to move to all directions and make 180 if needed.
  Unfortunately it died before I found screen shot buttons from my phone and (s)he happened to become second in the race and thus was left as a unicorn who existed only once.
  Still remains mystery whether that mutation could survive and become the main species some day. 
  Further remarks:
  I noticed that when mutation takes place in snake code the same pattern is replicated many times. If snake gets idea of consecutive turn lefts or turn rights when it is long, a suicide with tail is quite often taking place.
  I tried to taggle this with a higher score on snake collision than starvation and around generation #180 the dna of suicidal 360 loop was born and remained some generations.
  However, this generation did not evolve to the desired intelligent 180 worm I previously had seen. Instead, the suicide was so rewarding that it killed that branch quite soon away.
  
  > **Comments:**
  > **username_0**: Thank you for your answer. I really appreciate it. I think that I've not explained myself very well. In your photos the path is still upwards (or sidewards). They do not turn 180 in both directions. They do left-right but not up-down, or they do up-down but not left-right. I want them to learn that if they pass near a piece of food, they can turn around and get it instead of continuing to the edge and appearing from the other side. If I added a scoring system to penalize the ones that do not turn to get the food, I don't know why I never get a mutation where they do.  
  > **username_0**: I think that I added all the necessary inputs for the network recognize it (food position, wall positions and penalize linear movements), but I'm sure that I've done something wrong.  
  
  ---
  
  