Answers.Body: <p>What you are trying to achieve, is a game that <strong>learns</strong> to play flappy bird. For doing this you need a neural network AND a genetic algorithm, those two things work together.<br>
  About your concerns on the output, you don't have to know if the action will benefit or not, i will soon explain why.<br><br></p>
  
  <h2>The neural network part</h2>
  
  <p>So, what you need is to know how to build a neural network, i don't know your knowledge about it, but i suggest starting from the basics. In this scenario, you need a feed forward neural network, because you just take the inputs from the current flappy bird scene/frame (such as the y position of the bird, the distance from the closes pipe ecc..) and feed it through a network that outputs either 1 or 0 (jump or don't jump) in the only output neuron we just decided it has.<br><br></p>
  
  <p>In python you can implement a neural network from scratch, or using a neural network framework that does al the dirty work for you.</p>
  
  <ul>
  <li>From scratch you would need to use numpy for matrix calculations, and you would need to learn matrix multiplication, dot products and all that fancy stuff (You can just let numpy taking care of the matrix calculations, but understanding how it works behind the scenes always helps understand new problems that you might come across when doing more advanced stuff)</li>
  <li>Using a framework like <a href="https://www.tensorflow.org/" rel="nofollow noreferrer">Tensorflow</a> for python, the only thing you need to do is find the right structure for the network you want to use. You will not have to worry about how activations work, or how the feed forward is performed (But again, it's a good thing to know when working with neural nets)</li>
  </ul>
  
  <h2>The genetic algorithm part or ""learning""</h2>
  
  <p>I say ""learning"" because at first sight it might look like learning, but really it is not.
  The genetic algorithm works like "the survival of the fittest", where the "smarter" birds, which are the ones that reached the higher score on the current generation, will have a chance to have their child little birds, that have the same brain as their parent, with either some minimal modifications, or a mix of their parent brains.<br>
  The process of this ""learning"", so the genetic algorithm, works like so:<br><br></p>
  
  <ol>
  <li>Create a generation of let's say 200 birds, every bird has a brain with random weights, so at the first run, they are all very...not smart</li>
  <li>The game starts, and every frame of the game, the brain of the bird recieves as input some data that is taken from the current frame ( y pos of the bird, distance from pipe...)</li>
  <li>The brain ( neural network ) of each bird, performs a feed forward with that data, and outputs what at the beginning is a very random result, let's say 0.75 for one bird</li>
  <li>At this point you decided that 0.75 is greater than 0.5, so you take that as a 1, which stands for "jump", while if it was 0.3, so 0, the bird does nothing and keeps falling</li>
  <li>Shortly the bird will die cause he has no idea of what he is doing, so he most likely collides with a pipe or the ground.</li>
  <li>After all birds met their fate, you see that some birds reached further than others, so you choose, for example, 5 of the best performing ones.</li>
  <li>Now you try to create a new generation of 200 birds using only the brains of those 5 that were choosen, by mixing and modifying theyr brains</li>
  <li>Now the new birds have a brand new brain, that in some cases might be better than the previous one, so chances are that some of those birds will reach a higher score, therefore flap further into the level.</li>
  <li>Repeat from point 6</li>
  </ol>
  
  <p>In practice your "perform_genetic_algorithm" function in python, will have to choose the birds with the highest score, and as wild as it sounds, mix their brains and modify them, hoping that some modifications will improve the performance of the bird.<br></p>
  
  <blockquote>
    <p>I can't think of output since you don't really know if the action of flapping will benefit you or not</p>
  </blockquote>
  
  <p>The mechanism above explains why you basically do not care at all about the output, except saying to the game engine: "hey the bird decided to flap, do it". Whether it's the right action or not, doesn't matter, as the smarter birds are <strong>naturally</strong> gonna get further and so be more likely to be choosen for next generation.</p>
  
  <p>Hopefully now it's all more clear.
  <br>
  Here is some useful links for building a neural network and for understanding the genetic algorithm:<br></p>
  
  <ul>
  <li><a href="http://neuralnetworksanddeeplearning.com/chap1.html" rel="nofollow noreferrer">How to build a neural network</a>: I am linking this because it contains all useful information about how to build a very basic neural network in python. In your case, you would have to ignore all the part about backpropagation, loss &amp; error calculation and SGD, and just look at the <strong>feed forward</strong> part.<br></li>
  <li><a href="https://enlight.nyc/projects/neural-network/" rel="nofollow noreferrer">How to build a neural network - 2</a>: This is another example of building a neural network that i found really useful, probably it's simpler and more straight forward than the previous link, but again, the backpropagation part is not needed for this genetic based learning.<br></li>
  <li><a href="https://www.youtube.com/watch?v=9zfeTw-uFCw&amp;list=PLRqwX-V7Uu6bJM3VgzjNV5YxVxUwzALHV" rel="nofollow noreferrer">Video tutorials on genetic algorithm</a>: This is a very long but very explanatory playlist of videos that dives into the nature of genetic algorithms and how to implement one</li>
  <li><a href="https://towardsdatascience.com/introduction-to-optimization-with-genetic-algorithm-2f5001d9964b" rel="nofollow noreferrer">Genetic algorithm optimization</a>: Other source about genetic algorithms</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-07T08:06:27.260Z
Answers.LastActivityDate: 2020-04-07T09:03:59.717Z
Answers.LastEditDate: 2020-04-07T09:03:59.717Z
Answers.UserDisplayName: Alessandro Valentino
Answers.Body: <p>I would recommend learning about Reinforcement learning first. You don't need a dataset as you train your network by letting it play the game over and over again. but knowing how to do so doea mean finding out about markov decision process and how you can use the neural network to solve this.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-28T09:56:13.963Z
Answers.LastActivityDate: 2019-11-28T09:56:13.963Z
Answers.UserDisplayName: Lustwelpintje
Answers.Body: <p>I agree a neural net is a good start and you might want to add a constitutional neural network to list of models you want to test and evaluate. </p>
  
  <p>However your question was really on getting something up and running using python. I don't have enough time for me to play games, much more let my computers do it for me... but I know <a href="https://github.com/NJacobsohn/Your-Learning-is-in-Another-Castle#the-numerical-model" rel="nofollow noreferrer">someone who has</a> done such a project. </p>
  
  <p>Hopefully that helps.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-28T23:53:10.360Z
Answers.LastActivityDate: 2019-10-28T23:53:10.360Z
Answers.UserDisplayName: hrokr
Body: <p>I am new in the field of AI. I am working to create the flappy bird using Genetic Algorithm. After reading and seeing some examples, I saw that most implementations use a Neural Network + Genetic Algorithm and after certain generations, you achieve a very good agent that survives very long.</p>
  
  <p>I currently struggling to implement the Neural Network since I have never taken a Machine Learning course.
  On many examples that I have read neural networks require training inputs and outputs. For the flappy bird, I can't think of output since you don't really know if the action of flapping will benefit you or not.</p>
  
  <p>In the example that I followed, Synaptic.js is used and it is pretty straight-forward. However, in Python, I can't find a simple library that will initialize randomly and adjust the weights and biases depending on the good agents that survive longer.</p>
  
  <p>What would be the right way to implement this Neural Network without having a training dataset?</p>
  
  <p>Is there any way to create Flappy Bird without using Neural Networks, just Genetic Algorithm?</p>
  
  <p>The example in Javascript that I am referring to: <a href="https://github.com/ssusnic/Machine-Learning-Flappy-Bird" rel="noreferrer">Flappy Bird Using Machine Learning</a></p>
  
Tags: <neural-networks><python><genetic-algorithms>
Title: How to implement a neural network for Flappy Bird in Python?
UserDisplayName: souleatzz
ThreadText: # How to implement a neural network for Flappy Bird in Python?
  
  (Asked by: username_0 on 2019-05-17)
  
  I am new in the field of AI. I am working to create the flappy bird using Genetic Algorithm. After reading and seeing some examples, I saw that most implementations use a Neural Network + Genetic Algorithm and after certain generations, you achieve a very good agent that survives very long.
  
  I currently struggling to implement the Neural Network since I have never taken a Machine Learning course.
  On many examples that I have read neural networks require training inputs and outputs. For the flappy bird, I can't think of output since you don't really know if the action of flapping will benefit you or not.
  
  In the example that I followed, Synaptic.js is used and it is pretty straight-forward. However, in Python, I can't find a simple library that will initialize randomly and adjust the weights and biases depending on the good agents that survive longer.
  
  What would be the right way to implement this Neural Network without having a training dataset?
  
  Is there any way to create Flappy Bird without using Neural Networks, just Genetic Algorithm?
  
  The example in Javascript that I am referring to: [Flappy Bird Using Machine Learning](https://github.com/ssusnic/Machine-Learning-Flappy-Bird)
  
  **Tags:** neural-networks, python, genetic-algorithms
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2020-04-07. Score: 2
  
  What you are trying to achieve, is a game that **learns** to play flappy bird. For doing this you need a neural network AND a genetic algorithm, those two things work together.  
  About your concerns on the output, you don't have to know if the action will benefit or not, i will soon explain why.
  
  ## The neural network part
  
  So, what you need is to know how to build a neural network, i don't know your knowledge about it, but i suggest starting from the basics. In this scenario, you need a feed forward neural network, because you just take the inputs from the current flappy bird scene/frame (such as the y position of the bird, the distance from the closes pipe ecc..) and feed it through a network that outputs either 1 or 0 (jump or don't jump) in the only output neuron we just decided it has.
  
  In python you can implement a neural network from scratch, or using a neural network framework that does al the dirty work for you.
  
  * From scratch you would need to use numpy for matrix calculations, and you would need to learn matrix multiplication, dot products and all that fancy stuff (You can just let numpy taking care of the matrix calculations, but understanding how it works behind the scenes always helps understand new problems that you might come across when doing more advanced stuff)
  * Using a framework like [Tensorflow](https://www.tensorflow.org/) for python, the only thing you need to do is find the right structure for the network you want to use. You will not have to worry about how activations work, or how the feed forward is performed (But again, it's a good thing to know when working with neural nets)
  
  ## The genetic algorithm part or ""learning""
  
  I say ""learning"" because at first sight it might look like learning, but really it is not.
  The genetic algorithm works like "the survival of the fittest", where the "smarter" birds, which are the ones that reached the higher score on the current generation, will have a chance to have their child little birds, that have the same brain as their parent, with either some minimal modifications, or a mix of their parent brains.  
  The process of this ""learning"", so the genetic algorithm, works like so:
  
  1. Create a generation of let's say 200 birds, every bird has a brain with random weights, so at the first run, they are all very...not smart
  2. The game starts, and every frame of the game, the brain of the bird recieves as input some data that is taken from the current frame ( y pos of the bird, distance from pipe...)
  3. The brain ( neural network ) of each bird, performs a feed forward with that data, and outputs what at the beginning is a very random result, let's say 0.75 for one bird
  4. At this point you decided that 0.75 is greater than 0.5, so you take that as a 1, which stands for "jump", while if it was 0.3, so 0, the bird does nothing and keeps falling
  5. Shortly the bird will die cause he has no idea of what he is doing, so he most likely collides with a pipe or the ground.
  6. After all birds met their fate, you see that some birds reached further than others, so you choose, for example, 5 of the best performing ones.
  7. Now you try to create a new generation of 200 birds using only the brains of those 5 that were choosen, by mixing and modifying theyr brains
  8. Now the new birds have a brand new brain, that in some cases might be better than the previous one, so chances are that some of those birds will reach a higher score, therefore flap further into the level.
  9. Repeat from point 6
  
  In practice your "perform\_genetic\_algorithm" function in python, will have to choose the birds with the highest score, and as wild as it sounds, mix their brains and modify them, hoping that some modifications will improve the performance of the bird.
  
  > I can't think of output since you don't really know if the action of flapping will benefit you or not
  
  The mechanism above explains why you basically do not care at all about the output, except saying to the game engine: "hey the bird decided to flap, do it". Whether it's the right action or not, doesn't matter, as the smarter birds are **naturally** gonna get further and so be more likely to be choosen for next generation.
  
  Hopefully now it's all more clear.
    
  Here is some useful links for building a neural network and for understanding the genetic algorithm:
  
  * [How to build a neural network](http://neuralnetworksanddeeplearning.com/chap1.html): I am linking this because it contains all useful information about how to build a very basic neural network in python. In your case, you would have to ignore all the part about backpropagation, loss & error calculation and SGD, and just look at the **feed forward** part.
  * [How to build a neural network - 2](https://enlight.nyc/projects/neural-network/): This is another example of building a neural network that i found really useful, probably it's simpler and more straight forward than the previous link, but again, the backpropagation part is not needed for this genetic based learning.
  * [Video tutorials on genetic algorithm](https://www.youtube.com/watch?v=9zfeTw-uFCw&list=PLRqwX-V7Uu6bJM3VgzjNV5YxVxUwzALHV): This is a very long but very explanatory playlist of videos that dives into the nature of genetic algorithms and how to implement one
  * [Genetic algorithm optimization](https://towardsdatascience.com/introduction-to-optimization-with-genetic-algorithm-2f5001d9964b): Other source about genetic algorithms
  
  
  ---
  
  ### Answer by username_2 on 2019-11-28. Score: 0
  
  I would recommend learning about Reinforcement learning first. You don't need a dataset as you train your network by letting it play the game over and over again. but knowing how to do so doea mean finding out about markov decision process and how you can use the neural network to solve this.
  
  
  ---
  
  ### Answer by username_3 on 2019-10-28. Score: 0
  
  I agree a neural net is a good start and you might want to add a constitutional neural network to list of models you want to test and evaluate.
  
  However your question was really on getting something up and running using python. I don't have enough time for me to play games, much more let my computers do it for me... but I know [someone who has](https://github.com/NJacobsohn/Your-Learning-is-in-Another-Castle#the-numerical-model) done such a project.
  
  Hopefully that helps.
  
  
  ---
  
  