Answers.Body: <p>You're correct that it is related to the fitness function, but only indirectly.  Recall that a function is the mathematical embodiment of a conceptual relationship.  The model is a motion model.  Proximity is not simply applying the distance formula <span class="math-container">$D = \sqrt{(x_2 - x_2)^2 + (y_2 - y_2)^2}$</span>.</p>
  
  <p>Proximity as we "feel it" when driving a car is based on the risks we see in the trajectories of ourselves and the other moving objects along with the stationary objects.  Mathematically, this is an integral over time of a distribution of probable locations.</p>
  
  <p>Look for work in the aeronautics industry about collision avoidance.  The mathematics and algorithms are well developed for antiaircraft weaponry (where the fitness function is the inverse of the one you want) and air traffic control, where the fitness is like yours except with an altitude dimension.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-19T14:52:41.953Z
Answers.LastActivityDate: 2019-06-26T12:24:42.413Z
Answers.LastEditDate: 2019-06-26T12:24:42.413Z
Answers.UserDisplayName: Douglas Daseeco
Body: <p>I have been working for ages on a neuro-evolution AI program, where cars learn how to race around a track.  Presently, I have a rudimentary fitness function that awards points for every degree traveled in the CW direction about the <strong>center of the window</strong> (removes points in CCW dir) and removes a certain amount of points for every collision that occurs. Cars also lose points for every moment they stay still and/or are colliding with something. My end goal is to create cars that can complete a track full of obstacles, faster than a human controlled car.</p>
  
  <p><em>Is there a better fitness function that would result in more efficient cars that</em> </p>
  
  <ol>
  <li>make better use of their sensors,</li>
  <li>are efficient in getting around the track (don't weave like a drunk driver), and </li>
  <li>are faster in general (cars are too cautious, but this is a race!)</li>
  </ol>
  
  <p>?</p>
  
  <p>Half of the population seems to just spin around.</p>
  
  <p>I have fully implemented my neuro-evolution program. Here's <a href="https://youtu.be/hnrQV0nKJxU" rel="nofollow noreferrer">my implementation</a>. However, you'll find that it isn't perfect. </p>
  
  <p><em>How should I alter my fitness function to generate better driving cars?</em>  </p>
  
  <p>Currently, the cars will only turn left or right if the outer sensors are activated.  However, if the sensors directly in front are activated, the neural network is configured in such a way that "ignores" this signal. So, the cars crash head-on into obstacles but avoid obstacles in their periphery.  I think this is due to the fact that the fitness function (which gives points for the displacement in 3 second time period) is too generous and removes the incentive for cars to avoid obstacles.  I've tried altering the punishment for collisions and the reward for driving in the right direction but it still isn't performing the way I would like it to.</p>
  
  <p>Here are some screenshots</p>
  
  <p><a href="https://i.sstatic.net/22rM5.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/22rM5.png" alt="Population of Cars in Racing Ring"></a>
  <a href="https://i.sstatic.net/ER7uW.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ER7uW.png" alt="Revive mode (revisiting past generations of cars"></a></p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-10-07T19:03:20.900Z
Comments.Text: It would be nice to know how you exactly determine that the car travelled (rotated) 1 degree. Does the car get positive reward every time when it goes around its own little circle?
Comments.UserDisplayName: Molnár István
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-10-11T17:21:56.280Z
Comments.Text: It seems like you might want to add additional layering for score instead of just the immediate I moved one degree I get a point. Perhaps a moving window that calculates total degrees moved over the last TBD-seconds. I would probably give that more weight than near-term score. I would think that should help prevent weaving and circling. Which might beg the question of why keep a near-term score at all but I would keep the near-term score because I suspect it will help get around obstacles.
Comments.UserDisplayName: Dunk
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-10-11T17:25:55.887Z
Comments.Text: You also might need to take into account 'obstacles' movement. That includes speed and direction. Perhaps your cars are circling because they are trying to avoid other cars. However, in a real race, cars race side-by-side because they have to assume the other drivers are going to follow certain 'driving rules'. Thus, they don't need to be concerned about hitting cars driving next to them. Their biggest concern is not hitting the car in front of them.
Comments.UserDisplayName: Dunk
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-22T19:28:13.427Z
Comments.Text: You can set a series of checkpoints - see this excellent article by Byte Master on codeproject.com: https://www.codeproject.com/Articles/1220276/ReInventing-Neural-Networks
Comments.UserDisplayName: Mike AI
Tags: <neural-networks><machine-learning><genetic-algorithms><autonomous-vehicles><fitness-functions>
Title: What is an appropriate fitness function for a simulated self-driving car?
UserDisplayName: Aniekan Umoren
ThreadText: # What is an appropriate fitness function for a simulated self-driving car?
  
  (Asked by: username_0 on 2017-10-06)
  
  I have been working for ages on a neuro-evolution AI program, where cars learn how to race around a track. Presently, I have a rudimentary fitness function that awards points for every degree traveled in the CW direction about the **center of the window** (removes points in CCW dir) and removes a certain amount of points for every collision that occurs. Cars also lose points for every moment they stay still and/or are colliding with something. My end goal is to create cars that can complete a track full of obstacles, faster than a human controlled car.
  
  *Is there a better fitness function that would result in more efficient cars that*
  
  1. make better use of their sensors,
  2. are efficient in getting around the track (don't weave like a drunk driver), and
  3. are faster in general (cars are too cautious, but this is a race!)
  
  ?
  
  Half of the population seems to just spin around.
  
  I have fully implemented my neuro-evolution program. Here's [my implementation](https://youtu.be/hnrQV0nKJxU). However, you'll find that it isn't perfect.
  
  *How should I alter my fitness function to generate better driving cars?*
  
  Currently, the cars will only turn left or right if the outer sensors are activated. However, if the sensors directly in front are activated, the neural network is configured in such a way that "ignores" this signal. So, the cars crash head-on into obstacles but avoid obstacles in their periphery. I think this is due to the fact that the fitness function (which gives points for the displacement in 3 second time period) is too generous and removes the incentive for cars to avoid obstacles. I've tried altering the punishment for collisions and the reward for driving in the right direction but it still isn't performing the way I would like it to.
  
  Here are some screenshots
  
  [![Population of Cars in Racing Ring](https://i.sstatic.net/22rM5.png)](https://i.sstatic.net/22rM5.png)
  [![Revive mode (revisiting past generations of cars](https://i.sstatic.net/ER7uW.png)](https://i.sstatic.net/ER7uW.png)
  
  **Tags:** neural-networks, machine-learning, genetic-algorithms, autonomous-vehicles, fitness-functions
  
  > **Comments:**
  > **username_1**: It would be nice to know how you exactly determine that the car travelled (rotated) 1 degree. Does the car get positive reward every time when it goes around its own little circle?  
  > **username_2**: It seems like you might want to add additional layering for score instead of just the immediate I moved one degree I get a point. Perhaps a moving window that calculates total degrees moved over the last TBD-seconds. I would probably give that more weight than near-term score. I would think that should help prevent weaving and circling. Which might beg the question of why keep a near-term score at all but I would keep the near-term score because I suspect it will help get around obstacles.  
  > **username_2**: You also might need to take into account 'obstacles' movement. That includes speed and direction. Perhaps your cars are circling because they are trying to avoid other cars. However, in a real race, cars race side-by-side because they have to assume the other drivers are going to follow certain 'driving rules'. Thus, they don't need to be concerned about hitting cars driving next to them. Their biggest concern is not hitting the car in front of them.  
  > **username_3**: You can set a series of checkpoints - see this excellent article by Byte Master on codeproject.com: https://www.codeproject.com/Articles/1220276/ReInventing-Neural-Networks  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_4 on 2018-07-19. Score: 1
  
  You're correct that it is related to the fitness function, but only indirectly. Recall that a function is the mathematical embodiment of a conceptual relationship. The model is a motion model. Proximity is not simply applying the distance formula $D = \sqrt{(x\_2 - x\_2)^2 + (y\_2 - y\_2)^2}$.
  
  Proximity as we "feel it" when driving a car is based on the risks we see in the trajectories of ourselves and the other moving objects along with the stationary objects. Mathematically, this is an integral over time of a distribution of probable locations.
  
  Look for work in the aeronautics industry about collision avoidance. The mathematics and algorithms are well developed for antiaircraft weaponry (where the fitness function is the inverse of the one you want) and air traffic control, where the fitness is like yours except with an altitude dimension.
  
  
  ---
  
  