Answers.Body: <p>As I can see, at each frame, you're giving 1 fitness if the snake gets closer to the food, and you remove 10 fitness if it gets away from the food. Am I right? Also, you're removing 50 fitness when a snake dies, and giving 15 fitness when it eats a food?</p>
  
  <p>Try to remove the -10 fitness at each frame where the snake gets away from food, it's not necessary. Not getting fitness is also a penalty, you don't need to exaggerate the penalty as it's making your -50 fitness being negligible. Because of that, dying quickly is better than surviving for 6 frames without getting closer to the food. So the first strategy your AIs are going to find is spinning around to die quickly (because surviving is bad for them). This is not what you want.</p>
  
  <p>But these are only assumptions, try it, and tell me if it works!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T03:59:22.983Z
Answers.LastActivityDate: 2020-04-04T03:59:22.983Z
Answers.UserDisplayName: Dara Kong
Body: <p>I am trying to implement NEAT for the snake game. My game logic is ready, which is working properly and NEAT configured. But even after 100 generations with 200 genomes per generation, the snakes perform very poorly. It barely ever eats more than 2 food. Below is the snip of the eval_genome function:</p>
  
  <pre><code>def eval_genome(genomes, config):
      clock = pygame.time.Clock()
      win = pygame.display.set_mode((WIN_WIDTH, WIN_HEIGHT))
      for genome_id, g in genomes:
          net = neat.nn.FeedForwardNetwork.create(g, config)
          g.fitness = 0
          snake = Snake()
          food = Food(snake.body)
          run = True
          UP = DOWN = RIGHT = LEFT = MOVE_SNAKE = False
          moveToFood = 0
          score = 0
          moveCount = 0
          while run:
              pygame.time.delay(50)
              clock.tick_busy_loop(10)
              for event in pygame.event.get():
                  if event.type == pygame.QUIT:
                      run = False
              snakeHeadX = snake.body[0]['x']
              snakeHeadY = snake.body[0]['y']
              snakeTailX = snake.body[len(snake.body)-1]['x']
              snakeTailY = snake.body[len(snake.body)-1]['y']
              snakeLength = len(snake.body)
              snakeHeadBottomDist = WIN_HEIGHT - snakeHeadY - STEP
              snakeHeadRightDist = WIN_WIDTH - snakeHeadX - STEP
              foodBottomDist = WIN_HEIGHT - food.y - STEP
              foodRightDist = WIN_WIDTH - food.x - STEP
              snakeFoodDistEuclidean = math.sqrt((snakeHeadX - food.x)**2 + (snakeHeadY - food.y)**2)
              snakeFoodDistManhattan = abs(snakeHeadX - food.x) + abs(snakeHeadY - food.y)
              viewDirections = snake.checkDirections(food, UP, DOWN, LEFT, RIGHT)
              deltaFoodDist = snakeFoodDistEuclidean
  
              outputs = net.activate((snakeHeadX, snakeHeadY, snakeHeadBottomDist, snakeHeadRightDist, snakeTailX, snakeTailY, snakeLength, moveCount, moveToFood, food.x, food.y, foodBottomDist, foodRightDist, snakeFoodDistEuclidean, snakeFoodDistManhattan, viewDirections[0], viewDirections[1], viewDirections[2], viewDirections[3], viewDirections[4], viewDirections[5], viewDirections[6], viewDirections[7], deltaFoodDist))
  
              if (outputs[0] == max(outputs) and not DOWN):
                  snake.setDir(0,-1)
                  UP = True
                  LEFT = False
                  RIGHT = False
                  MOVE_SNAKE = True
              elif (outputs[1] == max(outputs) and not UP):
                  snake.setDir(0,1)
                  DOWN = True
                  LEFT = False
                  RIGHT = False
                  MOVE_SNAKE = True
              elif (outputs[2] == max(outputs) and not RIGHT):
                  snake.setDir(-1,0)
                  LEFT = True
                  UP = False
                  DOWN = False
                  MOVE_SNAKE = True
              elif (outputs[3] == max(outputs) and not LEFT):
                  snake.setDir(1,0)
                  RIGHT = True
                  UP = False
                  DOWN = False
                  MOVE_SNAKE = True
              elif (not MOVE_SNAKE):
                  if (outputs[0] == max(outputs)):
                      snake.setDir(0,-1)
                      UP = True
                      MOVE_SNAKE = True
                  elif (outputs[1] == max(outputs)):
                      snake.setDir(0,1)
                      DOWN = True
                      MOVE_SNAKE = True
                  elif (outputs[2] == max(outputs)):
                      snake.setDir(-1,0)
                      LEFT = True
                      MOVE_SNAKE = True
                  elif (outputs[3] == max(outputs)):
                      snake.setDir(1,0)
                      RIGHT = True
                      MOVE_SNAKE = True  
  
              win.fill((0, 0, 0))
              food.showFood(win)
              if(MOVE_SNAKE):
                  snake.update()
                  newSnakeHeadX = snake.body[0]['x']
                  newSnakeHeadY = snake.body[0]['y']
                  newFoodDist = math.sqrt((newSnakeHeadX - food.x)**2 + (newSnakeHeadY - food.y)**2)
                  deltaFoodDist = newFoodDist - snakeFoodDistEuclidean
                  moveCount += 1
                  if (newFoodDist &lt;= snakeFoodDistEuclidean):
                      g.fitness += 1
                  else:
                      g.fitness -= 10
              snake.show(win)
              if(snake.collision()):
                  if score != 0:
                      print('FINAL SCORE IS: '+ str(score))
                  g.fitness -= 50
                  break
  
              if(snake.eat(food,win)):
                  g.fitness += 15
                  score += 1
                  if score == 1 :
                      moveToFood = moveCount
                      # foodEatenMove = pygame.time.get_ticks()/1000
                  else:
                      moveToFood = moveCount - moveToFood
                  food.foodLocation(snake.body)
                  food.showFood(win)
  </code></pre>
  
  <p>Additionally, I am putting the definition of the checkDirections function. What it does is returns an array of size 8 corresponding to 8 directions where each value can be either 0 (not food or body), 1(food found but no body), 3(body found but no food), or 4(both body and food found).</p>
  
  <pre><code>def checkDirections(self, food, up, down, left, right):
          '''
          x+STEP, y-STEP
          x+STEP, y+STEP
          x-STEP, y-STEP
          x-STEP, y+STEP
          x+STEP, y
          x, y-STEP
          x, y+STEP
          x-STEP, y
          '''
          view = []
          x = self.xdir
          y = self.ydir
  
          view.append(self.check(x, y, STEP, -STEP, food.x, food.y))
          view.append(self.check(x, y, STEP, STEP, food.x, food.y))
          view.append(self.check(x, y, -STEP, -STEP, food.x, food.y))
          view.append(self.check(x, y, -STEP, STEP, food.x, food.y))
          view.append(self.check(x, y, STEP, 0, food.x, food.y))
          view.append(self.check(x, y, 0, -STEP, food.x, food.y))
          view.append(self.check(x, y, 0, STEP, food.x, food.y))
          view.append(self.check(x, y, -STEP, 0, food.x, food.y))
  
          if up == True:
              view[6] = -999
          elif down == True:
              view[5] = -999
          elif left == True:
              view[4] == -999
          elif right == True:
              view[7] == -999        
          return view
  
      def check(self, x, y, xIncrement, yIncrement, foodX, foodY):
          value = 0
          foodFound = False
          bodyFound = False
          while (x &gt;= 0 and x &lt;= WIN_WIDTH and y &gt;= 0 and y &lt;= WIN_HEIGHT):
              x += xIncrement
              y += yIncrement
              if (not foodFound):
                  if (foodX == x and foodY == y):
                      foodFound = True
              if (not bodyFound):
                  for i in range(1, len(self.body)):
                      if ((x == self.body[i]['x']) and (y == self.body[i]['y'])):
                          bodyFound = True
              if (not bodyFound and not foodFound):
                  value = 0
              elif (not bodyFound and foodFound):
                  value = 1
              elif (bodyFound and not foodFound):
                  value = 2
              else:
                  value = 3
          return value
  </code></pre>
  
  <p>I am using sigmoid as the activation function. Although I have tried with tanh and relu as well with no luck. Below is the NEAT config file that I am using:</p>
  
  <pre><code>[NEAT]
  fitness_criterion     = max
  fitness_threshold     = 10000
  pop_size              = 200
  reset_on_extinction   = False
  
  [DefaultGenome]
  # node activation options
  activation_default      = sigmoid
  activation_mutate_rate  = 0.0
  activation_options      = sigmoid
  
  # node aggregation options
  aggregation_default     = sum
  aggregation_mutate_rate = 0.0
  aggregation_options     = sum
  
  # node bias options
  bias_init_mean          = 0.0
  bias_init_stdev         = 1.0
  # was 30 max and -30 for min bias
  bias_max_value          = 100.0
  bias_min_value          = -100.0
  bias_mutate_power       = 0.5
  bias_mutate_rate        = 0.7
  bias_replace_rate       = 0.3
  
  # genome compatibility options
  compatibility_disjoint_coefficient = 1.0
  compatibility_weight_coefficient   = 0.5
  
  # connection add/remove rates
  conn_add_prob           = 0.8
  conn_delete_prob        = 0.56
  
  # connection enable options
  enabled_default         = True
  # below was 0.01
  enabled_mutate_rate     = 0.3
  
  feed_forward            = True
  initial_connection      = full
  
  # node add/remove rates
  node_add_prob           = 0.7
  node_delete_prob        = 0.4
  
  # network parameters
  num_hidden              = 0
  num_inputs              = 24
  num_outputs             = 4
  
  # node response options
  response_init_mean      = 1.0
  response_init_stdev     = 0.0
  response_max_value      = 30.0
  response_min_value      = -30.0
  response_mutate_power   = 0.0
  response_mutate_rate    = 0.0
  response_replace_rate   = 0.0
  
  # connection weight options
  weight_init_mean        = 0.0
  weight_init_stdev       = 1.0
  weight_max_value        = 30
  weight_min_value        = -30
  weight_mutate_power     = 0.5
  weight_mutate_rate      = 0.8
  weight_replace_rate     = 0.1
  
  [DefaultSpeciesSet]
  compatibility_threshold = 3.0
  
  [DefaultStagnation]
  species_fitness_func = max
  max_stagnation       = 20
  species_elitism      = 2
  
  [DefaultReproduction]
  elitism            = 2
  survival_threshold = 0.2
  </code></pre>
  
  <p>If anyone has any insights or thoughts that could help improve the performance of the snake AI, please let me know.</p>
  
Tags: <neural-networks><game-ai><genetic-algorithms><neat><neuroevolution>
Title: Unable to achieve expected outputs using NEAT for the snake game
UserDisplayName: Ayan Chowdhury
ThreadText: # Unable to achieve expected outputs using NEAT for the snake game
  
  (Asked by: username_0 on 2019-10-24)
  
  I am trying to implement NEAT for the snake game. My game logic is ready, which is working properly and NEAT configured. But even after 100 generations with 200 genomes per generation, the snakes perform very poorly. It barely ever eats more than 2 food. Below is the snip of the eval\_genome function:
  
  ```
  def eval_genome(genomes, config):
      clock = pygame.time.Clock()
      win = pygame.display.set_mode((WIN_WIDTH, WIN_HEIGHT))
      for genome_id, g in genomes:
          net = neat.nn.FeedForwardNetwork.create(g, config)
          g.fitness = 0
          snake = Snake()
          food = Food(snake.body)
          run = True
          UP = DOWN = RIGHT = LEFT = MOVE_SNAKE = False
          moveToFood = 0
          score = 0
          moveCount = 0
          while run:
              pygame.time.delay(50)
              clock.tick_busy_loop(10)
              for event in pygame.event.get():
                  if event.type == pygame.QUIT:
                      run = False
              snakeHeadX = snake.body[0]['x']
              snakeHeadY = snake.body[0]['y']
              snakeTailX = snake.body[len(snake.body)-1]['x']
              snakeTailY = snake.body[len(snake.body)-1]['y']
              snakeLength = len(snake.body)
              snakeHeadBottomDist = WIN_HEIGHT - snakeHeadY - STEP
              snakeHeadRightDist = WIN_WIDTH - snakeHeadX - STEP
              foodBottomDist = WIN_HEIGHT - food.y - STEP
              foodRightDist = WIN_WIDTH - food.x - STEP
              snakeFoodDistEuclidean = math.sqrt((snakeHeadX - food.x)**2 + (snakeHeadY - food.y)**2)
              snakeFoodDistManhattan = abs(snakeHeadX - food.x) + abs(snakeHeadY - food.y)
              viewDirections = snake.checkDirections(food, UP, DOWN, LEFT, RIGHT)
              deltaFoodDist = snakeFoodDistEuclidean
  
              outputs = net.activate((snakeHeadX, snakeHeadY, snakeHeadBottomDist, snakeHeadRightDist, snakeTailX, snakeTailY, snakeLength, moveCount, moveToFood, food.x, food.y, foodBottomDist, foodRightDist, snakeFoodDistEuclidean, snakeFoodDistManhattan, viewDirections[0], viewDirections[1], viewDirections[2], viewDirections[3], viewDirections[4], viewDirections[5], viewDirections[6], viewDirections[7], deltaFoodDist))
  
              if (outputs[0] == max(outputs) and not DOWN):
                  snake.setDir(0,-1)
                  UP = True
                  LEFT = False
                  RIGHT = False
                  MOVE_SNAKE = True
              elif (outputs[1] == max(outputs) and not UP):
                  snake.setDir(0,1)
                  DOWN = True
                  LEFT = False
                  RIGHT = False
                  MOVE_SNAKE = True
              elif (outputs[2] == max(outputs) and not RIGHT):
                  snake.setDir(-1,0)
                  LEFT = True
                  UP = False
                  DOWN = False
                  MOVE_SNAKE = True
              elif (outputs[3] == max(outputs) and not LEFT):
                  snake.setDir(1,0)
                  RIGHT = True
                  UP = False
                  DOWN = False
                  MOVE_SNAKE = True
              elif (not MOVE_SNAKE):
                  if (outputs[0] == max(outputs)):
                      snake.setDir(0,-1)
                      UP = True
                      MOVE_SNAKE = True
                  elif (outputs[1] == max(outputs)):
                      snake.setDir(0,1)
                      DOWN = True
                      MOVE_SNAKE = True
                  elif (outputs[2] == max(outputs)):
                      snake.setDir(-1,0)
                      LEFT = True
                      MOVE_SNAKE = True
                  elif (outputs[3] == max(outputs)):
                      snake.setDir(1,0)
                      RIGHT = True
                      MOVE_SNAKE = True  
  
              win.fill((0, 0, 0))
              food.showFood(win)
              if(MOVE_SNAKE):
                  snake.update()
                  newSnakeHeadX = snake.body[0]['x']
                  newSnakeHeadY = snake.body[0]['y']
                  newFoodDist = math.sqrt((newSnakeHeadX - food.x)**2 + (newSnakeHeadY - food.y)**2)
                  deltaFoodDist = newFoodDist - snakeFoodDistEuclidean
                  moveCount += 1
                  if (newFoodDist <= snakeFoodDistEuclidean):
                      g.fitness += 1
                  else:
                      g.fitness -= 10
              snake.show(win)
              if(snake.collision()):
                  if score != 0:
                      print('FINAL SCORE IS: '+ str(score))
                  g.fitness -= 50
                  break
  
              if(snake.eat(food,win)):
                  g.fitness += 15
                  score += 1
                  if score == 1 :
                      moveToFood = moveCount
                      # foodEatenMove = pygame.time.get_ticks()/1000
                  else:
                      moveToFood = moveCount - moveToFood
                  food.foodLocation(snake.body)
                  food.showFood(win)
  
  ```
  
  Additionally, I am putting the definition of the checkDirections function. What it does is returns an array of size 8 corresponding to 8 directions where each value can be either 0 (not food or body), 1(food found but no body), 3(body found but no food), or 4(both body and food found).
  
  ```
  def checkDirections(self, food, up, down, left, right):
          '''
          x+STEP, y-STEP
          x+STEP, y+STEP
          x-STEP, y-STEP
          x-STEP, y+STEP
          x+STEP, y
          x, y-STEP
          x, y+STEP
          x-STEP, y
          '''
          view = []
          x = self.xdir
          y = self.ydir
  
          view.append(self.check(x, y, STEP, -STEP, food.x, food.y))
          view.append(self.check(x, y, STEP, STEP, food.x, food.y))
          view.append(self.check(x, y, -STEP, -STEP, food.x, food.y))
          view.append(self.check(x, y, -STEP, STEP, food.x, food.y))
          view.append(self.check(x, y, STEP, 0, food.x, food.y))
          view.append(self.check(x, y, 0, -STEP, food.x, food.y))
          view.append(self.check(x, y, 0, STEP, food.x, food.y))
          view.append(self.check(x, y, -STEP, 0, food.x, food.y))
  
          if up == True:
              view[6] = -999
          elif down == True:
              view[5] = -999
          elif left == True:
              view[4] == -999
          elif right == True:
              view[7] == -999        
          return view
  
      def check(self, x, y, xIncrement, yIncrement, foodX, foodY):
          value = 0
          foodFound = False
          bodyFound = False
          while (x >= 0 and x <= WIN_WIDTH and y >= 0 and y <= WIN_HEIGHT):
              x += xIncrement
              y += yIncrement
              if (not foodFound):
                  if (foodX == x and foodY == y):
                      foodFound = True
              if (not bodyFound):
                  for i in range(1, len(self.body)):
                      if ((x == self.body[i]['x']) and (y == self.body[i]['y'])):
                          bodyFound = True
              if (not bodyFound and not foodFound):
                  value = 0
              elif (not bodyFound and foodFound):
                  value = 1
              elif (bodyFound and not foodFound):
                  value = 2
              else:
                  value = 3
          return value
  
  ```
  
  I am using sigmoid as the activation function. Although I have tried with tanh and relu as well with no luck. Below is the NEAT config file that I am using:
  
  ```
  [NEAT]
  fitness_criterion     = max
  fitness_threshold     = 10000
  pop_size              = 200
  reset_on_extinction   = False
  
  [DefaultGenome]
  # node activation options
  activation_default      = sigmoid
  activation_mutate_rate  = 0.0
  activation_options      = sigmoid
  
  # node aggregation options
  aggregation_default     = sum
  aggregation_mutate_rate = 0.0
  aggregation_options     = sum
  
  # node bias options
  bias_init_mean          = 0.0
  bias_init_stdev         = 1.0
  # was 30 max and -30 for min bias
  bias_max_value          = 100.0
  bias_min_value          = -100.0
  bias_mutate_power       = 0.5
  bias_mutate_rate        = 0.7
  bias_replace_rate       = 0.3
  
  # genome compatibility options
  compatibility_disjoint_coefficient = 1.0
  compatibility_weight_coefficient   = 0.5
  
  # connection add/remove rates
  conn_add_prob           = 0.8
  conn_delete_prob        = 0.56
  
  # connection enable options
  enabled_default         = True
  # below was 0.01
  enabled_mutate_rate     = 0.3
  
  feed_forward            = True
  initial_connection      = full
  
  # node add/remove rates
  node_add_prob           = 0.7
  node_delete_prob        = 0.4
  
  # network parameters
  num_hidden              = 0
  num_inputs              = 24
  num_outputs             = 4
  
  # node response options
  response_init_mean      = 1.0
  response_init_stdev     = 0.0
  response_max_value      = 30.0
  response_min_value      = -30.0
  response_mutate_power   = 0.0
  response_mutate_rate    = 0.0
  response_replace_rate   = 0.0
  
  # connection weight options
  weight_init_mean        = 0.0
  weight_init_stdev       = 1.0
  weight_max_value        = 30
  weight_min_value        = -30
  weight_mutate_power     = 0.5
  weight_mutate_rate      = 0.8
  weight_replace_rate     = 0.1
  
  [DefaultSpeciesSet]
  compatibility_threshold = 3.0
  
  [DefaultStagnation]
  species_fitness_func = max
  max_stagnation       = 20
  species_elitism      = 2
  
  [DefaultReproduction]
  elitism            = 2
  survival_threshold = 0.2
  
  ```
  
  If anyone has any insights or thoughts that could help improve the performance of the snake AI, please let me know.
  
  **Tags:** neural-networks, game-ai, genetic-algorithms, neat, neuroevolution
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-04. Score: 1
  
  As I can see, at each frame, you're giving 1 fitness if the snake gets closer to the food, and you remove 10 fitness if it gets away from the food. Am I right? Also, you're removing 50 fitness when a snake dies, and giving 15 fitness when it eats a food?
  
  Try to remove the -10 fitness at each frame where the snake gets away from food, it's not necessary. Not getting fitness is also a penalty, you don't need to exaggerate the penalty as it's making your -50 fitness being negligible. Because of that, dying quickly is better than surviving for 6 frames without getting closer to the food. So the first strategy your AIs are going to find is spinning around to die quickly (because surviving is bad for them). This is not what you want.
  
  But these are only assumptions, try it, and tell me if it works!
  
  
  ---
  
  