Body: <p>I'm trying to make a bot to the famous &quot;<a href="https://en.wikipedia.org/wiki/Icy_Tower" rel="nofollow noreferrer">Icy Tower</a>&quot; game.
  I rebuilt the game using pygame and I'm trying to build the bot using <a href="https://neat-python.readthedocs.io/en/latest/" rel="nofollow noreferrer">Python-NEAT</a>.</p>
  <p>Every generation a population of 70 characters tries to jump to the next platform and increase their fitness. right now the fitness is the number of platforms they jumped on, each platform gives +10.</p>
  <p><a href="https://i.sstatic.net/fYHPF.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/fYHPF.png" alt="example" /></a></p>
  <p><strong>The problem</strong> I'm facing is that the bot <strong>isn't learning</strong> good enough after 1000 generations the best score was around 200 (it can get to 200 even at the first few generations by mistake. 200 means 20 platforms which is not a lot).</p>
  <p>when I look at the characters jumping it looks like they just always jump and go left or right and not deliberately aiming to the next platform.</p>
  <p>I tried several input configurations to make the bot perform better. but nothing really helped.</p>
  <p><strong>these are the inputs I tried to mess around with</strong>:</p>
  <ul>
  <li>pos.x, pos.y</li>
  <li>velocity.x, velocity.y</li>
  <li>isOnPlatform (bool)</li>
  <li>[plat.x, plat.y, plat.width] (list of the 3-7 next platforms locations, also tried distance from character in x,y)</li>
  <li>[prev.x, prev.y] (2-6 previous character positions)</li>
  </ul>
  <p>I'm not so proficient with neuroevolution and I'm probably doing something wrong. glad if you could explain what's causing the bot to be so bad or what's not helping him to learn properly.</p>
  <p>Although I think that the fitness function and the inputs should be the only problem I'm attaching the python-NEAT config file.</p>
  <pre><code>[NEAT]
  fitness_criterion     = max
  fitness_threshold     = 10000
  pop_size              = 70
  reset_on_extinction   = False
  
  [DefaultGenome]
  # node activation options
  activation_default      = tanh
  activation_mutate_rate  = 0.0
  activation_options      = tanh
  
  # node aggregation options
  aggregation_default     = sum
  aggregation_mutate_rate = 0.0
  aggregation_options     = sum
  
  # node bias options
  bias_init_mean          = 0.0
  bias_init_stdev         = 1.0
  bias_max_value          = 30.0
  bias_min_value          = -30.0
  bias_mutate_power       = 0.5
  bias_mutate_rate        = 0.7
  bias_replace_rate       = 0.1
  
  # genome compatibility options
  compatibility_disjoint_coefficient = 1.0
  compatibility_weight_coefficient   = 0.5
  
  # connection add/remove rates
  conn_add_prob           = 0.5
  conn_delete_prob        = 0.5
  
  # connection enable options
  enabled_default         = True
  enabled_mutate_rate     = 0.01
  
  feed_forward            = True
  initial_connection      = full
  
  # node add/remove rates
  node_add_prob           = 0.2
  node_delete_prob        = 0.2
  
  # network parameters
  num_hidden              = 6
  num_inputs              = 11
  num_outputs             = 3
  
  # node response options
  response_init_mean      = 1.0
  response_init_stdev     = 0.0
  response_max_value      = 30.0
  response_min_value      = -30.0
  response_mutate_power   = 0.0
  response_mutate_rate    = 0.0
  response_replace_rate   = 0.0
  
  # connection weight options
  weight_init_mean        = 0.0
  weight_init_stdev       = 1.0
  weight_max_value        = 30
  weight_min_value        = -30
  weight_mutate_power     = 0.5
  weight_mutate_rate      = 0.8
  weight_replace_rate     = 0.1
  
  [DefaultSpeciesSet]
  compatibility_threshold = 3.0
  
  [DefaultStagnation]
  species_fitness_func = max
  max_stagnation       = 3
  species_elitism      = 2
  
  [DefaultReproduction]
  elitism            = 3
  survival_threshold = 0.2
  </code></pre>
  <ul>
  <li><p><em>Note:</em> the previous character positions are the position in the previous frame, and if the game runs at 60 fps the previous position is not that different from the current one...</p>
  </li>
  <li><p><em>Note2:</em> the game score is a bit more complex than just jumping on platforms, the bot should also be rewarded for combos that can make him jump higher. the combo system is already implemented but I first want to see the bot aiming to the next platform before he learns to jump combo.</p>
  </li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-15T08:03:27.613Z
Comments.Text: Is platforms' arrengment all the time the same or it changes every time, or maybe every couple of times?
Comments.UserDisplayName: GKozinski
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-15T20:08:56.143Z
Comments.Text: The platform location is random every time. but even if it wasn't, the bot can't memorize the level's layout. right?
Comments.UserDisplayName: Lidor shimoni
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-16T09:38:40.923Z
Comments.Text: You should try evolution with all the time constant arrangement of the platform. I don't know what are your parameters that you try to find, but if this is the list of movements that your character do in the game (step by step) then of course it will not work, because the fitness function of one of such movement list will be totally irrelevant on a different platform configuration. What means that the results from one generation will not be the same in the next one, therefore evolution can not take place.
Comments.UserDisplayName: GKozinski
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-16T11:55:32.457Z
Comments.Text: @Makintosz I'm not quite sure I understand you, I saw bots for "flappy bird" with random levels and it worked with NEAT. why it won't work on my game?
  why training on the same level layout should work? the bot cant memorizes as far as I know.
Comments.UserDisplayName: Lidor shimoni
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-16T16:14:27.113Z
Comments.Text: I think it depends on how you set it up, and what is that you feed to the network. Situation you described seems like randomness is the key here. What do you feed the netwrok with?
Comments.UserDisplayName: GKozinski
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-16T18:31:44.040Z
Comments.Text: @Makintosz  the inputs I tried are listed in the question...
Comments.UserDisplayName: Lidor shimoni
Tags: <neural-networks><python><evolutionary-algorithms><neat><neuroevolution>
Title: How to select good inputs and fitness function to achive good results with NEAT for Icy Tower bot
UserDisplayName: Lidor shimoni
ThreadText: # How to select good inputs and fitness function to achive good results with NEAT for Icy Tower bot
  
  (Asked by: username_0 on 2020-05-05)
  
  I'm trying to make a bot to the famous "Icy Tower" game.
  I rebuilt the game using pygame and I'm trying to build the bot using Python-NEAT.
  Every generation a population of 70 characters tries to jump to the next platform and increase their fitness. right now the fitness is the number of platforms they jumped on, each platform gives +10.
  
  The problem I'm facing is that the bot isn't learning good enough after 1000 generations the best score was around 200 (it can get to 200 even at the first few generations by mistake. 200 means 20 platforms which is not a lot).
  when I look at the characters jumping it looks like they just always jump and go left or right and not deliberately aiming to the next platform.
  I tried several input configurations to make the bot perform better. but nothing really helped.
  these are the inputs I tried to mess around with:
  
  pos.x, pos.y
  velocity.x, velocity.y
  isOnPlatform (bool)
  [plat.x, plat.y, plat.width] (list of the 3-7 next platforms locations, also tried distance from character in x,y)
  [prev.x, prev.y] (2-6 previous character positions)
  
  I'm not so proficient with neuroevolution and I'm probably doing something wrong. glad if you could explain what's causing the bot to be so bad or what's not helping him to learn properly.
  Although I think that the fitness function and the inputs should be the only problem I'm attaching the python-NEAT config file.
  [NEAT]
  fitness_criterion     = max
  fitness_threshold     = 10000
  pop_size              = 70
  reset_on_extinction   = False
  
  [DefaultGenome]
  # node activation options
  activation_default      = tanh
  activation_mutate_rate  = 0.0
  activation_options      = tanh
  
  # node aggregation options
  aggregation_default     = sum
  aggregation_mutate_rate = 0.0
  aggregation_options     = sum
  
  # node bias options
  bias_init_mean          = 0.0
  bias_init_stdev         = 1.0
  bias_max_value          = 30.0
  bias_min_value          = -30.0
  bias_mutate_power       = 0.5
  bias_mutate_rate        = 0.7
  bias_replace_rate       = 0.1
  
  # genome compatibility options
  compatibility_disjoint_coefficient = 1.0
  compatibility_weight_coefficient   = 0.5
  
  # connection add/remove rates
  conn_add_prob           = 0.5
  conn_delete_prob        = 0.5
  
  # connection enable options
  enabled_default         = True
  enabled_mutate_rate     = 0.01
  
  feed_forward            = True
  initial_connection      = full
  
  # node add/remove rates
  node_add_prob           = 0.2
  node_delete_prob        = 0.2
  
  # network parameters
  num_hidden              = 6
  num_inputs              = 11
  num_outputs             = 3
  
  # node response options
  response_init_mean      = 1.0
  response_init_stdev     = 0.0
  response_max_value      = 30.0
  response_min_value      = -30.0
  response_mutate_power   = 0.0
  response_mutate_rate    = 0.0
  response_replace_rate   = 0.0
  
  # connection weight options
  weight_init_mean        = 0.0
  weight_init_stdev       = 1.0
  weight_max_value        = 30
  weight_min_value        = -30
  weight_mutate_power     = 0.5
  weight_mutate_rate      = 0.8
  weight_replace_rate     = 0.1
  
  [DefaultSpeciesSet]
  compatibility_threshold = 3.0
  
  [DefaultStagnation]
  species_fitness_func = max
  max_stagnation       = 3
  species_elitism      = 2
  
  [DefaultReproduction]
  elitism            = 3
  survival_threshold = 0.2
  
  
  Note: the previous character positions are the position in the previous frame, and if the game runs at 60 fps the previous position is not that different from the current one...
  
  Note2: the game score is a bit more complex than just jumping on platforms, the bot should also be rewarded for combos that can make him jump higher. the combo system is already implemented but I first want to see the bot aiming to the next platform before he learns to jump combo.
  
  **Tags:** neural-networks, python, evolutionary-algorithms, neat, neuroevolution
  > **Comments:**
  > **username_1**: Is platforms' arrengment all the time the same or it changes every time, or maybe every couple of times?  
  > **username_0**: The platform location is random every time. but even if it wasn't, the bot can't memorize the level's layout. right?  
  > **username_1**: You should try evolution with all the time constant arrangement of the platform. I don't know what are your parameters that you try to find, but if this is the list of movements that your character do in the game (step by step) then of course it will not work, because the fitness function of one of such movement list will be totally irrelevant on a different platform configuration. What means that the results from one generation will not be the same in the next one, therefore evolution can not take place.  
  > **username_0**: @Makintosz I'm not quite sure I understand you, I saw bots for "flappy bird" with random levels and it worked with NEAT. why it won't work on my game? why training on the same level layout should work? the bot cant memorizes as far as I know.  
  > **username_1**: I think it depends on how you set it up, and what is that you feed to the network. Situation you described seems like randomness is the key here. What do you feed the netwrok with?  
  > **username_0**: @Makintosz  the inputs I tried are listed in the question...  
  
  ---
  
  ## 0 answers
  
  