Body: <p>I think reinforcement learning would be a good fit for this problem, but I am not sure of how to deal with a seemingly infinite number of actions. In the beginning of each game (generic RTS game), the player places down units anywhere on the map. Then as the game progresses, the player can move units around by selecting on them and clicking on a valid location on the map. They must take into consideration things like distance and travel time. An AI agent must do the same. </p>
  
  <p>How would I represent these actions? It’s not as simple as selecting ‘up’, ‘down’, ‘right’,...etc. Should the agent just randomly pick locations on the map? </p>
  
  <p>Are there any papers or implementations I can look at to help me get started?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-21T18:56:16.920Z
Comments.Text: Randomly selecting positions on the map will be problematic with RTS because of the combinatorial explosion--although the space is not formally continuous, it is effectively continuous in terms of game tree tractability.  (Have you thought about trying to abstract a little by reducing the map to a still-intractable-but-more-manageable number of cells or regions?
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-21T19:29:44.203Z
Comments.Text: (how much computing power do you have?)
Comments.UserDisplayName: Guillermo Mosse
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-21T19:57:57.327Z
Comments.Text: You should probably check out https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/
Comments.UserDisplayName: Jaden Travnik
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-21T20:42:39.793Z
Comments.Text: you could have an entire 2d map represented as a coordinate system and then actions would be coordinates of where you want to send the units on the map. This is just an idea, didn't try it and don't know if it would work.
Comments.UserDisplayName: Brale
Tags: <reinforcement-learning><algorithm><game-ai>
Title: Action spaces for an RTS game
UserDisplayName: Quaxton Hale
ThreadText: # Action spaces for an RTS game
  
  (Asked by: username_0 on 2019-05-21)
  
  I think reinforcement learning would be a good fit for this problem, but I am not sure of how to deal with a seemingly infinite number of actions. In the beginning of each game (generic RTS game), the player places down units anywhere on the map. Then as the game progresses, the player can move units around by selecting on them and clicking on a valid location on the map. They must take into consideration things like distance and travel time. An AI agent must do the same.
  
  How would I represent these actions? It’s not as simple as selecting ‘up’, ‘down’, ‘right’,...etc. Should the agent just randomly pick locations on the map?
  
  Are there any papers or implementations I can look at to help me get started?
  
  **Tags:** reinforcement-learning, algorithm, game-ai
  
  > **Comments:**
  > **username_1**: Randomly selecting positions on the map will be problematic with RTS because of the combinatorial explosion--although the space is not formally continuous, it is effectively continuous in terms of game tree tractability.  (Have you thought about trying to abstract a little by reducing the map to a still-intractable-but-more-manageable number of cells or regions?  
  > **username_2**: (how much computing power do you have?)  
  > **username_3**: You should probably check out https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/  
  > **username_4**: you could have an entire 2d map represented as a coordinate system and then actions would be coordinates of where you want to send the units on the map. This is just an idea, didn't try it and don't know if it would work.  
  
  ---
  
  ## 0 answers
  
  