Body: <p>I have developed an RPG in likeness to the features showcased in the Final Fantasy series; multiple character classes which utilise unique action sets, sequential turn-based combat, front/back row modifiers, item usage and a crisis mechanic which bears similarity to the limit break feature.</p>
  <p>The problem is that the greater portion of my project depends on the use of some means of Machine Learning to, in some manner, act as an actor in the game environment, however, I do not know of my options under the bare-bones environment of a command line game; I am more familiar with the use of pixel data and a neural network for action selection on a frame-by-frame basis.</p>
  <p>Could I use reinforcement learning to learn a policy for action selection under a custom environment or should I apply a machine learning algorithm to character data, find the example outlined below, determine the best action to use on a particular turn state?</p>
  <pre><code>+-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+
  | Level | CurrHP | CurrMP | AtkStat | MagStat | StatusList | TargetsInEnemyParty | IsInCrisis | TurnNumber   | BattleParams       |ActionOutput  | DamageOutput |
  +-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+
  |    65 |   6500 |    320 |      47 |      56 |          0 | Behemoth            |0           | 7            | None               |ThiefAttack   |4254          |
  |    92 |   8000 |    250 |      65 |      32 |          0 | Behemoth            |1           | 4            | None               |WarriorLimit  |6984          |
  +-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+
  </code></pre>
  <p>I would like to prioritise the ease of implementation of an algorithm over how optimal the potential algorithm could be, I just need a baseline to work towards. Many thanks.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-18T16:19:36.727Z
Comments.Text: Your third paragraph is confusing to me. Importantly, reinforcement learning (RL) is a type of machine learning, and your description of your alternative to RL "machine learning algorithm ... determine the best action" reads like it is RL too. Do you mean that you wiill personally decide what the best action is for a large number of cases, and use that as training data for a supervised learning algorithm?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-18T16:55:05.037Z
Comments.Text: @NeilSlater I understand why it's confusing, I didn't properly distinguish between actions permissible by a player in the game and actions for a learning agent as, from my knowledge, they would be indistinguishable.    I am planning to both play as an expert and use collected data from a few testers as training data a supervised learning algorithm; though I do understand that this could concur bias towards particular actions.
Comments.UserDisplayName: SeunOsiko
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-19T05:07:49.363Z
Comments.Text: @SeunOsiko How good does the AI need to be? You might benefit from training the agent against itself, or rather an agent that takes the role of a potential player, with all their actions. I think in this case though, your best bet is to learn a policy for action selection.
Comments.UserDisplayName: Recessive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-19T12:29:35.033Z
Comments.Text: @Recessive It's for an undergrad final thesis; I am more concerned about the process of developing the agent and analyzing the actions chosen on a particular game start, rather than the quality of outcome. How would I go about building an agent to learn a policy, could I use libraries such as TensorFlow or PyTorch considering the lack of visual data?
Comments.UserDisplayName: SeunOsiko
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-20T01:55:13.977Z
Comments.Text: @SeunOsiko I think you'd be best off using something like openai gym: https://gym.openai.com/ . Then, it's just a matter of how you input the data. You could actually represent this as an image if you wanted - just construct an image where each pixel is related to some action or game state. White if present or black otherwise. Keep in mind that images are ultimately arrays, so technically anything you do input can be an image if you wanted it to be.
Comments.UserDisplayName: Recessive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-20T17:47:23.870Z
Comments.Text: @Recessive I had considered using a custom env but porting the game to an env brings up more questions than answers. Whilst I think I understand my action space is variable based on the character's usable actions, how should I store them and how to I manage the human players needing manual input to make choices whilst the enemy has pre-determined logic and the agent choices being learnt. Can I use an observation space by using purely numerical data, should I just store the agents data or the data of all allied characters in the game?
Comments.UserDisplayName: SeunOsiko
Tags: <machine-learning><reinforcement-learning><game-ai><getting-started><gaming>
Title: What would be the good choice of algorithm to use for character action selection in an RPG, implemented in Python?
UserDisplayName: SeunOsiko
ThreadText: # What would be the good choice of algorithm to use for character action selection in an RPG, implemented in Python?
  
  (Asked by: username_0 on 2020-06-17)
  
  I have developed an RPG in likeness to the features showcased in the Final Fantasy series; multiple character classes which utilise unique action sets, sequential turn-based combat, front/back row modifiers, item usage and a crisis mechanic which bears similarity to the limit break feature.
  The problem is that the greater portion of my project depends on the use of some means of Machine Learning to, in some manner, act as an actor in the game environment, however, I do not know of my options under the bare-bones environment of a command line game; I am more familiar with the use of pixel data and a neural network for action selection on a frame-by-frame basis.
  Could I use reinforcement learning to learn a policy for action selection under a custom environment or should I apply a machine learning algorithm to character data, find the example outlined below, determine the best action to use on a particular turn state?
  +-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+
  | Level | CurrHP | CurrMP | AtkStat | MagStat | StatusList | TargetsInEnemyParty | IsInCrisis | TurnNumber   | BattleParams       |ActionOutput  | DamageOutput |
  +-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+
  |    65 |   6500 |    320 |      47 |      56 |          0 | Behemoth            |0           | 7            | None               |ThiefAttack   |4254          |
  |    92 |   8000 |    250 |      65 |      32 |          0 | Behemoth            |1           | 4            | None               |WarriorLimit  |6984          |
  +-------+--------+--------+---------+---------+------------+---------------------+------------+--------------+--------------------+--------------+--------------+
  
  I would like to prioritise the ease of implementation of an algorithm over how optimal the potential algorithm could be, I just need a baseline to work towards. Many thanks.
  
  **Tags:** machine-learning, reinforcement-learning, game-ai, getting-started, gaming
  > **Comments:**
  > **username_1**: Your third paragraph is confusing to me. Importantly, reinforcement learning (RL) is a type of machine learning, and your description of your alternative to RL "machine learning algorithm ... determine the best action" reads like it is RL too. Do you mean that you wiill personally decide what the best action is for a large number of cases, and use that as training data for a supervised learning algorithm?  
  > **username_0**: @NeilSlater I understand why it's confusing, I didn't properly distinguish between actions permissible by a player in the game and actions for a learning agent as, from my knowledge, they would be indistinguishable.    I am planning to both play as an expert and use collected data from a few testers as training data a supervised learning algorithm; though I do understand that this could concur bias towards particular actions.  
  > **username_2**: @SeunOsiko How good does the AI need to be? You might benefit from training the agent against itself, or rather an agent that takes the role of a potential player, with all their actions. I think in this case though, your best bet is to learn a policy for action selection.  
  > **username_0**: @Recessive It's for an undergrad final thesis; I am more concerned about the process of developing the agent and analyzing the actions chosen on a particular game start, rather than the quality of outcome. How would I go about building an agent to learn a policy, could I use libraries such as TensorFlow or PyTorch considering the lack of visual data?  
  > **username_2**: @SeunOsiko I think you'd be best off using something like openai gym: https://gym.openai.com/ . Then, it's just a matter of how you input the data. You could actually represent this as an image if you wanted - just construct an image where each pixel is related to some action or game state. White if present or black otherwise. Keep in mind that images are ultimately arrays, so technically anything you do input can be an image if you wanted it to be.  
  > **username_0**: @Recessive I had considered using a custom env but porting the game to an env brings up more questions than answers. Whilst I think I understand my action space is variable based on the character's usable actions, how should I store them and how to I manage the human players needing manual input to make choices whilst the enemy has pre-determined logic and the agent choices being learnt. Can I use an observation space by using purely numerical data, should I just store the agents data or the data of all allied characters in the game?  
  
  ---
  
  ## 0 answers
  
  