Answers.Body: <p>I would set up a list of goals for your bot. These could be 'maintain a minimum level of health', 'knock out human player', 'block way to location X', etc. This obviously depends on the domain of your MMO.</p>
  
  <p>Then you can use a planner to achieve these goals in the game. You define a set of actions with preconditions and effects, set the current goal, and the planner will work out a list of actions for the bot to achieve the goal. You can easily express your actions (and the domain) in <a href="https://en.wikipedia.org/wiki/Planning_Domain_Definition_Language" rel="noreferrer">PDDL</a>.</p>
  
  <p>Examples for actions would be 'move to location X', 'eat X', 'attack player X'. A precondition of 'attack player X' could be 'health(X) is low', and an effect could be 'health(X) is reduced by Y'. There are different ways of expressing these depending on the planner's capabilities.</p>
  
  <p>The beauty of this is that you don't actually have to explicitly code any behaviour. You describe the domain, and tell the bot what it should achieve, and what capabilities it has. The actual behaviour then emerges out of that description. If the bot only attacks a player if the player has lower health, then observing the player eat (and thus up their health) could result in the bot eating (to push its own health above the player's so that it can attack) &mdash; but you have not told the bot directly to do that.</p>
  
  <p>For a starting point, go to <a href="http://education.planning.domains/" rel="noreferrer">http://education.planning.domains/</a> for a list of resources.</p>
  
  <p>If you only have a few actions available, it might appear predictable to a human user, but with a variety of goals and actions, this will quickly become more complex and seem more 'intelligent'. </p>
  
  <p><strong>Update:</strong> Here is a link to a paper, <em><a href="http://alumni.media.mit.edu/~jorkin/GOAP_draft_AIWisdom2_2003.pdf" rel="noreferrer">Applying Goal-Oriented Action Planning to Games</em></a>, which describes how this can be applied in a game.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-05T11:18:23.773Z
Answers.LastActivityDate: 2020-03-05T15:11:02.993Z
Answers.LastEditDate: 2020-03-05T15:11:02.993Z
Answers.UserDisplayName: Oliver Mason
Answers.Body: <p>You can train your bot using reinforcement learning (in particular Q-Learning). </p>
  
  <blockquote>
    <p>The most important part of the RL is a reward function. If we want agent to do some thing speciﬁc, we must provide rewards to it in such a way that it will achieve our goals. It is thus very important that the reward function accurately indicates the exact behaviour</p>
  </blockquote>
  
  <p>So you can construct your own reward function that will satisfy your requirements. If the bot does something you want, you will reward it with higher score, otherwise you will punish it with negative reward.</p>
  
  <p>AlhpaGo and OpenAI teams used a similar technique to train their model which could then beat humans in games like <a href="https://en.wikipedia.org/wiki/AlphaGo" rel="nofollow noreferrer">Go</a>, <a href="https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii" rel="nofollow noreferrer">StarCraft 2</a> and <a href="https://openai.com/blog/openai-five/" rel="nofollow noreferrer">Dota 2</a></p>
  
  <p>Also, check out this <a href="https://classroom.udacity.com/courses/ud893-preview" rel="nofollow noreferrer">Deep Reinforcement Learning Free Preview</a> on udacity.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-05T11:01:43.183Z
Answers.LastActivityDate: 2020-03-05T11:27:52.133Z
Answers.LastEditDate: 2020-03-05T11:27:52.133Z
Answers.UserDisplayName: Aray Karjauv
Answers.Body: <p><a href="https://ai.stackexchange.com/a/18435/34038">Oliver Mason's answer</a> is great for specific methods and tools to use, but I wanted to pull out a more general principle which was mentioned in a comment.</p>
  
  <p>The distinction your friend is making is not one that would be generally recognised. One of my university lecturers defined AI as something like "an artificial system that exhibits behaviour that resembles how an intelligent being would behave". </p>
  
  <p>If an intelligent being would <em>always</em> use the special attack in a particular situation, then an algorithm that always does so in the same situation is behaving intelligently, even though the algorithm behind it is incredibly simple. If you can come up with a complete description of an intelligent player, you have what is called an <a href="https://en.wikipedia.org/wiki/Expert_system" rel="noreferrer">expert system</a>, i.e. a system which captures the decision-making process of a real expert.</p>
  
  <p>Your friend is not even correct that your proposed AI "does not have any heuristic functions". When you write a condition like "if the AI's health is below 50%, it will eat food", you're <em>approximating</em> the rule a human would use. You can make the heuristic <em>more complex</em> by increasing the probability of eating in proportion to current health; that might in turn make the heuristic <em>closer to optimal</em>.</p>
  
  <p>You can only really find out how "good" your AI is by putting it into different situations and observing it - sometimes, a simple set of rules gives rise to "emergent behaviours" that look surprisingly intelligent. As you build up more complex rules - i.e. more optimal heuristics - the emergent behaviour will change, and you can tweak it for the desired effect.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-05T17:50:47.393Z
Answers.Text: Yes, I agree. Presumably the issue here hinges on 'predictability', in the sense that a very predictable behaviour does not appear very 'intelligent', whereas a more complex and varied behaviour which is less predictable seems to be more intelligent (when it isn't really).
Answers.UserDisplayName: Oliver Mason
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-05T17:24:31.830Z
Answers.LastActivityDate: 2020-03-05T17:24:31.830Z
Answers.UserDisplayName: IMSoP
Answers.Body: <p>What about GANs or genetic algorithms?</p>
  
  <p>The first idea (GAN) is that you basically create 2+ random bots who fight each other, and they keep adjusting their weights so that they can beat the other bot. That means, that those 2+ bots keep improving their "fighting performance" for as long as you want, eventually being even better than humans.</p>
  
  <p>The second idea (genetic algorithms) is to generate a lot of bots who genetically differ from each other just for a slight mutation. You make them fight, and the best perforing/last standing will become the new clone where the new "lot of bots" gets generated from.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-19T18:33:44.063Z
Answers.LastActivityDate: 2020-03-19T18:33:44.063Z
Answers.UserDisplayName: Orly
Answers.Body: <p>If you are willing to take an evolutionary approach, you may employ the NEAT algorithm (Neural Evolution of Augmenting Topologies) to train your bot. It will take some work setting it up and all, but it then will gradually improve over time.</p>
  
  <p>Check out the following:</p>
  
  <ul>
  <li><a href="http://gekkoquant.com/2016/03/13/evolving-neural-networks-through-augmenting-topologies-part-1-of-4/" rel="nofollow noreferrer">http://gekkoquant.com/2016/03/13/evolving-neural-networks-through-augmenting-topologies-part-1-of-4/</a></li>
  <li><a href="https://www.youtube.com/watch?v=WiPZSieT6qs" rel="nofollow noreferrer">https://www.youtube.com/watch?v=WiPZSieT6qs</a></li>
  <li><a href="https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies</a></li>
  </ul>
  
  <p>That should be enough to pique your interests and get you started. That last link links to a number of NEAT implementations available in a number of languages.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-16T15:59:25.310Z
Answers.LastActivityDate: 2020-03-16T15:59:25.310Z
Answers.UserDisplayName: Lord Alveric
Body: <p>I have an MMO game where I have players. I wanted to invent something new to the game, and add player-bots to make the game be single-playable as well. The AI I want to add is simply only for fighting other players or other player-bots that he sees around at his level.</p>
  <p>So, I thought of implementing my fighting strategy, exactly how I play, to the bot which is basically using if statements and randoms. For example, when the opponent has low health, and the bot has enough special attack power, he will use this chance and use his special attack power in order to try to knock the opponent down, or if the bot has low health he will eat in time but not too much because there is a point in risking fights, if you eat too much your opponent will do too. Or, for example, if the bot detects the opponent player is eating too much and gains health, he will do the same.</p>
  <p>I told this idea of the implementation to one of my friends and he simply responded with: This is not AI, it's simply just a set of conditions, it does not have any heuristic functions.</p>
  <p><strong>For that type of game, what are some ideas to create a real AI to achieve these conditions?</strong></p>
  <p>Basically, the AI should know what to do in order to beat the opponent, based on the opponent's data such as current health, Armour and weapons, and level, if he risks his health or not and so on.</p>
  <p>I am a beginner and it really interests me to do it in the right way.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-05T10:43:18.060Z
Comments.Text: I would guess that your friend has not heard of *expert systems* https://en.wikipedia.org/wiki/Expert_system - a lot of video game AI is expert systems plus pathfinding algorithms. Whether or not someone calls it "real" AI is kind of secondary to is the game playing fun, but I am sure someone here can help you with some guidance. You might also consider searching on https://gamedev.stackexchange.com/ for advice
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-06T08:46:59.057Z
Comments.Text: An expert system is really old tech, and extremely tedious to code (code a pile of conditions, test, fix bugs, improve, test, fix bugs, improve, etc). If the OP is new, he might as well go straight into what Oliver mentioned instead of coding an expert system. There are enough open source AI systems out there that people really should go into it if they're interested.
Comments.UserDisplayName: Nelson
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-06T11:00:46.827Z
Comments.Text: What does "make the player be single-player" mean?
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-06T11:01:25.727Z
Comments.Text: Also, game programming people would call it an AI. Machine learning people would not call it an AI. Players don't care whether your game AI has machine learning or not, only how good it is.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-06T17:00:16.840Z
Comments.Text: @user253751 I meant that with proper bot AI you can make your game be playable as a singleplayer, no need for other players
Comments.UserDisplayName: Ben Beri
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-06T17:05:55.277Z
Comments.Text: But if I think about it, when I play they game, my mind also kinda performs a condition, like I personally will pull out my special weapon if he has a low health, or i will eat more food if he eats too to make me safe aswell as he does. but the thing is, my mind somehow knows that this is the right way or atleast thinks so because it helps me win fights if I do so. Thats what interests me aswell, how can I achieve that kind of a machine learning?
Comments.UserDisplayName: Ben Beri
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-15T09:54:54.427Z
Comments.Text: I think this question has potential for deeper answers, I started a bounty because this concept really interests me, to see what other answers I can get for this.
Comments.UserDisplayName: Ben Beri
Tags: <reference-request><game-ai><algorithm-request>
Title: What are examples of approaches to create an AI for a fighting robot in an MMO game?
UserDisplayName: Ben Beri
ThreadText: # What are examples of approaches to create an AI for a fighting robot in an MMO game?
  
  (Asked by: username_0 on 2020-03-05)
  
  I have an MMO game where I have players. I wanted to invent something new to the game, and add player-bots to make the game be single-playable as well. The AI I want to add is simply only for fighting other players or other player-bots that he sees around at his level.
  
  So, I thought of implementing my fighting strategy, exactly how I play, to the bot which is basically using if statements and randoms. For example, when the opponent has low health, and the bot has enough special attack power, he will use this chance and use his special attack power in order to try to knock the opponent down, or if the bot has low health he will eat in time but not too much because there is a point in risking fights, if you eat too much your opponent will do too. Or, for example, if the bot detects the opponent player is eating too much and gains health, he will do the same.
  
  I told this idea of the implementation to one of my friends and he simply responded with: This is not AI, it's simply just a set of conditions, it does not have any heuristic functions.
  
  **For that type of game, what are some ideas to create a real AI to achieve these conditions?**
  
  Basically, the AI should know what to do in order to beat the opponent, based on the opponent's data such as current health, Armour and weapons, and level, if he risks his health or not and so on.
  
  I am a beginner and it really interests me to do it in the right way.
  
  **Tags:** reference-request, game-ai, algorithm-request
  > **Comments:**
  > **username_1**: I would guess that your friend has not heard of *expert systems* https://en.wikipedia.org/wiki/Expert_system - a lot of video game AI is expert systems plus pathfinding algorithms. Whether or not someone calls it "real" AI is kind of secondary to is the game playing fun, but I am sure someone here can help you with some guidance. You might also consider searching on https://gamedev.stackexchange.com/ for advice  
  > **username_2**: An expert system is really old tech, and extremely tedious to code (code a pile of conditions, test, fix bugs, improve, test, fix bugs, improve, etc). If the OP is new, he might as well go straight into what Oliver mentioned instead of coding an expert system. There are enough open source AI systems out there that people really should go into it if they're interested.  
  > **username_3**: What does "make the player be single-player" mean?  
  > **username_3**: Also, game programming people would call it an AI. Machine learning people would not call it an AI. Players don't care whether your game AI has machine learning or not, only how good it is.  
  > **username_0**: @user253751 I meant that with proper bot AI you can make your game be playable as a singleplayer, no need for other players  
  > **username_0**: But if I think about it, when I play they game, my mind also kinda performs a condition, like I personally will pull out my special weapon if he has a low health, or i will eat more food if he eats too to make me safe aswell as he does. but the thing is, my mind somehow knows that this is the right way or atleast thinks so because it helps me win fights if I do so. Thats what interests me aswell, how can I achieve that kind of a machine learning?  
  > **username_0**: I think this question has potential for deeper answers, I started a bounty because this concept really interests me, to see what other answers I can get for this.  
  
  ---
  
  ## 5 answers
  
  ### Answer by username_4 on 2020-03-05. Score: 10
  
  I would set up a list of goals for your bot. These could be 'maintain a minimum level of health', 'knock out human player', 'block way to location X', etc. This obviously depends on the domain of your MMO.
  
  Then you can use a planner to achieve these goals in the game. You define a set of actions with preconditions and effects, set the current goal, and the planner will work out a list of actions for the bot to achieve the goal. You can easily express your actions (and the domain) in [PDDL](https://en.wikipedia.org/wiki/Planning_Domain_Definition_Language).
  
  Examples for actions would be 'move to location X', 'eat X', 'attack player X'. A precondition of 'attack player X' could be 'health(X) is low', and an effect could be 'health(X) is reduced by Y'. There are different ways of expressing these depending on the planner's capabilities.
  
  The beauty of this is that you don't actually have to explicitly code any behaviour. You describe the domain, and tell the bot what it should achieve, and what capabilities it has. The actual behaviour then emerges out of that description. If the bot only attacks a player if the player has lower health, then observing the player eat (and thus up their health) could result in the bot eating (to push its own health above the player's so that it can attack) — but you have not told the bot directly to do that.
  
  For a starting point, go to <http://education.planning.domains/> for a list of resources.
  
  If you only have a few actions available, it might appear predictable to a human user, but with a variety of goals and actions, this will quickly become more complex and seem more 'intelligent'.
  
  **Update:** Here is a link to a paper, *[Applying Goal-Oriented Action Planning to Games](http://alumni.media.mit.edu/~jorkin/GOAP_draft_AIWisdom2_2003.pdf)*, which describes how this can be applied in a game.
  
  
  ---
  
  ### Answer by username_6 on 2020-03-05. Score: 8
  
  [Oliver Mason's answer](https://ai.stackexchange.com/a/18435/34038) is great for specific methods and tools to use, but I wanted to pull out a more general principle which was mentioned in a comment.
  
  The distinction your friend is making is not one that would be generally recognised. One of my university lecturers defined AI as something like "an artificial system that exhibits behaviour that resembles how an intelligent being would behave".
  
  If an intelligent being would *always* use the special attack in a particular situation, then an algorithm that always does so in the same situation is behaving intelligently, even though the algorithm behind it is incredibly simple. If you can come up with a complete description of an intelligent player, you have what is called an [expert system](https://en.wikipedia.org/wiki/Expert_system), i.e. a system which captures the decision-making process of a real expert.
  
  Your friend is not even correct that your proposed AI "does not have any heuristic functions". When you write a condition like "if the AI's health is below 50%, it will eat food", you're *approximating* the rule a human would use. You can make the heuristic *more complex* by increasing the probability of eating in proportion to current health; that might in turn make the heuristic *closer to optimal*.
  
  You can only really find out how "good" your AI is by putting it into different situations and observing it - sometimes, a simple set of rules gives rise to "emergent behaviours" that look surprisingly intelligent. As you build up more complex rules - i.e. more optimal heuristics - the emergent behaviour will change, and you can tweak it for the desired effect.
  
  > **Comments:**
  > **username_4**: Yes, I agree. Presumably the issue here hinges on 'predictability', in the sense that a very predictable behaviour does not appear very 'intelligent', whereas a more complex and varied behaviour which is less predictable seems to be more intelligent (when it isn't really).  
  
  ---
  
  ### Answer by username_5 on 2020-03-05. Score: 3
  
  You can train your bot using reinforcement learning (in particular Q-Learning).
  
  > The most important part of the RL is a reward function. If we want agent to do some thing speciﬁc, we must provide rewards to it in such a way that it will achieve our goals. It is thus very important that the reward function accurately indicates the exact behaviour
  
  So you can construct your own reward function that will satisfy your requirements. If the bot does something you want, you will reward it with higher score, otherwise you will punish it with negative reward.
  
  AlhpaGo and OpenAI teams used a similar technique to train their model which could then beat humans in games like [Go](https://en.wikipedia.org/wiki/AlphaGo), [StarCraft 2](https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii) and [Dota 2](https://openai.com/blog/openai-five/)
  
  Also, check out this [Deep Reinforcement Learning Free Preview](https://classroom.udacity.com/courses/ud893-preview) on udacity.
  
  
  ---
  
  ### Answer by username_7 on 2020-03-19. Score: 0
  
  What about GANs or genetic algorithms?
  
  The first idea (GAN) is that you basically create 2+ random bots who fight each other, and they keep adjusting their weights so that they can beat the other bot. That means, that those 2+ bots keep improving their "fighting performance" for as long as you want, eventually being even better than humans.
  
  The second idea (genetic algorithms) is to generate a lot of bots who genetically differ from each other just for a slight mutation. You make them fight, and the best perforing/last standing will become the new clone where the new "lot of bots" gets generated from.
  
  
  ---
  
  ### Answer by username_8 on 2020-03-16. Score: 0
  
  If you are willing to take an evolutionary approach, you may employ the NEAT algorithm (Neural Evolution of Augmenting Topologies) to train your bot. It will take some work setting it up and all, but it then will gradually improve over time.
  
  Check out the following:
  
  * <http://gekkoquant.com/2016/03/13/evolving-neural-networks-through-augmenting-topologies-part-1-of-4/>
  * <https://www.youtube.com/watch?v=WiPZSieT6qs>
  * <https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies>
  
  That should be enough to pique your interests and get you started. That last link links to a number of NEAT implementations available in a number of languages.
  
  
  ---
  
  