Answers.Body: <p>Most of the so called 'AI' enemies in games nowadays are based on human logics and not machine learning, although they're called AIs.</p>
  <p>Mentioning about games, the first thing for ML is reinforcement learning. Let the bots play round to get rewards (positive) or punishment (negative rewards). These bots may finally know how to hide or move out a little bit to shoot.</p>
  <p>I can guess of some possible actions for bots in shooter games:<br/>
  Move U/D/L/R, jump, duck, find target (coordinates already in memory, valid only in view angle), shoot.</p>
  <p>Training bot in 3D environments is much harder than in 2D games, since every looking angle of the bot is another environment state, 360 degrees around and 360 degrees up/down. And enemies can be at any location, in or off the frame.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T07:47:27.980Z
Answers.Text: OP is not asking how to implement ML for games, but how existing game AI works.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T07:53:15.060Z
Answers.Text: yeah, i said it's all human logics for present games, no ML
Answers.UserDisplayName: Dan D.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T03:35:48.720Z
Answers.LastActivityDate: 2021-03-16T03:42:37.333Z
Answers.LastEditDate: 2021-03-16T03:42:37.333Z
Answers.UserDisplayName: Dan D.
Answers.Body: <h2>Overlap between AI and &quot;Game AI&quot;</h2>
  <p>Nowadays, if you search for AI online, you will find a lot of material about machine learning, natural language processing, intelligent agents and neural networks. These are not the whole of AI by any means, expecially in a historical context, but they have recently been very successful, there is lots of published material about them.</p>
  <p>Games, especially action games, tend not to use these new popular technologies because they have other priorities. However, under the broadest definitions, there is good overlap between AI in general and game AI, or more specifically <em>enemy AI</em> within a game:</p>
  <ul>
  <li><p>Computer-controlled opponents within a game are effectively autonomous agents operating within an environment.</p>
  </li>
  <li><p>The game's purpose is to entertain the player, and this usually translates into requiring challenging and believable behaviour for actions taken by opponents (for some value of &quot;believable&quot;).</p>
  </li>
  <li><p>A 3D map with a physics engine, other moving agents and possible hazards is a complex environment that needs some effort from a developer in order to define working behaviours.</p>
  </li>
  <li><p>Many games need to solve one or two classic AI problems such as <a href="https://arrow.tudublin.ie/cgi/viewcontent.cgi?article=1063&amp;context=itbj" rel="nofollow noreferrer">pathfinding</a>.</p>
  </li>
  </ul>
  <p>All the above means that the term &quot;Game AI&quot; is a good choice for describing how enemy agents are designed and implemented. However, there are large differences between goals of AI applied today in a 3D shooter and the goals of AI used in industry to control decision making.</p>
  <ul>
  <li><p>Action games have a low CPU budget available per enemy for decision-making. Control systems in industry can have dedicated machinery, sometimes multiple machines, just to run the AI.</p>
  </li>
  <li><p>Action game priority is the game experience of the player, and this translates into strict targeting of a definition of &quot;believable&quot; for enemy behaviour that will entertain. In contrast, AI used in industry has accuracy as a high priority.</p>
  </li>
  </ul>
  <h2>Typical game AI in 3D shooters</h2>
  <p>Developing enemy AI in games is a specialist skill, and only partly related to the purpose of this site. For detailed discussion you may want to look into <a href="https://gamedev.stackexchange.com/">Game Development Stack Exchange</a> which has many questions and answers on topics like <a href="https://gamedev.stackexchange.com/search?q=enemy%20AI">Enemy AI</a></p>
  <p>Very roughly, the key traits of enemy AI in a game <em>like</em> Call of Duty could look like this (I have no inside knowledge of how COD does this, so may be inaccurate in places):</p>
  <ul>
  <li><p>Enemies will have one or more modes of behaviour defined. This might be as simple as &quot;Idle&quot; and &quot;Attack Player&quot;, or there could be many. For COD I would suspect there are many, and some may be in a hierarchy - e.g. there may be several sub-types of &quot;Attack&quot; behaviour depending on the enemy design.</p>
  </li>
  <li><p>Within a mode of behaviour, there may be a few different components defined, only some of which use AI routines. For instance, there will be specific animations related to walking or standing which are not AI, but there will also be some pathfinding AI if the agent is attempting to move around.</p>
  </li>
  <li><p>Decisions to switch between different modes of behaviour are often scripted with simple triggers, such as detecting player visibility using ray-casting between the location of player and enemy. When game AI fails to produce realistic results, it is often these high level triggers being brittle and not covering edge cases that causes it. Depending on complexity of the game and number of behaviours, there may be an algorithm managing the transitions between them, such as a <a href="http://www.ai-junkie.com/architecture/state_driven/tut_state1.html" rel="nofollow noreferrer">finite state engine</a> or <a href="https://jahej.com/alt/2011_02_24_introduction-to-behavior-trees.html" rel="nofollow noreferrer">behaviour trees</a>.</p>
  </li>
  <li><p>Enemies will be presented with highly simplified observations of the game world from their perspective, in order to speed AI decisions.</p>
  </li>
  <li><p>AI systems will have CPU budget restricted. A 3D game spends significant resources rendering scenes, and will often try to render quickly, e.g. 100 times per second. There are multiple ways that AI budget can be allocated, but it is relatively common for AI calculations to be spread over muliple frames, and for search structures for tasks like path-finding to persist over time.</p>
  </li>
  <li><p>There will be a middle ground between scripted behaviour and AI-driven behaviour where analysis is done as part of game design. For instance, pathfinding routes may be pre-calculated to some degree. A system of <a href="http://jceipek.com/Olin-Coding-Tutorials/pathing.html#waypoints" rel="nofollow noreferrer">way points</a> is one example of this - it might be set by the game designer, it may be calculated by an AI component of the game asset-building pipeline, or it may be dynamically calculated and cached during a game session so that multiple enemy units can share it.</p>
  </li>
  <li><p>When complex AI is not needed to achieve a goal, when a simple caclulation or &quot;puppet-like&quot; behaviour would do just fine, then this could be chosen instead. For instance, an enemy &quot;aiming&quot; at a player can be a simple vector calculation, perhaps with a fudge factor of a miss chance depending on the range to make the enemy seem fallible and not as much like a machine.</p>
  </li>
  </ul>
  <p>I do not know how the behaviour of enemies using cover is implemented in COD. A simple variant would be to have each enemy hard-coded to use a specific piece of cover that usually works well against the player due to map design. However, it is definitley <em>possible</em> to have a search algorithm running (perhaps over multiple frames) that assesses nearby locations that the enemy could reach against the player's current position, and then pick those as variables to plug into the &quot;take cover and fire on player&quot; scripted behaviour. That assessment would use the same kind of visibilty detection between enemy and player that is used to trigger changes between &quot;Idle&quot; and &quot;Attack&quot; behaviours for the enemy.</p>
  <hr />
  <p>As an aside, one related thing I find interesting is in <a href="https://www.youtube.com/watch?v=Mnu1DzFzRWs" rel="nofollow noreferrer">using modern AI techniques to blend animations and make interactions between actors and the environment look more realistic</a>. Although it is a lower-level feature than the question about enemy behaviour you are asking, it is an interesting cross-over between robotics and game playing that we will likely see in next-generation games, and probably applied first to the more detailed player model animations. <a href="https://www.youtube.com/watch?v=cTqVhcrilrE" rel="nofollow noreferrer">Here is another example applied to a humanoid agent switching smoothly between different tasks</a>, which any game player will recognise as something which game engines cannot do well at the moment - there are usually many jarring transitions caused by events in the game.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T11:03:44.463Z
Answers.Text: This answer seems reasonable to me, but given that some people, like me, are not really familiar with the details of the implementation of these "AI systems" in games, it may be a good idea to provide a reference to games that use the approaches that you mention. For example, when you say "Decisions to switch between different modes of behaviour are often scripted with simple triggers", this seems reasonable for anyone (including me) that played any 3d shooter/game, but maybe you could provide a reference that shows this is true.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T15:06:17.810Z
Answers.Text: @nbro: I think the best reference in general will be the Game Dev Stack Exchange link. I will try to find answers in there that back up my assertions, but I have to re-state that I have no evidence about the specific outlined approach being accurate for COD. There is definitely more than one way to structure enemy behaviour logic in a game.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T15:26:34.337Z
Answers.Text: @nbro: I added links to some ways of managing enemy behaviour transitions in games, which at least demonstrates that these are real things discussed by professional game developers and not something I just made up to answer the question
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T15:52:20.417Z
Answers.Text: I have also added an aside to a related topic where modern AI and game development are overlapping and we should expect to see some results from it in near future with smoother movement and behaviour transitions in animations. So there is room for this kind of AI in games soon, although we're a long way off from each enemy in a 3D shooter having a decision-making neural network as standard.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T15:57:43.520Z
Answers.Text: Thanks! Now, I think your answer can be more useful for people like me that don't have a good knowledge of the topic. A few weeks ago, I also read a paper (I think [Game AI Revisited](https://dl.acm.org/doi/pdf/10.1145/2212908.2212954) by Yannakakis), which, if I remember correctly, is consistent with what you just said, so that's why I also upvoted your answer. You may be interested in that paper, which also talks about procedural content generation and stuff like that (if I remember correctly).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-16T09:52:09.253Z
Answers.LastActivityDate: 2021-03-16T17:23:52.413Z
Answers.LastEditDate: 2021-03-16T17:23:52.413Z
Answers.UserDisplayName: Neil Slater
Body: <p>A few days ago, I started looking a bit more into AI and learning about the way it works, and it is very interesting, but I can't find a clear answer on how the artificial intelligence is implemented in 3d shooter games, like COD or practically any 3d game.</p>
  <p>I just don't understand how they teach the enemies such different things based on the game to fit its narratives. For example, is the enemy &quot;AI&quot; in 3d games just a bunch of if-else statements, or do they actually teach the enemies to think strategically? In big AAA games, you can clearly see that enemies hide from you in shootings and peek to shoot not just rush and get killed.</p>
  <p>So, how is the AI in 3d games implemented? How do they code it? You don't need to explain in detail, but just give me the idea. Do they use algorithms?</p>
  
Tags: <reinforcement-learning><game-ai><implementation><gaming><algorithm-request>
Title: How is the AI in 3d games implemented?
UserDisplayName: Irkl1_
ThreadText: # How is the AI in 3d games implemented?
  
  (Asked by: username_0 on 2021-03-15)
  
  A few days ago, I started looking a bit more into AI and learning about the way it works, and it is very interesting, but I can't find a clear answer on how the artificial intelligence is implemented in 3d shooter games, like COD or practically any 3d game.
  
  I just don't understand how they teach the enemies such different things based on the game to fit its narratives. For example, is the enemy "AI" in 3d games just a bunch of if-else statements, or do they actually teach the enemies to think strategically? In big AAA games, you can clearly see that enemies hide from you in shootings and peek to shoot not just rush and get killed.
  
  So, how is the AI in 3d games implemented? How do they code it? You don't need to explain in detail, but just give me the idea. Do they use algorithms?
  
  **Tags:** reinforcement-learning, game-ai, implementation, gaming, algorithm-request
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-03-16. Score: 5 | Accepted answer
  
  ## Overlap between AI and "Game AI"
  
  Nowadays, if you search for AI online, you will find a lot of material about machine learning, natural language processing, intelligent agents and neural networks. These are not the whole of AI by any means, expecially in a historical context, but they have recently been very successful, there is lots of published material about them.
  
  Games, especially action games, tend not to use these new popular technologies because they have other priorities. However, under the broadest definitions, there is good overlap between AI in general and game AI, or more specifically *enemy AI* within a game:
  
  * Computer-controlled opponents within a game are effectively autonomous agents operating within an environment.
  * The game's purpose is to entertain the player, and this usually translates into requiring challenging and believable behaviour for actions taken by opponents (for some value of "believable").
  * A 3D map with a physics engine, other moving agents and possible hazards is a complex environment that needs some effort from a developer in order to define working behaviours.
  * Many games need to solve one or two classic AI problems such as [pathfinding](https://arrow.tudublin.ie/cgi/viewcontent.cgi?article=1063&context=itbj).
  
  All the above means that the term "Game AI" is a good choice for describing how enemy agents are designed and implemented. However, there are large differences between goals of AI applied today in a 3D shooter and the goals of AI used in industry to control decision making.
  
  * Action games have a low CPU budget available per enemy for decision-making. Control systems in industry can have dedicated machinery, sometimes multiple machines, just to run the AI.
  * Action game priority is the game experience of the player, and this translates into strict targeting of a definition of "believable" for enemy behaviour that will entertain. In contrast, AI used in industry has accuracy as a high priority.
  
  ## Typical game AI in 3D shooters
  
  Developing enemy AI in games is a specialist skill, and only partly related to the purpose of this site. For detailed discussion you may want to look into [Game Development Stack Exchange](https://gamedev.stackexchange.com/) which has many questions and answers on topics like [Enemy AI](https://gamedev.stackexchange.com/search?q=enemy%20AI)
  
  Very roughly, the key traits of enemy AI in a game *like* Call of Duty could look like this (I have no inside knowledge of how COD does this, so may be inaccurate in places):
  
  * Enemies will have one or more modes of behaviour defined. This might be as simple as "Idle" and "Attack Player", or there could be many. For COD I would suspect there are many, and some may be in a hierarchy - e.g. there may be several sub-types of "Attack" behaviour depending on the enemy design.
  * Within a mode of behaviour, there may be a few different components defined, only some of which use AI routines. For instance, there will be specific animations related to walking or standing which are not AI, but there will also be some pathfinding AI if the agent is attempting to move around.
  * Decisions to switch between different modes of behaviour are often scripted with simple triggers, such as detecting player visibility using ray-casting between the location of player and enemy. When game AI fails to produce realistic results, it is often these high level triggers being brittle and not covering edge cases that causes it. Depending on complexity of the game and number of behaviours, there may be an algorithm managing the transitions between them, such as a [finite state engine](http://www.ai-junkie.com/architecture/state_driven/tut_state1.html) or [behaviour trees](https://jahej.com/alt/2011_02_24_introduction-to-behavior-trees.html).
  * Enemies will be presented with highly simplified observations of the game world from their perspective, in order to speed AI decisions.
  * AI systems will have CPU budget restricted. A 3D game spends significant resources rendering scenes, and will often try to render quickly, e.g. 100 times per second. There are multiple ways that AI budget can be allocated, but it is relatively common for AI calculations to be spread over muliple frames, and for search structures for tasks like path-finding to persist over time.
  * There will be a middle ground between scripted behaviour and AI-driven behaviour where analysis is done as part of game design. For instance, pathfinding routes may be pre-calculated to some degree. A system of [way points](http://jceipek.com/Olin-Coding-Tutorials/pathing.html#waypoints) is one example of this - it might be set by the game designer, it may be calculated by an AI component of the game asset-building pipeline, or it may be dynamically calculated and cached during a game session so that multiple enemy units can share it.
  * When complex AI is not needed to achieve a goal, when a simple caclulation or "puppet-like" behaviour would do just fine, then this could be chosen instead. For instance, an enemy "aiming" at a player can be a simple vector calculation, perhaps with a fudge factor of a miss chance depending on the range to make the enemy seem fallible and not as much like a machine.
  
  I do not know how the behaviour of enemies using cover is implemented in COD. A simple variant would be to have each enemy hard-coded to use a specific piece of cover that usually works well against the player due to map design. However, it is definitley *possible* to have a search algorithm running (perhaps over multiple frames) that assesses nearby locations that the enemy could reach against the player's current position, and then pick those as variables to plug into the "take cover and fire on player" scripted behaviour. That assessment would use the same kind of visibilty detection between enemy and player that is used to trigger changes between "Idle" and "Attack" behaviours for the enemy.
  
  ---
  
  As an aside, one related thing I find interesting is in [using modern AI techniques to blend animations and make interactions between actors and the environment look more realistic](https://www.youtube.com/watch?v=Mnu1DzFzRWs). Although it is a lower-level feature than the question about enemy behaviour you are asking, it is an interesting cross-over between robotics and game playing that we will likely see in next-generation games, and probably applied first to the more detailed player model animations. [Here is another example applied to a humanoid agent switching smoothly between different tasks](https://www.youtube.com/watch?v=cTqVhcrilrE), which any game player will recognise as something which game engines cannot do well at the moment - there are usually many jarring transitions caused by events in the game.
  
  > **Comments:**
  > **username_3**: This answer seems reasonable to me, but given that some people, like me, are not really familiar with the details of the implementation of these "AI systems" in games, it may be a good idea to provide a reference to games that use the approaches that you mention. For example, when you say "Decisions to switch between different modes of behaviour are often scripted with simple triggers", this seems reasonable for anyone (including me) that played any 3d shooter/game, but maybe you could provide a reference that shows this is true.  
  > **username_2**: @username_3: I think the best reference in general will be the Game Dev Stack Exchange link. I will try to find answers in there that back up my assertions, but I have to re-state that I have no evidence about the specific outlined approach being accurate for COD. There is definitely more than one way to structure enemy behaviour logic in a game.  
  > **username_2**: @username_3: I added links to some ways of managing enemy behaviour transitions in games, which at least demonstrates that these are real things discussed by professional game developers and not something I just made up to answer the question  
  > **username_2**: I have also added an aside to a related topic where modern AI and game development are overlapping and we should expect to see some results from it in near future with smoother movement and behaviour transitions in animations. So there is room for this kind of AI in games soon, although we're a long way off from each enemy in a 3D shooter having a decision-making neural network as standard.  
  > **username_3**: Thanks! Now, I think your answer can be more useful for people like me that don't have a good knowledge of the topic. A few weeks ago, I also read a paper (I think [Game AI Revisited](https://dl.acm.org/doi/pdf/10.1145/2212908.2212954) by Yannakakis), which, if I remember correctly, is consistent with what you just said, so that's why I also upvoted your answer. You may be interested in that paper, which also talks about procedural content generation and stuff like that (if I remember correctly).  
  
  ---
  
  ### Answer by username_1 on 2021-03-16. Score: 0
  
  Most of the so called 'AI' enemies in games nowadays are based on human logics and not machine learning, although they're called AIs.
  
  Mentioning about games, the first thing for ML is reinforcement learning. Let the bots play round to get rewards (positive) or punishment (negative rewards). These bots may finally know how to hide or move out a little bit to shoot.
  
  I can guess of some possible actions for bots in shooter games:  
  Move U/D/L/R, jump, duck, find target (coordinates already in memory, valid only in view angle), shoot.
  
  Training bot in 3D environments is much harder than in 2D games, since every looking angle of the bot is another environment state, 360 degrees around and 360 degrees up/down. And enemies can be at any location, in or off the frame.
  
  > **Comments:**
  > **username_2**: OP is not asking how to implement ML for games, but how existing game AI works.  
  > **username_1**: yeah, i said it's all human logics for present games, no ML  
  
  ---
  
  