Answers.Body: <h2>Theory</h2>
  <p>Learn the basic Reinforcement Learning (RL) concepts: Agent, Environment, Action, Reward, Return, Episode, Exploration. Maybe look at Q-Learning, but you don't need to dive deep yet.</p>
  <p>Check out the <a href="https://github.com/Farama-Foundation/Gymnasium#environments" rel="nofollow noreferrer">Gymn environments</a> or <a href="https://github.com/DLR-RM/stable-baselines3#stable-baselines3" rel="nofollow noreferrer">stable-baselines3</a>. Both are Python libraries used for RL. Look at those for inspiration, not for getting programming experience. It's better to implement something much simpler on your own first.</p>
  <h2>Programming</h2>
  <p>If you're new to programming, even a single-player snakes will be a hard challenge. (But a good one, and there are several tutorials.) Interactive games are generally some of the hardest programming challenges.</p>
  <p>I advice against Unity3D. Not because it is bad, but because it's too complex. The easiest way to start with programming is with text output. If you're doing a game played entirely by AI, you may even get away without GUI. (Just print the game-world as an array.)</p>
  <p>Languages: Python is the natural choice because it's everywhere in machine learning and easy to pick up. C# if you want to use Unity3D later. Rust only if you are serious about investing time into programming. (Avoid C++ for now.) Python, Javascript and C# will all get you faster to something that works.</p>
  <h2>Step 1: Simple Environment</h2>
  <p>Some ideas:</p>
  <ul>
  <li>A racing-game where everything is a circle (walls, vehicles). Easy to detect collisions. Goal/Reward: going fast.</li>
  <li>A grid-world with random stuff lying around (obstacles, apples). Similar to snakes, but without the snake getting longer. Goal/Reward: apples collected.</li>
  </ul>
  <p>Don't implement the AI yet. Instead, add an agent that ignores the inputs and makes a random choice every turn.</p>
  <p>This will a) get you to something that runs, and b) a baseline score that your AI should beat. It's important to design the game such that random actions can sometimes score a point by chance. This gives the learning algorithm a signal to improve.</p>
  <h2>Step 2: Machine Learning</h2>
  <p>Now we need some kind of mapping between the agent inputs (e.g. the 8 neighbours in the grid, or distances to the next obstacle) to outputs (e.g. actions).</p>
  <p>Personally I like to implement a neural network for that. It's actually simple (maybe 4 lines of code or so) if you only implement the forward-pass (no back-propagation) and choose the weights at random. With just random search, you should be able to discover weights that beat the baseline score.</p>
  <p>Also, it's often better if the outputs are action probabilities (instead of just one action). A common way to do that with neural networks is a softmax layer.</p>
  <h3>Algorithms:</h3>
  <ul>
  <li>Random search.
  <ul>
  <li>Really. It has been shown that some Atari games can be solved just by trying random weights on a neural network. Worth a shot.</li>
  <li>&quot;Random search&quot; is also the name of a specific algorithm, which is not what I mean. The point is to first implement the simplest possible method and debug/simplify until it can make <em>some</em> progress. This builds your understanding how easy/difficult the task is to learn.</li>
  </ul>
  </li>
  <li>Cross-Entropy Method (CEM). Fancy name, but can be simple to implement with basic probability theory (like, just a normal distribution).
  <ul>
  <li>Reference: <a href="https://www.researchgate.net/profile/Andras-Loerincz/publication/6743957_Learning_Tetris_Using_the_Noisy_Cross-Entropy_Method/links/5751509408ae6807faf953e5/Learning-Tetris-Using-the-Noisy-Cross-Entropy-Method.pdf" rel="nofollow noreferrer">Learning Tetris Using the Noisy Cross-Entropy Method</a></li>
  </ul>
  </li>
  <li><a href="https://blog.otoro.net/2017/10/29/visual-evolution-strategies/#what-is-an-evolution-strategy" rel="nofollow noreferrer">Evolution strategies</a> have been shown to be <a href="https://openai.com/research/evolution-strategies" rel="nofollow noreferrer">competitive with RL algorithms</a> sometimes.</li>
  <li>A genetic algorithm. The <a href="https://users.sussex.ac.uk/%7Einmanh/MicrobialGA_ECAL2009.pdf" rel="nofollow noreferrer">Microbial GA</a> is one that is easy to implement yourself.</li>
  <li>The neuroevolution algorithm <a href="https://neat-python.readthedocs.io/en/latest/neat_overview.html" rel="nofollow noreferrer">NEAT</a> is also popular for this kind of task.</li>
  <li>Any real RL algorithm from the <a href="https://github.com/DLR-RM/stable-baselines3#stable-baselines3" rel="nofollow noreferrer">stable-baselines3</a>.</li>
  </ul>
  <p>I wrote <a href="https://log2.ch/2019/exploring-a-pixel-maze-with-evolution-strategies/" rel="nofollow noreferrer">blogpost</a> in 2019 about an experiment in the same spirit.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-24T19:00:33.463Z
Answers.LastActivityDate: 2023-09-25T06:29:05.293Z
Answers.LastEditDate: 2023-09-25T06:29:05.293Z
Answers.UserDisplayName: maxy
Answers.Body: <p>You can start with Monte Carlo Tree Search to play a 2D game. It is a pretty common algorithm in Reinforcement Learning. If you are comfortable enough with Neural Networks, you can use AlphaZero to create an algorithm to play a 2D game.</p>
  <p>Here is a basic implementation of Monte Carlo Tree Search for TicTacToe: <a href="https://github.com/nityanandmathur/MCTS" rel="nofollow noreferrer">https://github.com/nityanandmathur/MCTS</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-24T17:57:21.553Z
Answers.Text: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-24T17:37:19.203Z
Answers.LastActivityDate: 2023-03-24T17:37:19.203Z
Answers.UserDisplayName: Nityanand Mathur
Answers.Body: <p>Basic knowledge in machine learning does typically not involve reinforcement learning, which is an ideal choice to make an agent in a video game. Reinforcement learning is famous for <a href="https://arxiv.org/abs/1312.5602" rel="nofollow noreferrer">beating the simple Atari games</a>, and later beating top players in more complicated games like <a href="https://en.wikipedia.org/wiki/AlphaStar_(software)" rel="nofollow noreferrer">StarCraft</a>.</p>
  <p>A good place to start would be pursuing skills and knowledge in both the field of game development and reinforcement learning. How about two independent courses at sites like Coursera and Udemy? There is probably a course that aims to solve some Atari games, which are avaliable at <a href="https://www.gymlibrary.dev/environments/atari/" rel="nofollow noreferrer">AI-Gym</a>.</p>
  <p>Then it will be trivial to determine if python or unity is right for you.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-24T09:48:16.560Z
Answers.LastActivityDate: 2023-03-24T10:03:52.820Z
Answers.LastEditDate: 2023-03-24T10:03:52.820Z
Answers.UserDisplayName: nammerkage
Body: <p>I am keen on creating a little project that can play a fairly basic 2D game (more complex than say, snake but not as complex as mario kart) and would like some pointers on where to begin. I'm entirely new to any coding/programming but have a basic knowledge in machine learning. Is the first step to learn python, or unity? Then from there, what would I look at learning next? Any pointers are appreciated.</p>
  
Tags: <reinforcement-learning>
Title: Where would I start if I wanted to create an AI agent to play a 2d game?
UserDisplayName: Eckavolo
ThreadText: # Where would I start if I wanted to create an AI agent to play a 2d game?
  
  (Asked by: username_0 on 2023-03-24)
  
  I am keen on creating a little project that can play a fairly basic 2D game (more complex than say, snake but not as complex as mario kart) and would like some pointers on where to begin. I'm entirely new to any coding/programming but have a basic knowledge in machine learning. Is the first step to learn python, or unity? Then from there, what would I look at learning next? Any pointers are appreciated.
  
  **Tags:** reinforcement-learning
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2023-03-24. Score: 3
  
  ## Theory
  
  Learn the basic Reinforcement Learning (RL) concepts: Agent, Environment, Action, Reward, Return, Episode, Exploration. Maybe look at Q-Learning, but you don't need to dive deep yet.
  
  Check out the [Gymn environments](https://github.com/Farama-Foundation/Gymnasium#environments) or [stable-baselines3](https://github.com/DLR-RM/stable-baselines3#stable-baselines3). Both are Python libraries used for RL. Look at those for inspiration, not for getting programming experience. It's better to implement something much simpler on your own first.
  
  ## Programming
  
  If you're new to programming, even a single-player snakes will be a hard challenge. (But a good one, and there are several tutorials.) Interactive games are generally some of the hardest programming challenges.
  
  I advice against Unity3D. Not because it is bad, but because it's too complex. The easiest way to start with programming is with text output. If you're doing a game played entirely by AI, you may even get away without GUI. (Just print the game-world as an array.)
  
  Languages: Python is the natural choice because it's everywhere in machine learning and easy to pick up. C# if you want to use Unity3D later. Rust only if you are serious about investing time into programming. (Avoid C++ for now.) Python, Javascript and C# will all get you faster to something that works.
  
  ## Step 1: Simple Environment
  
  Some ideas:
  
  * A racing-game where everything is a circle (walls, vehicles). Easy to detect collisions. Goal/Reward: going fast.
  * A grid-world with random stuff lying around (obstacles, apples). Similar to snakes, but without the snake getting longer. Goal/Reward: apples collected.
  
  Don't implement the AI yet. Instead, add an agent that ignores the inputs and makes a random choice every turn.
  
  This will a) get you to something that runs, and b) a baseline score that your AI should beat. It's important to design the game such that random actions can sometimes score a point by chance. This gives the learning algorithm a signal to improve.
  
  ## Step 2: Machine Learning
  
  Now we need some kind of mapping between the agent inputs (e.g. the 8 neighbours in the grid, or distances to the next obstacle) to outputs (e.g. actions).
  
  Personally I like to implement a neural network for that. It's actually simple (maybe 4 lines of code or so) if you only implement the forward-pass (no back-propagation) and choose the weights at random. With just random search, you should be able to discover weights that beat the baseline score.
  
  Also, it's often better if the outputs are action probabilities (instead of just one action). A common way to do that with neural networks is a softmax layer.
  
  ### Algorithms:
  
  * Random search.
    + Really. It has been shown that some Atari games can be solved just by trying random weights on a neural network. Worth a shot.
    + "Random search" is also the name of a specific algorithm, which is not what I mean. The point is to first implement the simplest possible method and debug/simplify until it can make *some* progress. This builds your understanding how easy/difficult the task is to learn.
  * Cross-Entropy Method (CEM). Fancy name, but can be simple to implement with basic probability theory (like, just a normal distribution).
    + Reference: [Learning Tetris Using the Noisy Cross-Entropy Method](https://www.researchgate.net/profile/Andras-Loerincz/publication/6743957_Learning_Tetris_Using_the_Noisy_Cross-Entropy_Method/links/5751509408ae6807faf953e5/Learning-Tetris-Using-the-Noisy-Cross-Entropy-Method.pdf)
  * [Evolution strategies](https://blog.otoro.net/2017/10/29/visual-evolution-strategies/#what-is-an-evolution-strategy) have been shown to be [competitive with RL algorithms](https://openai.com/research/evolution-strategies) sometimes.
  * A genetic algorithm. The [Microbial GA](https://users.sussex.ac.uk/%7Einmanh/MicrobialGA_ECAL2009.pdf) is one that is easy to implement yourself.
  * The neuroevolution algorithm [NEAT](https://neat-python.readthedocs.io/en/latest/neat_overview.html) is also popular for this kind of task.
  * Any real RL algorithm from the [stable-baselines3](https://github.com/DLR-RM/stable-baselines3#stable-baselines3).
  
  I wrote [blogpost](https://log2.ch/2019/exploring-a-pixel-maze-with-evolution-strategies/) in 2019 about an experiment in the same spirit.
  
  
  ---
  
  ### Answer by username_2 on 2023-03-24. Score: 0
  
  You can start with Monte Carlo Tree Search to play a 2D game. It is a pretty common algorithm in Reinforcement Learning. If you are comfortable enough with Neural Networks, you can use AlphaZero to create an algorithm to play a 2D game.
  
  Here is a basic implementation of Monte Carlo Tree Search for TicTacToe: <https://github.com/nityanandmathur/MCTS>
  
  > **Comments:**
  > **username_3**: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  
  ---
  
  ### Answer by username_4 on 2023-03-24. Score: 0
  
  Basic knowledge in machine learning does typically not involve reinforcement learning, which is an ideal choice to make an agent in a video game. Reinforcement learning is famous for [beating the simple Atari games](https://arxiv.org/abs/1312.5602), and later beating top players in more complicated games like [StarCraft](https://en.wikipedia.org/wiki/AlphaStar_(software)).
  
  A good place to start would be pursuing skills and knowledge in both the field of game development and reinforcement learning. How about two independent courses at sites like Coursera and Udemy? There is probably a course that aims to solve some Atari games, which are avaliable at [AI-Gym](https://www.gymlibrary.dev/environments/atari/).
  
  Then it will be trivial to determine if python or unity is right for you.
  
  
  ---
  
  