Answers.Body: <p>This is a whole sub-field of reinforcement learning known as model-based reinforcement learning. The idea in model based RL is to learn the mapping from current state/action to next state in order to facilitate learning good policies.</p>
  <p>If you are dealing with images as inputs I would recommend checking out the Dreamer papers. The most recent being <a href="https://danijar.com/project/dreamerv2/" rel="nofollow noreferrer">this one</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-24T12:58:29.897Z
Answers.Text: Note that, in model-based RL, we don't necessarily _learn_ the model: [the only requirement is that we **use** a **model** of the transition function](https://ai.stackexchange.com/q/4456/2444).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-27T00:17:54.453Z
Answers.LastActivityDate: 2021-04-27T00:17:54.453Z
Answers.UserDisplayName: James
Answers.Body: <p>I tried something similar before for 2048 game. I used the state of the board as <strong>x</strong>, and the move as <strong>y</strong>. I just trained the neural network with this dataset. The architecture is like a couple of layers with <strong>relu</strong> and the final layer as <strong>softmax</strong>. The major thing is that we should not feed the wrong moves in the dataset to the NN, or else the NN also tend to learn the bad moves, which in turn makes it less smarter.</p>
  
  <p>I gathered my dataset by running the minimax on 2048 and assigning a reward for each move, and then eliminating the bad ones on it.</p>
  
  <p>the above process also depends on the way you are taking the feature vector, if your feature vector is an image, then it makes sense to use CNN.</p>
  
  <p><a href="https://github.com/navjindervirdee/2048-deep-reinforcement-learning" rel="nofollow noreferrer">https://github.com/navjindervirdee/2048-deep-reinforcement-learning</a></p>
  
  <p>DQN is also a good option. but do checkout the above link, it helped me too.</p>
  
  <p>My repo: <a href="https://github.com/williamscott701/AI-vs-2048" rel="nofollow noreferrer">https://github.com/williamscott701/AI-vs-2048</a></p>
  
  <p>my results are not actually that great.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-08T18:44:06.283Z
Answers.LastActivityDate: 2018-12-08T18:44:06.283Z
Answers.UserDisplayName: William Scott
Answers.Body: <p>You'll probably want to start out with "Action-Conditional Video Prediction using Deep Networks in Atari Games" (arXiv link: <a href="https://arxiv.org/abs/1507.08750" rel="nofollow noreferrer">https://arxiv.org/abs/1507.08750</a>). That's from 2015 though, I'm sure there have been lots of other interesting developments since then. This paper may still be a good starting point though, and provide you with the correct terminology to plug into google / google scholar to find more recent papers that build on top of this. Google scholar also provides functionality to automatically find papers that cite this one (interesting recent papers will probably cite this one).</p>
  
  <p>As an additional point, you may want to reconsider your desired output. Given a current state-image and action, it may be easier to train a network to predict only the <em>change</em> in image (i.e., predicting NEW_IMAGE - OLD_IMAGE), rather than predicting the full image. You can then always still manually reconstruct the predicted new image simply by adding that output to the old image again. I'm quite sure I've seen this being done in a more recent paper too, but don't remember exactly the title / authors.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-18T13:12:00.410Z
Answers.LastActivityDate: 2017-12-18T13:12:00.410Z
Answers.UserDisplayName: Dennis Soemers
Body: <p>I am trying to find literature on a network architecture that takes the following as in input:</p>
  <ul>
  <li>Action (like 'Up', 'Down', etc)</li>
  <li>Image of the current state</li>
  </ul>
  <p>and outputs:</p>
  <ul>
  <li>Image of next state</li>
  </ul>
  <p>I already have a lot of training data for the inputs. However, I am trying to find relevant literature/architecture for this problem.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-18T01:39:49.953Z
Comments.Text: What kind of images are you looking at?
Comments.UserDisplayName: Brian O'Donnell
Tags: <neural-networks><reinforcement-learning><reference-request><game-ai><model-request>
Title: Is there a neural network in the literature that predicts the next game state based on the current state and the action?
UserDisplayName: Sandeep Silwal
ThreadText: # Is there a neural network in the literature that predicts the next game state based on the current state and the action?
  
  (Asked by: username_0 on 2017-12-17)
  
  I am trying to find literature on a network architecture that takes the following as in input:
  
  * Action (like 'Up', 'Down', etc)
  * Image of the current state
  
  and outputs:
  
  * Image of next state
  
  I already have a lot of training data for the inputs. However, I am trying to find relevant literature/architecture for this problem.
  
  **Tags:** neural-networks, reinforcement-learning, reference-request, game-ai, model-request
  > **Comments:**
  > **username_1**: What kind of images are you looking at?  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2021-04-27. Score: 2
  
  This is a whole sub-field of reinforcement learning known as model-based reinforcement learning. The idea in model based RL is to learn the mapping from current state/action to next state in order to facilitate learning good policies.
  
  If you are dealing with images as inputs I would recommend checking out the Dreamer papers. The most recent being [this one](https://danijar.com/project/dreamerv2/).
  
  > **Comments:**
  > **username_3**: Note that, in model-based RL, we don't necessarily _learn_ the model: [the only requirement is that we **use** a **model** of the transition function](https://ai.stackexchange.com/q/4456/2444).  
  
  ---
  
  ### Answer by username_4 on 2018-12-08. Score: 0
  
  I tried something similar before for 2048 game. I used the state of the board as **x**, and the move as **y**. I just trained the neural network with this dataset. The architecture is like a couple of layers with **relu** and the final layer as **softmax**. The major thing is that we should not feed the wrong moves in the dataset to the NN, or else the NN also tend to learn the bad moves, which in turn makes it less smarter.
  
  I gathered my dataset by running the minimax on 2048 and assigning a reward for each move, and then eliminating the bad ones on it.
  
  the above process also depends on the way you are taking the feature vector, if your feature vector is an image, then it makes sense to use CNN.
  
  <https://github.com/navjindervirdee/2048-deep-reinforcement-learning>
  
  DQN is also a good option. but do checkout the above link, it helped me too.
  
  My repo: <https://github.com/williamscott701/AI-vs-2048>
  
  my results are not actually that great.
  
  
  ---
  
  ### Answer by username_5 on 2017-12-18. Score: 0
  
  You'll probably want to start out with "Action-Conditional Video Prediction using Deep Networks in Atari Games" (arXiv link: <https://arxiv.org/abs/1507.08750>). That's from 2015 though, I'm sure there have been lots of other interesting developments since then. This paper may still be a good starting point though, and provide you with the correct terminology to plug into google / google scholar to find more recent papers that build on top of this. Google scholar also provides functionality to automatically find papers that cite this one (interesting recent papers will probably cite this one).
  
  As an additional point, you may want to reconsider your desired output. Given a current state-image and action, it may be easier to train a network to predict only the *change* in image (i.e., predicting NEW\_IMAGE - OLD\_IMAGE), rather than predicting the full image. You can then always still manually reconstruct the predicted new image simply by adding that output to the old image again. I'm quite sure I've seen this being done in a more recent paper too, but don't remember exactly the title / authors.
  
  
  ---
  
  