Body: <p>The paper, <a href="https://arxiv.org/pdf/1507.06527.pdf" rel="nofollow noreferrer">Deep Recurrent Q-Learning for Partially Observable MDPs</a>, talks about stacking multiple observations in the input of a convolutional neural network.</p>
  <p>How does this exactly work? Do the convolutional filters loop over each observation (image)?</p>
  <p>(I know this isn't the right group to request this, but I'll highly appreciate if someone could also suggest a framework that helps with this.)</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-28T11:17:34.770Z
Comments.Text: I didn't read that paper, but it seems like you're talking about creating inputs (which, I suppose, represent _states_ of the environment) to the value network which are composed of $N > 1$ successive frames/images (i.e. a state is a sequence of successive frames). This was also done in the original DQN paper. This is done mainly to provide more context, i.e. to partially solve the "partial observability" problem. As far as I remember from DQN, you just stack the frames, and that's going to be treated as a single state to the value network, though that may not be actually a "state".
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-28T11:23:25.420Z
Comments.Text: Anyway, in that specific paper, they may be doing something different (especially after having looked at figure 2). Maybe later, if, meanwhile, someone doesn't answer this question, I will quickly read it and provide an answer.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><q-learning><dqn><deep-rl><pomdp>
Title: How does one stack multiple observations in the input layer of a convolutional neural network?
UserDisplayName: desert_ranger
ThreadText: # How does one stack multiple observations in the input layer of a convolutional neural network?
  
  (Asked by: username_0 on 2020-10-28)
  
  The paper, [Deep Recurrent Q-Learning for Partially Observable MDPs](https://arxiv.org/pdf/1507.06527.pdf), talks about stacking multiple observations in the input of a convolutional neural network.
  
  How does this exactly work? Do the convolutional filters loop over each observation (image)?
  
  (I know this isn't the right group to request this, but I'll highly appreciate if someone could also suggest a framework that helps with this.)
  
  **Tags:** reinforcement-learning, q-learning, dqn, deep-rl, pomdp
  
  > **Comments:**
  > **username_1**: I didn't read that paper, but it seems like you're talking about creating inputs (which, I suppose, represent _states_ of the environment) to the value network which are composed of $N > 1$ successive frames/images (i.e. a state is a sequence of successive frames). This was also done in the original DQN paper. This is done mainly to provide more context, i.e. to partially solve the "partial observability" problem. As far as I remember from DQN, you just stack the frames, and that's going to be treated as a single state to the value network, though that may not be actually a "state".  
  > **username_1**: Anyway, in that specific paper, they may be doing something different (especially after having looked at figure 2). Maybe later, if, meanwhile, someone doesn't answer this question, I will quickly read it and provide an answer.  
  
  ---
  
  ## 0 answers
  
  