<p>You do not output the batch of Q-values. Input frame stacking is needed to gain full observability of the environment. In your case the output would be 6 elements for your current frame. If <span class="math-container">$F$</span> is a frame then you would stack 4 frames <span class="math-container">$[F_{k-3}, F_{k-2}, F_{k-1}, F_k]$</span> and the output would be 6 Q-values for frame <span class="math-container">$F_k$</span>.</p>

CC BY-SA 4.0
2020-06-05T17:58:57.340Z
I still don't get the idea, when I input 4 frames It outputs 4 by 6 values, so what are these values?
dato nefaridze
CC BY-SA 4.0
2020-06-05T18:07:12.577Z
It outputs 4 by 6 matrix because you defined the network architecture such that it outputs 4 by 6 matrix. Tensoflow, Pytorch or whatever you used cannot read your mind and know how many outputs do you want. You need to fix your network architecture
Brale
CC BY-SA 4.0
2020-06-05T09:30:59.153Z
2020-06-05T09:30:59.153Z
Brale
<p>I decided to train deep Q-learning agent based on getting raw pixels from environment.I have one particular problem:when I input stack of frames, suppose 4 consecutive frames, if action space is 6,then output is 4 by 6 matrix.So which one is real Q-value?I mean, I input batch of frames and it outputs batch of values and question is which is real Q-value out of those batch values?</p>

<q-learning><deep-rl>
How to predict Q-values based on stack of frames
dato nefaridze
# How to predict Q-values based on stack of frames

(Asked by: username_0 on 2020-06-05)

I decided to train deep Q-learning agent based on getting raw pixels from environment.I have one particular problem:when I input stack of frames, suppose 4 consecutive frames, if action space is 6,then output is 4 by 6 matrix.So which one is real Q-value?I mean, I input batch of frames and it outputs batch of values and question is which is real Q-value out of those batch values?

**Tags:** q-learning, deep-rl


---

## 1 answers

### Answer by username_1 on 2020-06-05. Score: 2

You do not output the batch of Q-values. Input frame stacking is needed to gain full observability of the environment. In your case the output would be 6 elements for your current frame. If $F$ is a frame then you would stack 4 frames $[F\_{k-3}, F\_{k-2}, F\_{k-1}, F\_k]$ and the output would be 6 Q-values for frame $F\_k$.

> **Comments:**
> **username_0**: I still don't get the idea, when I input 4 frames It outputs 4 by 6 values, so what are these values?  
> **username_1**: It outputs 4 by 6 matrix because you defined the network architecture such that it outputs 4 by 6 matrix. Tensoflow, Pytorch or whatever you used cannot read your mind and know how many outputs do you want. You need to fix your network architecture  

---

