<p>By representing the Rewards and Values as vectors, the network is able
to model uncertainty. Instead of choosing one specific reward it can give multiple possible rewards, a non-zero probability.</p>
<p>This stabilizes the training process, since parameter updates are based on
more information.</p>

CC BY-SA 4.0
2024-08-30T10:23:03.057Z
2024-08-30T10:23:03.057Z
Lynix
<p>In the MuZero Paper in the Appendix F they explain that they represent values and rewards as vectors.
<a href="https://i.sstatic.net/pDrQY2fg.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/pDrQY2fg.png" alt="enter image description here" /></a></p>
<p>This means that the neural networks don't output the scalars directly,
instead, they output a probability distribution that later gets converted back to a scalar.</p>
<p>I wonder why it's done this way. Let's say they want to support a reward/value range of
[-60000, 60000]. They could have the network output a scalar y and then do tanh(y)*60000
or even output the actual reward or value directly.</p>
<p>What's the advantage of representing Scalars as Vectors?</p>

CC BY-SA 4.0
2024-08-29T09:29:06.727Z
The output becomes less deterministic
hff1
<neural-networks><machine-learning><ai-design><muzero>
Representing scalars as vectors for the network output
Lynix
# Representing scalars as vectors for the network output

(Asked by: username_0 on 2024-08-28)

In the MuZero Paper in the Appendix F they explain that they represent values and rewards as vectors.
[![enter image description here](https://i.sstatic.net/pDrQY2fg.png)](https://i.sstatic.net/pDrQY2fg.png)

This means that the neural networks don't output the scalars directly,
instead, they output a probability distribution that later gets converted back to a scalar.

I wonder why it's done this way. Let's say they want to support a reward/value range of
[-60000, 60000]. They could have the network output a scalar y and then do tanh(y)\*60000
or even output the actual reward or value directly.

What's the advantage of representing Scalars as Vectors?

**Tags:** neural-networks, machine-learning, ai-design, muzero

> **Comments:**
> **username_1**: The output becomes less deterministic  

---

## 1 answers

### Answer by username_0 on 2024-08-30. Score: 2 | Accepted answer

By representing the Rewards and Values as vectors, the network is able
to model uncertainty. Instead of choosing one specific reward it can give multiple possible rewards, a non-zero probability.

This stabilizes the training process, since parameter updates are based on
more information.


---

