Answers.Body: <p>By representing the Rewards and Values as vectors, the network is able
  to model uncertainty. Instead of choosing one specific reward it can give multiple possible rewards, a non-zero probability.</p>
  <p>This stabilizes the training process, since parameter updates are based on
  more information.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-30T10:23:03.057Z
Answers.LastActivityDate: 2024-08-30T10:23:03.057Z
Answers.UserDisplayName: Lynix
Body: <p>In the MuZero Paper in the Appendix F they explain that they represent values and rewards as vectors.
  <a href="https://i.sstatic.net/pDrQY2fg.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/pDrQY2fg.png" alt="enter image description here" /></a></p>
  <p>This means that the neural networks don't output the scalars directly,
  instead, they output a probability distribution that later gets converted back to a scalar.</p>
  <p>I wonder why it's done this way. Let's say they want to support a reward/value range of
  [-60000, 60000]. They could have the network output a scalar y and then do tanh(y)*60000
  or even output the actual reward or value directly.</p>
  <p>What's the advantage of representing Scalars as Vectors?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-29T09:29:06.727Z
Comments.Text: The output becomes less deterministic
Comments.UserDisplayName: hff1
Tags: <neural-networks><machine-learning><ai-design><muzero>
Title: Representing scalars as vectors for the network output
UserDisplayName: Lynix
ThreadText: # Representing scalars as vectors for the network output
  
  (Asked by: username_0 on 2024-08-28)
  
  In the MuZero Paper in the Appendix F they explain that they represent values and rewards as vectors.
  [![enter image description here](https://i.sstatic.net/pDrQY2fg.png)](https://i.sstatic.net/pDrQY2fg.png)
  
  This means that the neural networks don't output the scalars directly,
  instead, they output a probability distribution that later gets converted back to a scalar.
  
  I wonder why it's done this way. Let's say they want to support a reward/value range of
  [-60000, 60000]. They could have the network output a scalar y and then do tanh(y)\*60000
  or even output the actual reward or value directly.
  
  What's the advantage of representing Scalars as Vectors?
  
  **Tags:** neural-networks, machine-learning, ai-design, muzero
  
  > **Comments:**
  > **username_1**: The output becomes less deterministic  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2024-08-30. Score: 2 | Accepted answer
  
  By representing the Rewards and Values as vectors, the network is able
  to model uncertainty. Instead of choosing one specific reward it can give multiple possible rewards, a non-zero probability.
  
  This stabilizes the training process, since parameter updates are based on
  more information.
  
  
  ---
  
  