Answers.Body: <p>No the dimensionality size difference it's not a problem when defining a reward function using neural networks, but if you really want to do so, you can easily do that using two different mappings as first layer:
  <span class="math-container">$$
  r(a,s) = \phi(Wa+b, Us+c)\\
  |W| = 256\times N\\
  |U| = 6\times N 
  $$</span></p>
  <p>However, I don't really see a point to add this step instead of going for a fully connected layer since the beginning (as this ^^^ method can be seen as a masked fully connected layer, thus it's less powerful than it)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-31T12:40:08.817Z
Answers.LastActivityDate: 2023-07-31T12:40:08.817Z
Answers.UserDisplayName: Alberto
Body: <p>I'm currently writing code for a reward predictor function r=f(s,a) in reinforcement learning, where 's' is the state with 256 dimensions (the embedding dimension after visual input is processed by an encoder) and 'a' is the action with 6 dimensions. I could use a Fully Connected Neural Network (FCNN) for this function.</p>
  <p>I'm wondering if it's necessary or beneficial to adjust the dimensions of the state or action inputs given the significant difference in their dimensions? For instance, should I consider directly increasing the dimensionality of the action to match the state dimensionality using a linear layer? Or are there other methods to handle these large dimensionality differences?</p>
  <p>I welcome any related discussions, or references to similar concept code or code repository links.</p>
  
Tags: <reinforcement-learning><deep-learning><deep-rl><deep-neural-networks><action-spaces>
Title: How to handle large dimensionality differences between state and action inputs in a reinforcement learning predictor?
UserDisplayName: XiaoBanni
ThreadText: # How to handle large dimensionality differences between state and action inputs in a reinforcement learning predictor?
  
  (Asked by: username_0 on 2023-07-31)
  
  I'm currently writing code for a reward predictor function r=f(s,a) in reinforcement learning, where 's' is the state with 256 dimensions (the embedding dimension after visual input is processed by an encoder) and 'a' is the action with 6 dimensions. I could use a Fully Connected Neural Network (FCNN) for this function.
  
  I'm wondering if it's necessary or beneficial to adjust the dimensions of the state or action inputs given the significant difference in their dimensions? For instance, should I consider directly increasing the dimensionality of the action to match the state dimensionality using a linear layer? Or are there other methods to handle these large dimensionality differences?
  
  I welcome any related discussions, or references to similar concept code or code repository links.
  
  **Tags:** reinforcement-learning, deep-learning, deep-rl, deep-neural-networks, action-spaces
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-07-31. Score: 2
  
  No the dimensionality size difference it's not a problem when defining a reward function using neural networks, but if you really want to do so, you can easily do that using two different mappings as first layer:
  $$
  r(a,s) = \phi(Wa+b, Us+c)\\
  |W| = 256\times N\\
  |U| = 6\times N
  $$
  
  However, I don't really see a point to add this step instead of going for a fully connected layer since the beginning (as this ^^^ method can be seen as a masked fully connected layer, thus it's less powerful than it)
  
  
  ---
  
  