Answers.Body: <p>Discretizing the output will probably be counter-productive in this situation; it would remove flexibility by taking away the fine-grained continuous ranges between each discretization bucket, but also blow up the size of the network, reducing manageability and performance. Fragmenting the outputs into buckets in this way may also lead to information loss and more difficulty in convergence because of the fact that each bucket is partially isolated from the others. </p>
  
  <p>After causing myself needless hassle in the past by mismatching input and output dimensions, I'd simply do this if I were in your shoes: 1) keep it simple and use 3 continuous (well, semi-continuous, depending on the highest precision your programming framework allows) outputs for all of the above reasons; 2) clamp the angles between -45 to 225 by whatever method works best for you, like ceiling/floor hard clamping, adding weight terms, etc.; 3) go big on the hidden layer(s) to maximize information sharing across the inputs and eventual outputs, force, angle of strike, position of strike, etc. This is more likely to fine-tune the precision of the outputs, thereby making good use of the semi-continuous scales. </p>
  
  <p>I'm also wondering if a convolutional neural net might work in this situation; their most popular use case is in image processing, but I don't see why you can't treat the force, angle and position as surrogate spatial dimensions. I'm not sure how many or what type of inputs you have, but 3 continuous outputs might be conducive to a 3D rather than a 2D space. Convolutionals are often used for those, as well as higher-dimensional and temporal data. I hope that helps. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-04T04:38:15.947Z
Answers.LastActivityDate: 2016-11-04T04:38:15.947Z
Answers.UserDisplayName: SQLServerSteve
Body: <p>I am trying to build an agent to play carrom. The problem statement is roughly to estimate three parameters (normalized) : </p>
  
  <ul>
  <li>force</li>
  <li>angle of striker</li>
  <li>position of strike </li>
  </ul>
  
  <p>Since the state and action space both are continuous, I thought of discretizing the output such that I have 270 [ valid angles from -45 to 225 degrees ] outputs for the angle, 10 outputs for force [ranging from 0 to 1] and 20 outputs for the position [ranging from 0 to 1].</p>
  
  <p>Thus I will have 300 output of my neural network, but this number seems a bit too high compared to normal neural networks in practice. </p>
  
  <p>I was wondering if there is a better way of approaching the problem considering the fact that there are multiple parameters to a particular action.</p>
  
  <p>Is there a generic way to approach such problems represented in 2D space. </p>
  
Tags: <deep-learning><deep-neural-networks><models><reinforcement-learning>
Title: Network representation for Q-Learning in carrom
UserDisplayName: stochastic_zeitgeist
ThreadText: # Network representation for Q-Learning in carrom
  
  (Asked by: username_0 on 2016-11-01)
  
  I am trying to build an agent to play carrom. The problem statement is roughly to estimate three parameters (normalized) :
  
  * force
  * angle of striker
  * position of strike
  
  Since the state and action space both are continuous, I thought of discretizing the output such that I have 270 [ valid angles from -45 to 225 degrees ] outputs for the angle, 10 outputs for force [ranging from 0 to 1] and 20 outputs for the position [ranging from 0 to 1].
  
  Thus I will have 300 output of my neural network, but this number seems a bit too high compared to normal neural networks in practice.
  
  I was wondering if there is a better way of approaching the problem considering the fact that there are multiple parameters to a particular action.
  
  Is there a generic way to approach such problems represented in 2D space.
  
  **Tags:** deep-learning, deep-neural-networks, models, reinforcement-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2016-11-04. Score: 0 | Accepted answer
  
  Discretizing the output will probably be counter-productive in this situation; it would remove flexibility by taking away the fine-grained continuous ranges between each discretization bucket, but also blow up the size of the network, reducing manageability and performance. Fragmenting the outputs into buckets in this way may also lead to information loss and more difficulty in convergence because of the fact that each bucket is partially isolated from the others.
  
  After causing myself needless hassle in the past by mismatching input and output dimensions, I'd simply do this if I were in your shoes: 1) keep it simple and use 3 continuous (well, semi-continuous, depending on the highest precision your programming framework allows) outputs for all of the above reasons; 2) clamp the angles between -45 to 225 by whatever method works best for you, like ceiling/floor hard clamping, adding weight terms, etc.; 3) go big on the hidden layer(s) to maximize information sharing across the inputs and eventual outputs, force, angle of strike, position of strike, etc. This is more likely to fine-tune the precision of the outputs, thereby making good use of the semi-continuous scales.
  
  I'm also wondering if a convolutional neural net might work in this situation; their most popular use case is in image processing, but I don't see why you can't treat the force, angle and position as surrogate spatial dimensions. I'm not sure how many or what type of inputs you have, but 3 continuous outputs might be conducive to a 3D rather than a 2D space. Convolutionals are often used for those, as well as higher-dimensional and temporal data. I hope that helps.
  
  
  ---
  
  