Answers.Body: <p>If I'm interpreting this correctly this sounds like a question of output activation functions.  Here's how I would formulate it:</p>
  <ul>
  <li>Direction which is RELU</li>
  <li>Magnitude which is RELU, could be clipped if that's interesting to whatever control style.</li>
  </ul>
  <p>The direction output can be interpreted as radians and converted to degrees from Top Dead Center (TDC).  I'd be tempted when converting expert play to targets to start counting at 2pi so as an expert swings counter clockwise there isn't a temptation to jump from 0 to 2pi as it crosses TDC.  Not sure on the exact effect here but it feels nice to the underlying function approximator.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-05T19:19:21.020Z
Answers.LastActivityDate: 2024-02-05T19:19:21.020Z
Answers.UserDisplayName: foreverska
Body: <p>I've been playing around with some behavioral cloning of a simple old game that uses a joystick. As with behavioral cloning in general, if I record many games, then for each state there are many possible actions, so if <span class="math-container">$s$</span> is the current state and <span class="math-container">$a$</span> the action, then the marginal <span class="math-container">$p(a|s)$</span> is multimodal. The action in this case is basically a direction and a magnitude.</p>
  <p>The problem is how to represent the problem with an actual parametrized model, i.e. a neural network in this case. I see two approaches:</p>
  <ol>
  <li>Discretize the direction and magnitude and use a softmax layer for the outputs.</li>
  <li>Use some form of circular regression.</li>
  </ol>
  <p>For (2) I have no idea what that would be, but I assume something like it must exist. However, how would regression handle the multimodality of the marginals and sampling during inference time? I'm also unsure about the ability of (1) to handle well the multimodality. Also if I discretize the model into lots of buckets, then the output layer of the network also grows quite a bit.</p>
  <p>My question is essentially if more principled approaches exist and especially under the following &quot;real-world&quot; assumptions:</p>
  <ol>
  <li>There is lots of data, i.e. from an actual multiplayer game.</li>
  <li>The controls are fine-grained enough that you require a high number of buckets (e.g. precise aiming).</li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-05T10:44:52.623Z
Comments.Text: maybe a generative approach might help
Comments.UserDisplayName: Alberto
Tags: <regression><supervised-learning><behavioral-cloning>
Title: Circular regression for joystick movements?
UserDisplayName: eof
ThreadText: # Circular regression for joystick movements?
  
  (Asked by: username_0 on 2024-02-05)
  
  I've been playing around with some behavioral cloning of a simple old game that uses a joystick. As with behavioral cloning in general, if I record many games, then for each state there are many possible actions, so if $s$ is the current state and $a$ the action, then the marginal $p(a|s)$ is multimodal. The action in this case is basically a direction and a magnitude.
  
  The problem is how to represent the problem with an actual parametrized model, i.e. a neural network in this case. I see two approaches:
  
  1. Discretize the direction and magnitude and use a softmax layer for the outputs.
  2. Use some form of circular regression.
  
  For (2) I have no idea what that would be, but I assume something like it must exist. However, how would regression handle the multimodality of the marginals and sampling during inference time? I'm also unsure about the ability of (1) to handle well the multimodality. Also if I discretize the model into lots of buckets, then the output layer of the network also grows quite a bit.
  
  My question is essentially if more principled approaches exist and especially under the following "real-world" assumptions:
  
  1. There is lots of data, i.e. from an actual multiplayer game.
  2. The controls are fine-grained enough that you require a high number of buckets (e.g. precise aiming).
  
  **Tags:** regression, supervised-learning, behavioral-cloning
  > **Comments:**
  > **username_1**: maybe a generative approach might help  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-02-05. Score: 1
  
  If I'm interpreting this correctly this sounds like a question of output activation functions. Here's how I would formulate it:
  
  * Direction which is RELU
  * Magnitude which is RELU, could be clipped if that's interesting to whatever control style.
  
  The direction output can be interpreted as radians and converted to degrees from Top Dead Center (TDC). I'd be tempted when converting expert play to targets to start counting at 2pi so as an expert swings counter clockwise there isn't a temptation to jump from 0 to 2pi as it crosses TDC. Not sure on the exact effect here but it feels nice to the underlying function approximator.
  
  
  ---
  
  