Body: <p>I started to learn reinforcement learning a few days ago. And I want to use that to solve resource allocation problem something like given a constant number, find the best way to divide it into several real numbers each is non-negative.</p>
  
  <p>For example, to divide the number 1 into 3 real numbers, the allocation can be:</p>
  
  <p>[0.2, 0.7, 0.1]</p>
  
  <p>[0.95, 0.05, 0]
  ...</p>
  
  <p>I do not know how to represent the action space because each allocation is 3-dimensional and each dimension is real-valued and each other correlated.</p>
  
  <p>In actor-critic architecture, is it possible to have 3 outputs activated by softmax in the actor's network each represents one dimension in the allocation?</p>
  
  <hr>
  
  <p>Appended:</p>
  
  <p>There is a playlist of videos. A user can switch to the next video at any time. More buffer leads to better viewing experience but more bandwidth loss if user switches to the next video. I want to optimize the smoothness of playback with minimal bandwidth loss. At each time step, the agent decides the bandwidth allocation to download current video and the next 2 videos. So I guess the state will be the bandwidth, user's behavior and the player situation.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-02T15:06:57.830Z
Comments.Text: Does your problem naturally have states, time steps and sequences of actions? Without these things, framing it as a reinforcement learning problem can be counter-productive. So it might be worth adding if your primary goal is to solve your original problem (and maybe explain it some more, as you are stuck on representation), or to study RL?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-02T15:24:48.483Z
Comments.Text: What is the goal? If you just want to divide into 3 random parts it is an easy problem. What is a good result?
Comments.UserDisplayName: Miguel Saraiva
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-02T15:32:42.977Z
Comments.Text: I have edited the problem to make it clearer.
Comments.UserDisplayName: AlanRivers
Tags: <reinforcement-learning>
Title: How to represent action space in reinforcement learning?
UserDisplayName: AlanRivers
ThreadText: # How to represent action space in reinforcement learning?
  
  (Asked by: username_0 on 2019-06-02)
  
  I started to learn reinforcement learning a few days ago. And I want to use that to solve resource allocation problem something like given a constant number, find the best way to divide it into several real numbers each is non-negative.
  
  For example, to divide the number 1 into 3 real numbers, the allocation can be:
  
  [0.2, 0.7, 0.1]
  
  [0.95, 0.05, 0]
  ...
  
  I do not know how to represent the action space because each allocation is 3-dimensional and each dimension is real-valued and each other correlated.
  
  In actor-critic architecture, is it possible to have 3 outputs activated by softmax in the actor's network each represents one dimension in the allocation?
  
  ---
  
  Appended:
  
  There is a playlist of videos. A user can switch to the next video at any time. More buffer leads to better viewing experience but more bandwidth loss if user switches to the next video. I want to optimize the smoothness of playback with minimal bandwidth loss. At each time step, the agent decides the bandwidth allocation to download current video and the next 2 videos. So I guess the state will be the bandwidth, user's behavior and the player situation.
  
  **Tags:** reinforcement-learning
  
  > **Comments:**
  > **username_1**: Does your problem naturally have states, time steps and sequences of actions? Without these things, framing it as a reinforcement learning problem can be counter-productive. So it might be worth adding if your primary goal is to solve your original problem (and maybe explain it some more, as you are stuck on representation), or to study RL?  
  > **username_2**: What is the goal? If you just want to divide into 3 random parts it is an easy problem. What is a good result?  
  > **username_0**: I have edited the problem to make it clearer.  
  
  ---
  
  ## 0 answers
  
  