Body: <p>Suppose we are training an environment with 2 collaborative agents with Reinforcement Learning. We define the following example: There is a midfielder and a striker. The midfielder's reward depends on how many goals are scored, which however depends on the attacker's performance. And the striker's performance depends on how good the midfielder is at making his passes.</p>
  <p>For this type of problem, what do you recommend to study?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-20T07:35:34.007Z
Comments.Text: What do you already know about RL? Is there an absolute requirement in this case that the two agents are implemented separately? Could you outline the learning challenges faced by the midfielder and striker - is it really just a matter of accuracy in two specific skills, with no other interdependence?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-20T15:42:25.017Z
Comments.Text: The performance being dependent on the other agent should not end up mattering that much. They will learn to do the best they can with whatever the other agent is doing and hopefully they will learn to work together. If you are trying to play some form of soccer, then that will need a pretty advanced method because of the complexity. Since you have a continuous state space and probably a mix of continuous and discrete action space (movement is continuous, pass or not pass is discrete), I would suggest an advanced form of policy gradients like PPO.
Comments.UserDisplayName: S2673
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-20T15:51:32.060Z
Comments.Text: spinningup.openai.com is a good place to learn about reinforcement learning methods.
Comments.UserDisplayName: S2673
Tags: <reinforcement-learning><reference-request>
Title: Which reinforcement learning approach to use when there are 2 collaborative agents?
UserDisplayName: Vel
ThreadText: # Which reinforcement learning approach to use when there are 2 collaborative agents?
  
  (Asked by: username_0 on 2020-10-20)
  
  Suppose we are training an environment with 2 collaborative agents with Reinforcement Learning. We define the following example: There is a midfielder and a striker. The midfielder's reward depends on how many goals are scored, which however depends on the attacker's performance. And the striker's performance depends on how good the midfielder is at making his passes.
  
  For this type of problem, what do you recommend to study?
  
  **Tags:** reinforcement-learning, reference-request
  
  > **Comments:**
  > **username_1**: What do you already know about RL? Is there an absolute requirement in this case that the two agents are implemented separately? Could you outline the learning challenges faced by the midfielder and striker - is it really just a matter of accuracy in two specific skills, with no other interdependence?  
  > **username_2**: The performance being dependent on the other agent should not end up mattering that much. They will learn to do the best they can with whatever the other agent is doing and hopefully they will learn to work together. If you are trying to play some form of soccer, then that will need a pretty advanced method because of the complexity. Since you have a continuous state space and probably a mix of continuous and discrete action space (movement is continuous, pass or not pass is discrete), I would suggest an advanced form of policy gradients like PPO.  
  > **username_2**: spinningup.openai.com is a good place to learn about reinforcement learning methods.  
  
  ---
  
  ## 0 answers
  
  