Answers.Body: <p>Not particularly sure what you are asking, so the question that I will be answering is this: </p>
  
  <blockquote>
    <p>Can Q learning be used to estimate a value that depends on another value in the Q Learning Matrix even though there is a certain amount of unpredictability involved?</p>
  </blockquote>
  
  <p>The answer is <strong>yes!</strong></p>
  
  <p>I will use the example of a robotic arm trying to reach a point in space since that is what I am most familiar with. </p>
  
  <p>Imagine a robotic arm with a shoulder, elbow, and wrist joint. The desired elbow value depends very much on the shoulder value which is also being learned. Given enough iterations, the Q learning algorithm will come up with a solution (out of possibly many) for the elbow joint based on where the shoulder joint is at that time. </p>
  
  <p>The intrinsic unpredictability (air drag, motor power, etc in this case) is countered intuitively by the Q-learning algorithm iteratively learning what works best. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-18T16:57:58.577Z
Answers.Text: I know the question is difficult to understand with less context. But the way you have framed the question is exactly what I was trying to ask "unpredictability". Thanks! Could you add a few more comments as to where or in which environment will Q-learning fail?
Answers.UserDisplayName: amitection
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-18T17:26:10.567Z
Answers.Text: I am not an expert in the limits of reinforcement learning, but I know that time can be limiting factor. It takes a long time to train and will go through many iterations before the output becomes useful.
Answers.UserDisplayName: Joe S
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-18T14:59:10.010Z
Answers.LastActivityDate: 2018-06-18T14:59:10.010Z
Answers.UserDisplayName: Joe S
Body: <p>I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state.</p>
  
  <p>The central service dispatches some information at regular intervals to all the agents (Lets call this information as <strong>energy</strong>). The information can be very different for all the agents. </p>
  
  <p>The agents on reception of this information select a particular action. The execution of the action should leave the agent as well as the environment in a positive state. The action requires a limited amount of energy which might change on every timestep. If a agent does not have sufficient energy to it may request for energy from other agents. The other agents may grant or deny this request.</p>
  
  <p>If all the agents are able to successfully perform their actions and leave the environment in a positive state they get a positive reward.</p>
  
  <p>As the environment is stochastic, where a agent's behavior is dependent on another agent can approximate Q Learning be used here?</p>
  
Tags: <reinforcement-learning><q-learning><multi-agent-systems>
Title: Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?
UserDisplayName: amitection
ThreadText: # Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?
  
  (Asked by: username_0 on 2018-06-18)
  
  I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state.
  
  The central service dispatches some information at regular intervals to all the agents (Lets call this information as **energy**). The information can be very different for all the agents.
  
  The agents on reception of this information select a particular action. The execution of the action should leave the agent as well as the environment in a positive state. The action requires a limited amount of energy which might change on every timestep. If a agent does not have sufficient energy to it may request for energy from other agents. The other agents may grant or deny this request.
  
  If all the agents are able to successfully perform their actions and leave the environment in a positive state they get a positive reward.
  
  As the environment is stochastic, where a agent's behavior is dependent on another agent can approximate Q Learning be used here?
  
  **Tags:** reinforcement-learning, q-learning, multi-agent-systems
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-06-18. Score: 2 | Accepted answer
  
  Not particularly sure what you are asking, so the question that I will be answering is this:
  
  > Can Q learning be used to estimate a value that depends on another value in the Q Learning Matrix even though there is a certain amount of unpredictability involved?
  
  The answer is **yes!**
  
  I will use the example of a robotic arm trying to reach a point in space since that is what I am most familiar with.
  
  Imagine a robotic arm with a shoulder, elbow, and wrist joint. The desired elbow value depends very much on the shoulder value which is also being learned. Given enough iterations, the Q learning algorithm will come up with a solution (out of possibly many) for the elbow joint based on where the shoulder joint is at that time.
  
  The intrinsic unpredictability (air drag, motor power, etc in this case) is countered intuitively by the Q-learning algorithm iteratively learning what works best.
  
  > **Comments:**
  > **username_0**: I know the question is difficult to understand with less context. But the way you have framed the question is exactly what I was trying to ask "unpredictability". Thanks! Could you add a few more comments as to where or in which environment will Q-learning fail?  
  > **username_1**: I am not an expert in the limits of reinforcement learning, but I know that time can be limiting factor. It takes a long time to train and will go through many iterations before the output becomes useful.  
  
  ---
  
  