Answers.Body: <p>Your question is difficult to answer because it is rather vague. Whether or not you can train an agent on your data depends on a few things. The two main things are: how much data do you have and how well do they represent the problem.</p>
  <p>The first thing to do is to model the environment. What is your action space? What are potential states? What is the goal you want to optimise to? The answer to these questions determine what policy to use to reach that set goal. e-greedy is one of the simplest policies. I'd try that first and see how it performs. Even if it performs badly, you can find out why and then pick a better policy that will overcome these short comings.</p>
  <p>As for your last question, it shouldn't create a problem if you model this aspect well. The goal itself, if well defined, would not shift. The performance of the user does and with every iteration the gap between performance of the user and goal should decrease.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-24T04:43:43.177Z
Answers.LastActivityDate: 2022-05-24T04:43:43.177Z
Answers.UserDisplayName: emely_pi
Body: <p>I have a scenario where a user is wanting to exercise and improve over time. They attend around 10 exercise sessions, doing 20 repititions of an exercise each session.</p>
  <p>I want to develop a reinforcement learning agent to help the user get better at the exercises. The agent will observe the user exercising, and be rewarded when the user performs well (say, by completing a lot of repititions). The agent will then take actions to change the exercise routines to adapt to the skill level of the user.</p>
  <p>Furthermore:</p>
  <ul>
  <li>the RL agent will have no knowledge of the user's skill level before entering the first session</li>
  <li>the user's skill level should increase from session to session, but at an unknown rate</li>
  </ul>
  <p>I know that I must employ model-free RL methods, as the agent must be able to create unique policies for each user it works with. But I have three questions:</p>
  <ol>
  <li>How do I determine what initial policy to follow during the first sessions?</li>
  <li>I realise this problem description is vague, but generally speaking, would this be enough data for the agent to work with to create a policy quickly that performs well?</li>
  <li>Does the fact the user gets better over time create a problem for the RL agent? As it would be like chasing a moving target?</li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-24T09:26:50.497Z
Comments.Text: Please, **ask only one question per post**. Edit your post to fix that and put your main specific question in the title. "Using reinforcement learning for human-robot interaction" is not a question.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><q-learning><on-policy-methods>
Title: Using reinforcement learning for human-robot interaction
UserDisplayName: caaax
ThreadText: # Using reinforcement learning for human-robot interaction
  
  (Asked by: username_0 on 2022-05-23)
  
  I have a scenario where a user is wanting to exercise and improve over time. They attend around 10 exercise sessions, doing 20 repititions of an exercise each session.
  
  I want to develop a reinforcement learning agent to help the user get better at the exercises. The agent will observe the user exercising, and be rewarded when the user performs well (say, by completing a lot of repititions). The agent will then take actions to change the exercise routines to adapt to the skill level of the user.
  
  Furthermore:
  
  * the RL agent will have no knowledge of the user's skill level before entering the first session
  * the user's skill level should increase from session to session, but at an unknown rate
  
  I know that I must employ model-free RL methods, as the agent must be able to create unique policies for each user it works with. But I have three questions:
  
  1. How do I determine what initial policy to follow during the first sessions?
  2. I realise this problem description is vague, but generally speaking, would this be enough data for the agent to work with to create a policy quickly that performs well?
  3. Does the fact the user gets better over time create a problem for the RL agent? As it would be like chasing a moving target?
  
  **Tags:** reinforcement-learning, q-learning, on-policy-methods
  
  > **Comments:**
  > **username_1**: Please, **ask only one question per post**. Edit your post to fix that and put your main specific question in the title. "Using reinforcement learning for human-robot interaction" is not a question.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-05-24. Score: 1
  
  Your question is difficult to answer because it is rather vague. Whether or not you can train an agent on your data depends on a few things. The two main things are: how much data do you have and how well do they represent the problem.
  
  The first thing to do is to model the environment. What is your action space? What are potential states? What is the goal you want to optimise to? The answer to these questions determine what policy to use to reach that set goal. e-greedy is one of the simplest policies. I'd try that first and see how it performs. Even if it performs badly, you can find out why and then pick a better policy that will overcome these short comings.
  
  As for your last question, it shouldn't create a problem if you model this aspect well. The goal itself, if well defined, would not shift. The performance of the user does and with every iteration the gap between performance of the user and goal should decrease.
  
  
  ---
  
  