<p>If we have a working reward function, providing the desired behavior and optimal policy in a continuous action/state-space problem, would adding another action significantly affect the possible optimal policy?</p>
<p>For example, assume you have an RL problem with an action space of 1 (de/acceleration), state-space of 2 (distance from position and velocity), and the agent is tasked to accelerate in a straight line from position a to b.</p>
<p>Do you think the agent would behave majorly differently? I'm under the assumption that there would be minimal change aside from a longer training time assuming enough exploration, as the task is to still move in a straight line, but the agent would only have to account for steering action too now.</p>

CC BY-SA 4.0
2022-05-15T13:12:00.663Z
Hi and welcome to AI Stack Exchange. Could you clarify the sequence of events - when you will train the agent, when you make the change to environment? Also what you mean by "effect" - do you expect the trained agent to still work, or that re-training should be guaranteed to work after the change, or something else?
Neil Slater
CC BY-SA 4.0
2022-05-15T13:16:56.757Z
Re-reading your question, it is also not clear what the new action is that you propose to add, and how you would propose to add an action in a new dimension (e.g. steering) without tracking that dimension in the state.
Neil Slater
CC BY-SA 4.0
2022-05-16T00:06:21.063Z
Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Community
<reinforcement-learning><deep-rl><reward-functions><action-spaces><td3>
If we have a working reward function, would adding another action have a significant effect on the agent performance if task remains the same?
Philori
# If we have a working reward function, would adding another action have a significant effect on the agent performance if task remains the same?

(Asked by: username_0 on 2022-05-15)

If we have a working reward function, providing the desired behavior and optimal policy in a continuous action/state-space problem, would adding another action significantly affect the possible optimal policy?

For example, assume you have an RL problem with an action space of 1 (de/acceleration), state-space of 2 (distance from position and velocity), and the agent is tasked to accelerate in a straight line from position a to b.

Do you think the agent would behave majorly differently? I'm under the assumption that there would be minimal change aside from a longer training time assuming enough exploration, as the task is to still move in a straight line, but the agent would only have to account for steering action too now.

**Tags:** reinforcement-learning, deep-rl, reward-functions, action-spaces, td3

> **Comments:**
> **username_1**: Hi and welcome to AI Stack Exchange. Could you clarify the sequence of events - when you will train the agent, when you make the change to environment? Also what you mean by "effect" - do you expect the trained agent to still work, or that re-training should be guaranteed to work after the change, or something else?  
> **username_1**: Re-reading your question, it is also not clear what the new action is that you propose to add, and how you would propose to add an action in a new dimension (e.g. steering) without tracking that dimension in the state.  
> **username_2**: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.  

---

## 0 answers

