Body: <p>I'm working on a reinforcement learning problem where the environment returns a reward pair <span class="math-container">$(r_{t+1}^{(a)}, r_{t+1}^{(b)})$</span>. The goal is to maximize the following nonlinear objective function.
  <span class="math-container">$$
  E[\lim_{T \to  \infty } \frac{ \sum_{r=t}^{t+T-1} r_{r+1}^{(a)}}{\sum_{r=t}^{t+T-1} r_{r+1}^{(a)} + \sum_{r=t}^{t+T-1} r_{r+1}^{(b)}}]
  $$</span>
  which is the ratio of the agent's cumulative rewards as a fraction of the total cumulative rewards. My intention is to employ Deep Q-Networks as the primary model for reinforcement learning to solve this environment. However, due to the non-linear nature of the objective function, I encountered challenges in applying the original DQN algorithm. As a result, I explored an alternative approach by framing the problem as a multi-objective reinforcement learning problem. I seek validation regarding the appropriateness of this decision.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-13T16:51:19.213Z
Comments.Text: _agent's cumulative rewards as a fraction of the total cumulative rewards_ is it a multi agent env or what you mean with "total"
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-13T20:31:24.713Z
Comments.Text: @Alberto, I think the "total" is over the "rewards" $r^{(a)} + r^{(b)}$. These appear not to be rewards in the usual MDP model sense, but some other observed values that are taking a similar role to MDP reward.
Comments.UserDisplayName: Neil Slater
Tags: <neural-networks><reinforcement-learning><q-learning><objective-functions><multi-objective-rl>
Title: Optimizing a nonlinear objective function in Deep Reinforcement Learning
UserDisplayName: Alex
ThreadText: # Optimizing a nonlinear objective function in Deep Reinforcement Learning
  
  (Asked by: username_0 on 2024-05-13)
  
  I'm working on a reinforcement learning problem where the environment returns a reward pair $(r\_{t+1}^{(a)}, r\_{t+1}^{(b)})$. The goal is to maximize the following nonlinear objective function.
  $$
  E[\lim\_{T \to \infty } \frac{ \sum\_{r=t}^{t+T-1} r\_{r+1}^{(a)}}{\sum\_{r=t}^{t+T-1} r\_{r+1}^{(a)} + \sum\_{r=t}^{t+T-1} r\_{r+1}^{(b)}}]
  $$
  which is the ratio of the agent's cumulative rewards as a fraction of the total cumulative rewards. My intention is to employ Deep Q-Networks as the primary model for reinforcement learning to solve this environment. However, due to the non-linear nature of the objective function, I encountered challenges in applying the original DQN algorithm. As a result, I explored an alternative approach by framing the problem as a multi-objective reinforcement learning problem. I seek validation regarding the appropriateness of this decision.
  
  **Tags:** neural-networks, reinforcement-learning, q-learning, objective-functions, multi-objective-rl
  
  > **Comments:**
  > **username_1**: _agent's cumulative rewards as a fraction of the total cumulative rewards_ is it a multi agent env or what you mean with "total"  
  > **username_2**: @username_1, I think the "total" is over the "rewards" $r^{(a)} + r^{(b)}$. These appear not to be rewards in the usual MDP model sense, but some other observed values that are taking a similar role to MDP reward.  
  
  ---
  
  ## 0 answers
  
  