Body: <p><a href="https://i.sstatic.net/Yvf9w.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Yvf9w.png" alt="enter image description here" /></a></p>
  <p>i am work on a project.i use the dqn to maximize return. this picture are some env states.</p>
  <p>i found that dqn did learn a bit, but after a while it stopped improving and even started to decline.</p>
  <p>this my current Hyperparameters:lr 0.01,lr_decay 9e-7,mineps 0.1, maxeps 1,epsdecay 1/25000,memory buffer 200000,discount 0.999,Weight_update_frequency 20step,batch_size 16.</p>
  <p>i try a lot.i use the lr:0.001,0.01,0.0001,used eps:0.05,0,02. the lr_decay:1e-7,1.5e-7,1.2e-7,eps_decay:1/10000,1/20000,1/25000,1/40000,1/50000.Weight_update_frequency:4epoch,20step.batch_size:16,32,64.the model size:64,64,128,32,32,64.</p>
  <p>Its trend is almost the same, and it has learned something, but then the shaking is very serious, or even declines.
  <a href="https://i.sstatic.net/EuZs7.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/EuZs7.png" alt=" c" /></a></p>
  <p>this is current training figure, anyway,it is better than most of my other training figure.</p>
  <p>ADD:
  source of my dqn is from a book AI-Crash-Course,this is the github site <a href="https://github.com/PacktPublishing/AI-Crash-Course/tree/master/Chapter%2011" rel="nofollow noreferrer">https://github.com/PacktPublishing/AI-Crash-Course/tree/master/Chapter%2011</a>.</p>
  <p>the code is in chapter 11.i made Some changes  on the basis of these codes.</p>
  <p>about environment:my project is divided in two parts,the schedule algorithm and the DQN part.env will produce many tasks.the schedule algorithm will schedule tasks to a device for processing.The dqn is used to give the best device id.</p>
  <p>When a ready task is solved,the environment will change,then the schedule algorithm send the state、reward to DQN.DQN send the device id to schedule algorithm.</p>
  <p>action is device id.Range from 0 to 5.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-18T12:29:38.110Z
Comments.Text: What implementation do you use for DQN ? Is it something you wrote yourself, Tensorflow, something else ? People could probably help if you gave some link toward the detail of those parameters, some documentation.
Comments.UserDisplayName: nathan raynal
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-18T12:31:51.523Z
Comments.Text: Also, could you give some information about the markov decision process you're working on ? What's the environment, what kind of actions the agent can take ? All of these might help to find the right parameters.
Comments.UserDisplayName: nathan raynal
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-19T09:43:24.547Z
Comments.Text: I don't personnally have enough knowledge of DQN yet to go through this code and check for errors, but my suggestion would be to try what you did on another implementation of DQN such as the Tensorflow one. That way you'll know if's it's a parameter issue or an implementation issue.
Comments.UserDisplayName: nathan raynal
Tags: <deep-learning><deep-rl>
Title: Deep reinforcement learning:DQN can't converge
UserDisplayName: learner
ThreadText: # Deep reinforcement learning:DQN can't converge
  
  (Asked by: username_0 on 2022-09-16)
  
  [![enter image description here](https://i.sstatic.net/Yvf9w.png)](https://i.sstatic.net/Yvf9w.png)
  
  i am work on a project.i use the dqn to maximize return. this picture are some env states.
  
  i found that dqn did learn a bit, but after a while it stopped improving and even started to decline.
  
  this my current Hyperparameters:lr 0.01,lr\_decay 9e-7,mineps 0.1, maxeps 1,epsdecay 1/25000,memory buffer 200000,discount 0.999,Weight\_update\_frequency 20step,batch\_size 16.
  
  i try a lot.i use the lr:0.001,0.01,0.0001,used eps:0.05,0,02. the lr\_decay:1e-7,1.5e-7,1.2e-7,eps\_decay:1/10000,1/20000,1/25000,1/40000,1/50000.Weight\_update\_frequency:4epoch,20step.batch\_size:16,32,64.the model size:64,64,128,32,32,64.
  
  Its trend is almost the same, and it has learned something, but then the shaking is very serious, or even declines.
  [![ c](https://i.sstatic.net/EuZs7.png)](https://i.sstatic.net/EuZs7.png)
  
  this is current training figure, anyway,it is better than most of my other training figure.
  
  ADD:
  source of my dqn is from a book AI-Crash-Course,this is the github site <https://github.com/PacktPublishing/AI-Crash-Course/tree/master/Chapter%2011>.
  
  the code is in chapter 11.i made Some changes on the basis of these codes.
  
  about environment:my project is divided in two parts,the schedule algorithm and the DQN part.env will produce many tasks.the schedule algorithm will schedule tasks to a device for processing.The dqn is used to give the best device id.
  
  When a ready task is solved,the environment will change,then the schedule algorithm send the state、reward to DQN.DQN send the device id to schedule algorithm.
  
  action is device id.Range from 0 to 5.
  
  **Tags:** deep-learning, deep-rl
  > **Comments:**
  > **username_1**: What implementation do you use for DQN ? Is it something you wrote yourself, Tensorflow, something else ? People could probably help if you gave some link toward the detail of those parameters, some documentation.  
  > **username_1**: Also, could you give some information about the markov decision process you're working on ? What's the environment, what kind of actions the agent can take ? All of these might help to find the right parameters.  
  > **username_1**: I don't personnally have enough knowledge of DQN yet to go through this code and check for errors, but my suggestion would be to try what you did on another implementation of DQN such as the Tensorflow one. That way you'll know if's it's a parameter issue or an implementation issue.  
  
  ---
  
  ## 0 answers
  
  