Answers.Body: <p>If possible, I would try to calculate what the (theoretical) maximum  throughput through the intersection is for a given time interval. If the control behavior that the DQN produces comes empirically close to the maximally possible throughput score, the model is good. Otherwise, you could measure and compare the throughput of different models and choose the best performing one.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-21T22:24:38.983Z
Answers.Text: What does "throughput through the intersection is for a given time interval" actually mean in this context? What is a "throughput score"?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-21T22:48:56.547Z
Answers.Text: Each intersection has some limitations with respect to how many cars can cross it eg per hour. After a certain point, there can just not go 'more' cars through it per time interval, eg due to speed limitations. 
  Given reasonable simulation assumptions, the maximal number of cars that can go through the intersection per hour can be calculated. This measure can then serve as your gold standard to which you compare the results obtained from your DQN controller. 
  Unless, you want to evaluate it in a static scene, which seems odd to me to do, since it wouldn't assess the controller's dynamics.
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-21T23:02:00.103Z
Answers.Text: Some simulation packages that could come handy (also for training DQN on Traffic light control, btw): 
  https://sumo.dlr.de/pdf/dkrajzew_MESM2002_SUMO.pdf
  https://www.matsim.org/
  https://carla.org/
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-21T23:56:15.637Z
Answers.Text: Well, to be honest, the details of the OP's problem and model aren't clear because the part "My DQN model outputs the best traffic light state in an intersection" isn't very explicit and descriptive enough. Anyway, if I understand you correctly, your suggestion is to get some kind of constraint for each situation.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-22T07:29:40.350Z
Answers.Text: DQN is a Reinforcement Learning algorithm which is trained on solving a particular task in a given environment. Regardless of how ambiguous the original question is, for testing whether DQN does a good job or not, it is not enough to give it e.g. 1000 scene images and consider what the output is, eg because you could assess the respective output traffic light state, but not how long it would decide to remain in it. Therefore, you won't see whether traffic congestion is gonna kick in after some time of letting the controller run (sign for a bad controller) or not (sign for good controller).
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-22T07:32:44.057Z
Answers.Text: Therefore, yes, I would set up a simulation that comes close to the (real-life) conditions of the intersection for which the controller is designed and measure things like _the number of cars passing through the intersection per hour_ or _the average waiting time per car_ in order to get some objective, quantitative results that you can compare against other models.
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-21T21:24:02.753Z
Answers.LastActivityDate: 2020-06-21T21:24:02.753Z
Answers.UserDisplayName: Daniel B.
Body: <p>My DQN model outputs the best traffic light state in an intersection. I used different values of batch size and learning rate to find the best model. How would I know if I got the optimal hyperparameter values?</p>
  
Tags: <machine-learning><reinforcement-learning><q-learning><dqn>
Title: How to validate that my DQN hyperparameters are the optimal?
UserDisplayName: JAKE
ThreadText: # How to validate that my DQN hyperparameters are the optimal?
  
  (Asked by: username_0 on 2020-06-21)
  
  My DQN model outputs the best traffic light state in an intersection. I used different values of batch size and learning rate to find the best model. How would I know if I got the optimal hyperparameter values?
  
  **Tags:** machine-learning, reinforcement-learning, q-learning, dqn
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-21. Score: 1
  
  If possible, I would try to calculate what the (theoretical) maximum throughput through the intersection is for a given time interval. If the control behavior that the DQN produces comes empirically close to the maximally possible throughput score, the model is good. Otherwise, you could measure and compare the throughput of different models and choose the best performing one.
  
  > **Comments:**
  > **username_2**: What does "throughput through the intersection is for a given time interval" actually mean in this context? What is a "throughput score"?  
  > **username_1**: Each intersection has some limitations with respect to how many cars can cross it eg per hour. After a certain point, there can just not go 'more' cars through it per time interval, eg due to speed limitations.  Given reasonable simulation assumptions, the maximal number of cars that can go through the intersection per hour can be calculated. This measure can then serve as your gold standard to which you compare the results obtained from your DQN controller.  Unless, you want to evaluate it in a static scene, which seems odd to me to do, since it wouldn't assess the controller's dynamics.  
  > **username_1**: Some simulation packages that could come handy (also for training DQN on Traffic light control, btw):  https://sumo.dlr.de/pdf/dkrajzew_MESM2002_SUMO.pdf https://www.matsim.org/ https://carla.org/  
  > **username_2**: Well, to be honest, the details of the OP's problem and model aren't clear because the part "My DQN model outputs the best traffic light state in an intersection" isn't very explicit and descriptive enough. Anyway, if I understand you correctly, your suggestion is to get some kind of constraint for each situation.  
  > **username_1**: DQN is a Reinforcement Learning algorithm which is trained on solving a particular task in a given environment. Regardless of how ambiguous the original question is, for testing whether DQN does a good job or not, it is not enough to give it e.g. 1000 scene images and consider what the output is, eg because you could assess the respective output traffic light state, but not how long it would decide to remain in it. Therefore, you won't see whether traffic congestion is gonna kick in after some time of letting the controller run (sign for a bad controller) or not (sign for good controller).  
  > **username_1**: Therefore, yes, I would set up a simulation that comes close to the (real-life) conditions of the intersection for which the controller is designed and measure things like _the number of cars passing through the intersection per hour_ or _the average waiting time per car_ in order to get some objective, quantitative results that you can compare against other models.  
  
  ---
  
  