Body: <p>I'm working on a traffic signal control problem, which I am currently approaching with Reinforcement Learning, but I want to try some other control algorithms.</p>
  <p>This is hard for me because we don't have a model of the system:</p>
  <ol>
  <li><p>The number of incoming vehicles is stochastic, it depends on the time of the day and the particular scenario</p>
  </li>
  <li><p>The number of outgoing vehicles is also stochastic: when we give green to an approach, the number of vehicles that go through depends on how many are already waiting (measurable) the current position and speed of other vehicles on that road unmeasurable), the incoming flow (stochastic)</p>
  </li>
  </ol>
  <p>On the other hand, the state space and action space are small and discrete, so I guess they can be treated by many control algorithms.</p>
  <p>Could you point me to some algorithms or principles in control theory that could be suited for such a problem? I am also interested in algorithms that merge the learning capabilities of RL and the more robust and stable properties of control algorithms.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-08T03:47:58.457Z
Comments.Text: https://en.wikipedia.org/wiki/Stochastic_control#:~:text=Stochastic%20control%20or%20stochastic%20optimal,the%20evolution%20of%20the%20system.
Comments.UserDisplayName: insipidintegrator
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-08T23:21:51.563Z
Comments.Text: Check also https://en.wikipedia.org/wiki/Queueing_theory
Comments.UserDisplayName: Rexcirus
Tags: <reinforcement-learning><reference-request><pomdp><planning><control-theory>
Title: Control algorithms when system dynamics are stochastic and/or unknown
UserDisplayName: Federico Taschin
ThreadText: # Control algorithms when system dynamics are stochastic and/or unknown
  
  (Asked by: username_0 on 2024-02-07)
  
  I'm working on a traffic signal control problem, which I am currently approaching with Reinforcement Learning, but I want to try some other control algorithms.
  
  This is hard for me because we don't have a model of the system:
  
  1. The number of incoming vehicles is stochastic, it depends on the time of the day and the particular scenario
  2. The number of outgoing vehicles is also stochastic: when we give green to an approach, the number of vehicles that go through depends on how many are already waiting (measurable) the current position and speed of other vehicles on that road unmeasurable), the incoming flow (stochastic)
  
  On the other hand, the state space and action space are small and discrete, so I guess they can be treated by many control algorithms.
  
  Could you point me to some algorithms or principles in control theory that could be suited for such a problem? I am also interested in algorithms that merge the learning capabilities of RL and the more robust and stable properties of control algorithms.
  
  **Tags:** reinforcement-learning, reference-request, pomdp, planning, control-theory
  > **Comments:**
  > **username_1**: https://en.wikipedia.org/wiki/Stochastic_control#:~:text=Stochastic%20control%20or%20stochastic%20optimal,the%20evolution%20of%20the%20system.  
  > **username_2**: Check also https://en.wikipedia.org/wiki/Queueing_theory  
  
  ---
  
  ## 0 answers
  
  