Answers.Body: <p>First you'd need to mathematically model your real environment. Probably use some differential equations.</p>
  <p>Once you have a good model, you still won't have your real case parameters. So I can see 2 different approaches:</p>
  <ol>
  <li>Theoretically + Experimentally: Empirically measure real data to try to find those parameters. (Make a simple <a href="https://en.wikipedia.org/wiki/PID_controller" rel="nofollow noreferrer">PID controller</a>)</li>
  <li>Make a robust and general policy that adapts to any parameter.</li>
  </ol>
  <p>The first alternative is very straightforward, you basically just have 3 parameters to adjust and there are clear methods to adjust it depending on your data behavior.</p>
  <p>Once you are in AI.exchange, I assume you're choosing the 2nd way. I consider it's way harder to implement, but also more fun.
  So you'll need create a highly parametrized environment, make a reinforcement learning model, train it on all different kinds of parameters (just make sure your real parameter are somewhere inside that range).</p>
  <p>If you do it all right, you should have a robust general policy that can behave well in any general condition.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-17T20:16:53.553Z
Answers.LastActivityDate: 2021-08-17T20:16:53.553Z
Answers.UserDisplayName: Andre Goulart
Body: <p>I would like some pointers, possible projects that solve conceptually similar goals, code examples or tutorials.</p>
  <p>I am trying to achieve a system that is able to start or stop ventilation of a given space based different outside and inside metrics such as humidity, temperature, time, etc. to achieve a decrease of relative humidity.</p>
  <p>Actuating the system based on simple physics gave me questionable results, this is because I am not able to model the whole dynamic of the system.</p>
  <p>I was thinking if reinforced learning could help me learn a good policy. The system should learn on real life data, actuating real ventilation, with the obvious slowness of such system.</p>
  <p>I am quite new with AI, able to comprehend and create a simple OpenAi Gym. I am not even sure if and how something like this is achievable with so limited data flow.
  I am currently recording and analyze all possible data I can measure, together with some more or less random ventilation sessions. I am sure there are better ways to do this.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-17T22:25:36.490Z
Comments.Text: RL is very bad at learning in real time, it basically only works through simulators and requires a massive amount of data unfortunately. Generalization is a fundamental problem.
Comments.UserDisplayName: FourierFlux
Tags: <gym><real-time><applications>
Title: Where to start with reinforced learning on actions and rewards sampled from slow ongoing real life system
UserDisplayName: sanyi
ThreadText: # Where to start with reinforced learning on actions and rewards sampled from slow ongoing real life system
  
  (Asked by: username_0 on 2021-08-16)
  
  I would like some pointers, possible projects that solve conceptually similar goals, code examples or tutorials.
  
  I am trying to achieve a system that is able to start or stop ventilation of a given space based different outside and inside metrics such as humidity, temperature, time, etc. to achieve a decrease of relative humidity.
  
  Actuating the system based on simple physics gave me questionable results, this is because I am not able to model the whole dynamic of the system.
  
  I was thinking if reinforced learning could help me learn a good policy. The system should learn on real life data, actuating real ventilation, with the obvious slowness of such system.
  
  I am quite new with AI, able to comprehend and create a simple OpenAi Gym. I am not even sure if and how something like this is achievable with so limited data flow.
  I am currently recording and analyze all possible data I can measure, together with some more or less random ventilation sessions. I am sure there are better ways to do this.
  
  **Tags:** gym, real-time, applications
  > **Comments:**
  > **username_1**: RL is very bad at learning in real time, it basically only works through simulators and requires a massive amount of data unfortunately. Generalization is a fundamental problem.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-08-17. Score: 1
  
  First you'd need to mathematically model your real environment. Probably use some differential equations.
  
  Once you have a good model, you still won't have your real case parameters. So I can see 2 different approaches:
  
  1. Theoretically + Experimentally: Empirically measure real data to try to find those parameters. (Make a simple [PID controller](https://en.wikipedia.org/wiki/PID_controller))
  2. Make a robust and general policy that adapts to any parameter.
  
  The first alternative is very straightforward, you basically just have 3 parameters to adjust and there are clear methods to adjust it depending on your data behavior.
  
  Once you are in AI.exchange, I assume you're choosing the 2nd way. I consider it's way harder to implement, but also more fun.
  So you'll need create a highly parametrized environment, make a reinforcement learning model, train it on all different kinds of parameters (just make sure your real parameter are somewhere inside that range).
  
  If you do it all right, you should have a robust general policy that can behave well in any general condition.
  
  
  ---
  
  