Answers.Body: <p>It's debatable whether neural networks can find a better solution than PID, if your goal is to simply keep the output around a certain reference point PID should do a perfect job pretty much. If you really want to use "intelligent" control with NN you can look into <a href="https://en.wikipedia.org/wiki/Reinforcement_learning" rel="nofollow noreferrer">reinforcement learning</a>.<br>
  Few interesting papers that directly adress your problem:  </p>
  
  <p><a href="https://people.eecs.berkeley.edu/~jordan/papers/ng-etal03.pdf" rel="nofollow noreferrer">Autonomous helicopter flight via reinforcement learning</a><br>
  <a href="http://heli.stanford.edu/papers/nips06-aerobatichelicopter.pdf" rel="nofollow noreferrer">An Application of Reinforcement Learning to Aerobatic Helicopter Flight</a><br>
  <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.465.8751&amp;rep=rep1&amp;type=pdf" rel="nofollow noreferrer">Autonomous Helicopter Control Using Reinforcement Learning Policy Search Methods</a>  </p>
  
  <p>You can also find more in the references of these papers.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-15T10:35:21.977Z
Answers.Text: Probably worth adding something that more directly answers the title question: e.g. Reinforcement Learning addresses the issue of no supervised learning training data, by generating label data (predicted utility) or gradients (changes to control policy) from observations of interactions with the environment.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-15T10:01:37.943Z
Answers.LastActivityDate: 2019-05-15T10:01:37.943Z
Answers.UserDisplayName: Brale
Body: <p>I setupped a small drone simulator using PhysX, the time step is at 200 hz, while motors update like regular ESCs (at 50 Hz). I computed the inertia matrix, tweaked a bit mass of components to be real, air drag, gravity etc. After a first partial success in tuning a PID algorithm I got bored to find and hunt perfect values, and started thinking to tune it with a NN, but then I thinked, why using PID at all if NN can find better solutions?.</p>
  
  <p>I started playing with NN, <strong>but then I realized I have no traning data.</strong></p>
  
  <p>Sure I could do</p>
  
  <pre><code>NN.Train(input,expectedOutput);
  </code></pre>
  
  <p>but what is actually the expected output? To be useful it has to be the thrust force of propellers tuned to keep input (position and orientation) stable in the desired place.</p>
  
  <p>But actually I don't (Want to) know in advance the thrust that every single propeller has.</p>
  
  <p>Since it is a simulation it is ok spending some computing time between each simulation step (eventually I'll try to optimize it later to fit it in a microcontroller)..</p>
  
  <p>So, is it possible given a regular NN implementation where I can select number of:</p>
  
  <ul>
  <li>Input neurons</li>
  <li>Hidden neurons</li>
  <li>Output neurons</li>
  </ul>
  
  <p>find a way to train my model live? I need a way to tell the NN </p>
  
  <blockquote>
    <p>hei this time you performed X keep going or go the opposite way.</p>
  </blockquote>
  
Tags: <neural-networks><training>
Title: Drone training, how to train without training data?
UserDisplayName: CoffeDeveloper
ThreadText: # Drone training, how to train without training data?
  
  (Asked by: username_0 on 2019-05-15)
  
  I setupped a small drone simulator using PhysX, the time step is at 200 hz, while motors update like regular ESCs (at 50 Hz). I computed the inertia matrix, tweaked a bit mass of components to be real, air drag, gravity etc. After a first partial success in tuning a PID algorithm I got bored to find and hunt perfect values, and started thinking to tune it with a NN, but then I thinked, why using PID at all if NN can find better solutions?.
  
  I started playing with NN, **but then I realized I have no traning data.**
  
  Sure I could do
  
  ```
  NN.Train(input,expectedOutput);
  
  ```
  
  but what is actually the expected output? To be useful it has to be the thrust force of propellers tuned to keep input (position and orientation) stable in the desired place.
  
  But actually I don't (Want to) know in advance the thrust that every single propeller has.
  
  Since it is a simulation it is ok spending some computing time between each simulation step (eventually I'll try to optimize it later to fit it in a microcontroller)..
  
  So, is it possible given a regular NN implementation where I can select number of:
  
  * Input neurons
  * Hidden neurons
  * Output neurons
  
  find a way to train my model live? I need a way to tell the NN
  
  > hei this time you performed X keep going or go the opposite way.
  
  **Tags:** neural-networks, training
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-05-15. Score: 3
  
  It's debatable whether neural networks can find a better solution than PID, if your goal is to simply keep the output around a certain reference point PID should do a perfect job pretty much. If you really want to use "intelligent" control with NN you can look into [reinforcement learning](https://en.wikipedia.org/wiki/Reinforcement_learning).  
  Few interesting papers that directly adress your problem:
  
  [Autonomous helicopter flight via reinforcement learning](https://people.eecs.berkeley.edu/~jordan/papers/ng-etal03.pdf)  
  [An Application of Reinforcement Learning to Aerobatic Helicopter Flight](http://heli.stanford.edu/papers/nips06-aerobatichelicopter.pdf)  
  [Autonomous Helicopter Control Using Reinforcement Learning Policy Search Methods](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.465.8751&rep=rep1&type=pdf)
  
  You can also find more in the references of these papers.
  
  > **Comments:**
  > **username_2**: Probably worth adding something that more directly answers the title question: e.g. Reinforcement Learning addresses the issue of no supervised learning training data, by generating label data (predicted utility) or gradients (changes to control policy) from observations of interactions with the environment.  
  
  ---
  
  