Answers.Body: <p>There are two machine learning research fields that are related to your problem: one is <strong>curriculum learning</strong>, the other is <strong>active learning</strong>.</p>
  <h3>Curriculum Learning</h3>
  <p>The basic idea is to <em>manually</em> design increasingly difficult training tasks that aim to guide the learning of the network towards a better local minima. Say you have a very hard problem, from that you build a curriculum: initially it contains easy and simplified versions of your initial problem, and as the training progresses you switch to a more difficult curriculum, and keep doing this until the difficulty of the training tasks match or even surpasses the one of the original problem.</p>
  <p>Such curriculum learning approach has been shown to be beneficial both in deep learning and reinforcement learning problems.</p>
  <p>Actually there is much interest in <em>automatically building the curriculum</em>: there are some example of this in the literature, allowing to automatically generate a curriculum according to the network's training performance. This would, at least in principle, solve the issue about how to properly design an effective curriculum which is not trivial in some cases.</p>
  <p>References:</p>
  <ul>
  <li><a href="https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/15972/Bengio%2C%202009%20Curriculum%20Learning.pdf?sequence=1&amp;isAllowed=y" rel="nofollow noreferrer">Curriculum learning</a></li>
  <li><a href="http://proceedings.mlr.press/v70/graves17a/graves17a.pdf" rel="nofollow noreferrer">Automated Curriculum Learning for Neural Networks</a></li>
  </ul>
  <h3>Active Learning</h3>
  <p>The basic idea is to let the training algorithm be helped by a user from time to time.</p>
  <p>A use case of this is training data labeling, in which in an online fashion a user labels or even scores data points that are poorly predicted (or classified) by the model. In this regard, a working approach is called <strong>DAgger</strong> (dataset aggregation) that is used in imitation learning: basically a human manually label data with the best action to take (as the goal of the problem is to learn a policy - so learn best actions that are unknown.)</p>
  <p>References:</p>
  <ul>
  <li><a href="https://en.wikipedia.org/wiki/Active_learning_(machine_learning)" rel="nofollow noreferrer">Active Learning</a></li>
  <li><a href="https://arxiv.org/abs/1011.0686" rel="nofollow noreferrer">Dagger</a></li>
  </ul>
  <p><strong>The answer to your question is:</strong> in principle yes, you can speed up training by either manual or automatic guidance of the training, or by manually providing better labels or training data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-25T08:20:57.240Z
Answers.Text: @user253751 Well, there is also InstructGPT that is trained by reinforcement learning from human feedback of instruction following tasks
Answers.UserDisplayName: Luca Anzalone
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-22T08:09:09.277Z
Answers.LastActivityDate: 2023-04-22T08:09:09.277Z
Answers.UserDisplayName: Luca Anzalone
Body: <p>I have not found any neural network training methods that recommend manually intervening in the training process while it is happening.  However, some experiments I've done seem to show this can be an effective way to speed up training.</p>
  <p>For example, once the network has converged on a suboptimal solution, I can have the network focus on particular sections of the training data for awhile, and then switch back to the entire training data to get the network out of the local optima.  Gradient descent alone would not be able to do this because while I have the network focus on a subset of the training data the error becomes very large, and then after I release the focus the error drops below the previous local optima value.  Gradient descent would not choose to explore an area of the solution space with a significantly higher error than the current region.</p>
  <p>Has there been any research into whether these kinds of manual intervention methods can improve over purely automated training?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-23T05:43:45.333Z
Comments.Text: Do you have reasons to believe that your intervention methods cannot be automated?  Generally, when someone discovers a method by which they can improve a learning algorithm, they automate it.  Certainly vanilla gradient descent barely scratches the surface of the breadth of available automated optimization schemes.
Comments.UserDisplayName: Him
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-26T22:43:58.967Z
Comments.Text: @Him if they were, then we'd already have AGI.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-27T17:26:49.550Z
Comments.Text: Something tells me that the manual intervention method you have found is not the missing link to AGI.  My point here is that you may simply be googling the wrong thing:  You are looking for "how to use manual intervention to improve nn performance", but really what you've found is not that manual intervention helps, but that actively focused training helps.  There is almost certainly an abundance of research on this topic, and I would wager than approximately 0% of it suggests doing it manually.
Comments.UserDisplayName: Him
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-27T17:49:39.317Z
Comments.Text: @Him what I mean is that humans have a pattern identification ability that has yet to be fully captured algorithmically.  So, makes sense to let humans manually intervene when they spot patterns that haven't been captured algorithmically yet.  Also, a lot more accessible than understanding the sophisticated optimization algorithms.
Comments.UserDisplayName: yters
Tags: <neural-networks><training><human-computer-interaction>
Title: Can I speed up NN training by manually guiding training?
UserDisplayName: yters
ThreadText: # Can I speed up NN training by manually guiding training?
  
  (Asked by: username_0 on 2023-04-22)
  
  I have not found any neural network training methods that recommend manually intervening in the training process while it is happening. However, some experiments I've done seem to show this can be an effective way to speed up training.
  
  For example, once the network has converged on a suboptimal solution, I can have the network focus on particular sections of the training data for awhile, and then switch back to the entire training data to get the network out of the local optima. Gradient descent alone would not be able to do this because while I have the network focus on a subset of the training data the error becomes very large, and then after I release the focus the error drops below the previous local optima value. Gradient descent would not choose to explore an area of the solution space with a significantly higher error than the current region.
  
  Has there been any research into whether these kinds of manual intervention methods can improve over purely automated training?
  
  **Tags:** neural-networks, training, human-computer-interaction
  
  > **Comments:**
  > **username_1**: Do you have reasons to believe that your intervention methods cannot be automated?  Generally, when someone discovers a method by which they can improve a learning algorithm, they automate it.  Certainly vanilla gradient descent barely scratches the surface of the breadth of available automated optimization schemes.  
  > **username_0**: @username_1 if they were, then we'd already have AGI.  
  > **username_1**: Something tells me that the manual intervention method you have found is not the missing link to AGI.  My point here is that you may simply be googling the wrong thing:  You are looking for "how to use manual intervention to improve nn performance", but really what you've found is not that manual intervention helps, but that actively focused training helps.  There is almost certainly an abundance of research on this topic, and I would wager than approximately 0% of it suggests doing it manually.  
  > **username_0**: @username_1 what I mean is that humans have a pattern identification ability that has yet to be fully captured algorithmically.  So, makes sense to let humans manually intervene when they spot patterns that haven't been captured algorithmically yet.  Also, a lot more accessible than understanding the sophisticated optimization algorithms.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-04-22. Score: 3
  
  There are two machine learning research fields that are related to your problem: one is **curriculum learning**, the other is **active learning**.
  
  ### Curriculum Learning
  
  The basic idea is to *manually* design increasingly difficult training tasks that aim to guide the learning of the network towards a better local minima. Say you have a very hard problem, from that you build a curriculum: initially it contains easy and simplified versions of your initial problem, and as the training progresses you switch to a more difficult curriculum, and keep doing this until the difficulty of the training tasks match or even surpasses the one of the original problem.
  
  Such curriculum learning approach has been shown to be beneficial both in deep learning and reinforcement learning problems.
  
  Actually there is much interest in *automatically building the curriculum*: there are some example of this in the literature, allowing to automatically generate a curriculum according to the network's training performance. This would, at least in principle, solve the issue about how to properly design an effective curriculum which is not trivial in some cases.
  
  References:
  
  * [Curriculum learning](https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/15972/Bengio%2C%202009%20Curriculum%20Learning.pdf?sequence=1&isAllowed=y)
  * [Automated Curriculum Learning for Neural Networks](http://proceedings.mlr.press/v70/graves17a/graves17a.pdf)
  
  ### Active Learning
  
  The basic idea is to let the training algorithm be helped by a user from time to time.
  
  A use case of this is training data labeling, in which in an online fashion a user labels or even scores data points that are poorly predicted (or classified) by the model. In this regard, a working approach is called **DAgger** (dataset aggregation) that is used in imitation learning: basically a human manually label data with the best action to take (as the goal of the problem is to learn a policy - so learn best actions that are unknown.)
  
  References:
  
  * [Active Learning](https://en.wikipedia.org/wiki/Active_learning_(machine_learning))
  * [Dagger](https://arxiv.org/abs/1011.0686)
  
  **The answer to your question is:** in principle yes, you can speed up training by either manual or automatic guidance of the training, or by manually providing better labels or training data.
  
  > **Comments:**
  > **username_2**: @user253751 Well, there is also InstructGPT that is trained by reinforcement learning from human feedback of instruction following tasks  
  
  ---
  
  