<p>There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include:</p>

<ul>
<li>Probabalistically accepting worse solutions in the hope that this
will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing). </li>
<li>Maintaining a list of recently-encountered states (or attributes thereof) and not returning
to a recently-encountered one (like Tabu Search). </li>
<li>Performing a random walk of a length determined by the current state of the search (an explicit 'Diversification strategy', e.g. as used in 'Reactive Tabu Search').</li>
</ul>

<p>See the excellent (and free online) book <a href="https://cs.gmu.edu/~sean/book/metaheuristics/">'Essentials of Metaheuristics'</a> by Sean Luke for more details on these kind of techniques and some rules of thumb about when and how to use them.</p>

CC BY-SA 3.0
2016-08-05T11:00:14.427Z
2016-08-05T11:00:14.427Z
NietzscheanAI
<p>How do I avoid my gradient descent algorithm into falling into the "local minima" trap while backpropogating on my neural network?</p>

<p>Are there any methods which help me avoid it?</p>

<neural-networks><backpropagation><optimization><gradient-descent>
How to avoid falling into the "local minima" trap?
Dawny33
# How to avoid falling into the "local minima" trap?

(Asked by: username_0 on 2016-08-05)

How do I avoid my gradient descent algorithm into falling into the "local minima" trap while backpropogating on my neural network?

Are there any methods which help me avoid it?

**Tags:** neural-networks, backpropagation, optimization, gradient-descent


---

## 1 answers

### Answer by username_1 on 2016-08-05. Score: 9 | Accepted answer

There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include:

* Probabalistically accepting worse solutions in the hope that this
  will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing).
* Maintaining a list of recently-encountered states (or attributes thereof) and not returning
  to a recently-encountered one (like Tabu Search).
* Performing a random walk of a length determined by the current state of the search (an explicit 'Diversification strategy', e.g. as used in 'Reactive Tabu Search').

See the excellent (and free online) book ['Essentials of Metaheuristics'](https://cs.gmu.edu/~sean/book/metaheuristics/) by Sean Luke for more details on these kind of techniques and some rules of thumb about when and how to use them.


---

