Answers.Body: <p>Note that you can't really predict whether your escape from a local minimum will work or not - you might just wind up in another, worse local minimum. The probability function you describe increases the likelihood of this happening. By upweighting the likelihood of allowing small energy differences, you allow for the possibility of escaping local minima, while ensuring that whatever new minimum you find can't be that much worse that where you started. If you make the acceptance of large energy differences more likely, you can escape local minima more often, but you increase the likelihood that you'll just wind up in a region with an <em>even higher</em> local minimum.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-26T16:16:12.427Z
Answers.LastActivityDate: 2020-08-26T16:16:12.427Z
Answers.UserDisplayName: Nuclear Hoagie
Answers.Body: <p>Nice question!</p>
  <p>My guess is that, if the probability of acceptance increases the bigger the difference between the current and new solutions is, then there's the risk that you need to search a lot again to find a good solution, i.e. you may oscillate between different subspaces or you could actually often end up in subspaces where there are only bad solutions. Your reasoning probably makes sense at the beginning of the search when the initial solutions may not be good enough or maybe if you have parallel searches (and you want to explore the search space at different search subspaces), but once you have a good solution, you don't want to completely discard it and replace it with a quite worse solution.</p>
  <p>If you perform some experiments with your idea, I would like to see the results.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-26T15:50:03.213Z
Answers.LastActivityDate: 2020-08-26T16:03:48.660Z
Answers.LastEditDate: 2020-08-26T16:03:48.660Z
Answers.UserDisplayName: nbro
Body: <p>In Simulated Annealing, a worse solution is accepted with this probability:</p>
  <p><span class="math-container">$$p=e^{-\frac{E(y)-E(x)}{kT}}.$$</span></p>
  <p>If that understanding is correct: Why is this probability function used? This means that, the bigger the energy difference, the smaller the probability of accepting the new solution. I would say the bigger the difference the more we want to escape a local minimum. I plotted that function in Matlab in two dimensions:</p>
  <p><a href="https://i.sstatic.net/x1P0p.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/x1P0p.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-23T18:53:16.543Z
Comments.Text: [If any of the answers below answers your question, you should accept one of them](https://ai.stackexchange.com/help/someone-answers). In my view, both answer your question.
Comments.UserDisplayName: nbro
Tags: <optimization><simulated-annealing><meta-heuristics>
Title: Why does Simulated Annealing not take worse solution if the energy difference becomes higher?
UserDisplayName: MScott
ThreadText: # Why does Simulated Annealing not take worse solution if the energy difference becomes higher?
  
  (Asked by: username_0 on 2020-08-26)
  
  In Simulated Annealing, a worse solution is accepted with this probability:
  
  $$p=e^{-\frac{E(y)-E(x)}{kT}}.$$
  
  If that understanding is correct: Why is this probability function used? This means that, the bigger the energy difference, the smaller the probability of accepting the new solution. I would say the bigger the difference the more we want to escape a local minimum. I plotted that function in Matlab in two dimensions:
  
  [![enter image description here](https://i.sstatic.net/x1P0p.png)](https://i.sstatic.net/x1P0p.png)
  
  **Tags:** optimization, simulated-annealing, meta-heuristics
  
  > **Comments:**
  > **username_1**: [If any of the answers below answers your question, you should accept one of them](https://ai.stackexchange.com/help/someone-answers). In my view, both answer your question.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-08-26. Score: 1
  
  Note that you can't really predict whether your escape from a local minimum will work or not - you might just wind up in another, worse local minimum. The probability function you describe increases the likelihood of this happening. By upweighting the likelihood of allowing small energy differences, you allow for the possibility of escaping local minima, while ensuring that whatever new minimum you find can't be that much worse that where you started. If you make the acceptance of large energy differences more likely, you can escape local minima more often, but you increase the likelihood that you'll just wind up in a region with an *even higher* local minimum.
  
  
  ---
  
  ### Answer by username_1 on 2020-08-26. Score: 0
  
  Nice question!
  
  My guess is that, if the probability of acceptance increases the bigger the difference between the current and new solutions is, then there's the risk that you need to search a lot again to find a good solution, i.e. you may oscillate between different subspaces or you could actually often end up in subspaces where there are only bad solutions. Your reasoning probably makes sense at the beginning of the search when the initial solutions may not be good enough or maybe if you have parallel searches (and you want to explore the search space at different search subspaces), but once you have a good solution, you don't want to completely discard it and replace it with a quite worse solution.
  
  If you perform some experiments with your idea, I would like to see the results.
  
  
  ---
  
  