Answers.Body: <p>I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications.</p>
  <p>There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car.</p>
  <p><a href="https://mathworld.wolfram.com/SymmetricSuccessiveOverrelaxationMethod.html" rel="nofollow noreferrer">Symmetric relaxation</a> can be a challenging subject, so I'll include a few links <a href="https://www.researchgate.net/publication/245356183_Calculation_of_Laminar_Separated_Flow_in_Symmetric_Two-Dimensional_Diffusers" rel="nofollow noreferrer">academic research</a></p>
  <p><a href="https://arxiv.org/pdf/1902.05157.pdf" rel="nofollow noreferrer">Academic research</a> arxiv.org is another site I use for research. Hope this helps.</p>
  <p>I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. <a href="https://medium.com/bcggamma/lagrangian-relaxation-can-solve-your-optimization-problem-much-much-faster-daa9edc47cc9" rel="nofollow noreferrer">Medium Lagrangian Relaxation</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-30T19:08:33.353Z
Answers.Text: Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric.
Answers.UserDisplayName: EmmanuelMess
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-28T21:09:50.283Z
Answers.LastActivityDate: 2020-12-28T22:02:16.310Z
Answers.LastEditDate: 2020-12-28T22:02:16.310Z
Answers.UserDisplayName: CoffeeBaconAddict
Body: <p>In <a href="http://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf" rel="nofollow noreferrer">Chapter 8</a>, section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation.</p>
  <p>But the chapter doesn't explain what asymmetric relaxation would be or how it is done.</p>
  <p>So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes?</p>
  
Tags: <neural-networks><terminology><backpropagation>
Title: What is asymmetric relaxation backpropagation?
UserDisplayName: EmmanuelMess
ThreadText: # What is asymmetric relaxation backpropagation?
  
  (Asked by: username_0 on 2020-12-19)
  
  In [Chapter 8](http://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf), section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation.
  
  But the chapter doesn't explain what asymmetric relaxation would be or how it is done.
  
  So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes?
  
  **Tags:** neural-networks, terminology, backpropagation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-12-28. Score: 0
  
  I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications.
  
  There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car.
  
  [Symmetric relaxation](https://mathworld.wolfram.com/SymmetricSuccessiveOverrelaxationMethod.html) can be a challenging subject, so I'll include a few links [academic research](https://www.researchgate.net/publication/245356183_Calculation_of_Laminar_Separated_Flow_in_Symmetric_Two-Dimensional_Diffusers)
  
  [Academic research](https://arxiv.org/pdf/1902.05157.pdf) arxiv.org is another site I use for research. Hope this helps.
  
  I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. [Medium Lagrangian Relaxation](https://medium.com/bcggamma/lagrangian-relaxation-can-solve-your-optimization-problem-much-much-faster-daa9edc47cc9)
  
  > **Comments:**
  > **username_0**: Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric.  
  
  ---
  
  