Answers.Body: <p>Perturbed here means adding a small random value to the weight. That random value comes from a uniform distribution or from a gaussian (or any distribution really). 
  Imagine just nudging the weight by a little. </p>
  
  <p>It’s done to overcome the problem of local minima where models can get stuck with a good set of weights but not the best set of weights. By perturbing the weights a little, the model has a chance of finding a better set of weights. Searching for methods that deal with local minima like stochastic gradient descent will give you a better intuition. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-06T22:40:37.427Z
Answers.LastActivityDate: 2018-03-07T16:19:27.323Z
Answers.LastEditDate: 2018-03-07T16:19:27.323Z
Answers.UserDisplayName: Jaden Travnik
Body: <p>I am reading through the NEAT paper <a href="http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf" rel="nofollow noreferrer">here</a>. On page 14 of the PDF, there is this quote about mutation:</p>
  
  <blockquote>
    <p>There was an 80% chance of a genome having its connection weights mutated, in which case <strong>each weight had a 90% chance of being uniformly perturbed</strong> and a 10% chance of being assigned a new random value.</p>
  </blockquote>
  
  <p>What exactly does it mean to perturb weights? What is uniform vs. nonuniform perturbation?</p>
  
  <p>Is there an established method to do this? I am imagining the process as multiplying each connection weight by a random number, but I'm unfamiliar with the term.</p>
  
Tags: <neural-networks><evolutionary-algorithms><neat>
Title: How are connection weights "perturbed"?
UserDisplayName: natedogg
ThreadText: # How are connection weights "perturbed"?
  
  (Asked by: username_0 on 2018-03-06)
  
  I am reading through the NEAT paper [here](http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf). On page 14 of the PDF, there is this quote about mutation:
  
  > There was an 80% chance of a genome having its connection weights mutated, in which case **each weight had a 90% chance of being uniformly perturbed** and a 10% chance of being assigned a new random value.
  
  What exactly does it mean to perturb weights? What is uniform vs. nonuniform perturbation?
  
  Is there an established method to do this? I am imagining the process as multiplying each connection weight by a random number, but I'm unfamiliar with the term.
  
  **Tags:** neural-networks, evolutionary-algorithms, neat
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-03-06. Score: 4 | Accepted answer
  
  Perturbed here means adding a small random value to the weight. That random value comes from a uniform distribution or from a gaussian (or any distribution really).
  Imagine just nudging the weight by a little.
  
  It’s done to overcome the problem of local minima where models can get stuck with a good set of weights but not the best set of weights. By perturbing the weights a little, the model has a chance of finding a better set of weights. Searching for methods that deal with local minima like stochastic gradient descent will give you a better intuition.
  
  
  ---
  
  