Answers.Body: <p>I agree with @PhilipRaeisghasem, in most architectures, weights are initialized in a random manner. However, some research papers suggest applying a random normal distribution initialization to the weights in the case of Convolutional Neural Networks (for computer vision).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-17T23:18:08.240Z
Answers.Text: Absolutely. I didn't think getting into specific initialization distributions would be super beneficial for someone asking such a basic question. But you're right.
Answers.UserDisplayName: Philip Raeisghasem
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-18T21:07:53.407Z
Answers.Text: I absolutely overthought that question. Thanks for replying.
Answers.UserDisplayName: iustin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-17T21:44:44.843Z
Answers.LastActivityDate: 2019-03-17T21:44:44.843Z
Answers.UserDisplayName: iustin
Answers.Body: <p>Typically, weights are randomly initialized. Then, as the model is <a href="https://en.wikipedia.org/wiki/Artificial_neural_network#Optimization" rel="noreferrer">optimized</a> for its given task, those weights are steadily made "better" as determined by the network's <a href="https://en.wikipedia.org/wiki/Loss_function" rel="noreferrer">loss function</a>. This is also referred to as "training" the neural network. </p>
  
  <p>By far the most popular way of updating weights in a neural net is the <a href="https://en.wikipedia.org/wiki/Backpropagation" rel="noreferrer">backpropagation</a> algorithm, most simply with <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent" rel="noreferrer">stochastic gradient descent</a> (SGD). Essentially, the algorithm determines how much each individual weight contributed to the network's loss. It then updates that weight in the direction that would reduce the loss.</p>
  
  <p>I recommend going through Michael Nielsen's <a href="http://neuralnetworksanddeeplearning.com/" rel="noreferrer">online book</a> to learn the basics.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-17T19:48:10.363Z
Answers.LastActivityDate: 2019-03-17T19:55:17.307Z
Answers.LastEditDate: 2019-03-17T19:55:17.307Z
Answers.UserDisplayName: Philip Raeisghasem
Body: <p>I am trying to understand how weights are actually gotten. What is generating them in a neural network? What is the algorithm that gives them certain values?</p>
  
Tags: <neural-networks><machine-learning><neurons>
Title: What determines the values of weights in a neural network?
UserDisplayName: Aleksandar Kostovic
ThreadText: # What determines the values of weights in a neural network?
  
  (Asked by: username_0 on 2019-03-17)
  
  I am trying to understand how weights are actually gotten. What is generating them in a neural network? What is the algorithm that gives them certain values?
  
  **Tags:** neural-networks, machine-learning, neurons
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-03-17. Score: 5 | Accepted answer
  
  Typically, weights are randomly initialized. Then, as the model is [optimized](https://en.wikipedia.org/wiki/Artificial_neural_network#Optimization) for its given task, those weights are steadily made "better" as determined by the network's [loss function](https://en.wikipedia.org/wiki/Loss_function). This is also referred to as "training" the neural network.
  
  By far the most popular way of updating weights in a neural net is the [backpropagation](https://en.wikipedia.org/wiki/Backpropagation) algorithm, most simply with [stochastic gradient descent](https://en.wikipedia.org/wiki/Stochastic_gradient_descent) (SGD). Essentially, the algorithm determines how much each individual weight contributed to the network's loss. It then updates that weight in the direction that would reduce the loss.
  
  I recommend going through Michael Nielsen's [online book](http://neuralnetworksanddeeplearning.com/) to learn the basics.
  
  
  ---
  
  ### Answer by username_1 on 2019-03-17. Score: 0
  
  I agree with @username_2, in most architectures, weights are initialized in a random manner. However, some research papers suggest applying a random normal distribution initialization to the weights in the case of Convolutional Neural Networks (for computer vision).
  
  > **Comments:**
  > **username_2**: Absolutely. I didn't think getting into specific initialization distributions would be super beneficial for someone asking such a basic question. But you're right.  
  > **username_1**: I absolutely overthought that question. Thanks for replying.  
  
  ---
  
  