<p>I agree with @PhilipRaeisghasem, in most architectures, weights are initialized in a random manner. However, some research papers suggest applying a random normal distribution initialization to the weights in the case of Convolutional Neural Networks (for computer vision).</p>

CC BY-SA 4.0
2019-03-17T23:18:08.240Z
Absolutely. I didn't think getting into specific initialization distributions would be super beneficial for someone asking such a basic question. But you're right.
Philip Raeisghasem
CC BY-SA 4.0
2019-03-18T21:07:53.407Z
I absolutely overthought that question. Thanks for replying.
iustin
CC BY-SA 4.0
2019-03-17T21:44:44.843Z
2019-03-17T21:44:44.843Z
iustin
<p>Typically, weights are randomly initialized. Then, as the model is <a href="https://en.wikipedia.org/wiki/Artificial_neural_network#Optimization" rel="noreferrer">optimized</a> for its given task, those weights are steadily made "better" as determined by the network's <a href="https://en.wikipedia.org/wiki/Loss_function" rel="noreferrer">loss function</a>. This is also referred to as "training" the neural network. </p>

<p>By far the most popular way of updating weights in a neural net is the <a href="https://en.wikipedia.org/wiki/Backpropagation" rel="noreferrer">backpropagation</a> algorithm, most simply with <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent" rel="noreferrer">stochastic gradient descent</a> (SGD). Essentially, the algorithm determines how much each individual weight contributed to the network's loss. It then updates that weight in the direction that would reduce the loss.</p>

<p>I recommend going through Michael Nielsen's <a href="http://neuralnetworksanddeeplearning.com/" rel="noreferrer">online book</a> to learn the basics.</p>

CC BY-SA 4.0
2019-03-17T19:48:10.363Z
2019-03-17T19:55:17.307Z
2019-03-17T19:55:17.307Z
Philip Raeisghasem
<p>I am trying to understand how weights are actually gotten. What is generating them in a neural network? What is the algorithm that gives them certain values?</p>

<neural-networks><machine-learning><neurons>
What determines the values of weights in a neural network?
Aleksandar Kostovic
# What determines the values of weights in a neural network?

(Asked by: username_0 on 2019-03-17)

I am trying to understand how weights are actually gotten. What is generating them in a neural network? What is the algorithm that gives them certain values?

**Tags:** neural-networks, machine-learning, neurons


---

## 2 answers

### Answer by username_2 on 2019-03-17. Score: 5 | Accepted answer

Typically, weights are randomly initialized. Then, as the model is [optimized](https://en.wikipedia.org/wiki/Artificial_neural_network#Optimization) for its given task, those weights are steadily made "better" as determined by the network's [loss function](https://en.wikipedia.org/wiki/Loss_function). This is also referred to as "training" the neural network.

By far the most popular way of updating weights in a neural net is the [backpropagation](https://en.wikipedia.org/wiki/Backpropagation) algorithm, most simply with [stochastic gradient descent](https://en.wikipedia.org/wiki/Stochastic_gradient_descent) (SGD). Essentially, the algorithm determines how much each individual weight contributed to the network's loss. It then updates that weight in the direction that would reduce the loss.

I recommend going through Michael Nielsen's [online book](http://neuralnetworksanddeeplearning.com/) to learn the basics.


---

### Answer by username_1 on 2019-03-17. Score: 0

I agree with @PhilipRaeisghasem, in most architectures, weights are initialized in a random manner. However, some research papers suggest applying a random normal distribution initialization to the weights in the case of Convolutional Neural Networks (for computer vision).

> **Comments:**
> **username_2**: Absolutely. I didn't think getting into specific initialization distributions would be super beneficial for someone asking such a basic question. But you're right.  
> **username_1**: I absolutely overthought that question. Thanks for replying.  

---

