Answers.Body: <p>The Dense layers outputs 20 values per example. And since you have 10 examples in the batch the output is <code>(10, 20)</code> (one set of 20 values per example in the batch).
  The nn doesn't learn 10 separate sets of weights. Each set of 20 values is computed with the same weight (and bias if you have any). So if say example 2 and 5 had the same input values, they'll always have the same output values.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-27T00:37:52.123Z
Answers.Text: Yes I understand where you are coming from. There will be 20 different values per example because its features (lets call them x1,x2 and x3) are different. Then, there will be 10 examples because of mini batch gradient descent -  the average of the gradients of all 10 examples will be taken into account for the weights update. Is this correct?
Answers.UserDisplayName: jaksnak
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-28T10:09:22.453Z
Answers.Text: No. The gradients of the weights would be the sum (instead of average) of the gradients for each example.
Answers.UserDisplayName: dbalchev
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-26T19:41:28.677Z
Answers.LastActivityDate: 2020-08-26T19:41:28.677Z
Answers.UserDisplayName: dbalchev
Body: <p>I understand that the batch size is the number of examples you pass into the neural network (NN). If the batch size is 10, it means you feed the NN 10 examples at once.</p>
  <p>Assuming I have an NN with a single <a href="https://keras.io/api/layers/core_layers/dense/" rel="nofollow noreferrer"><code>Dense</code></a> layer. This <code>Dense</code> layer of <code>20 units</code> has an input shape <code>(10, 3)</code>. This means that I am feeding the NN 10 examples at once, with every example being represented by 3 values. This <code>Dense</code> layer will have an output shape of <code>(10, 20)</code>.</p>
  <p>I understand that the 20 in the 2nd dimension comes from the number of units in the <code>Dense</code> layer. However, what does the <code>10 (Batch Size)</code> in the first dimension mean? Does this mean that the NN learns 10 separate sets of weights (with each set of weights corresponding to one example, and one set of weights being a matrix of 60 values:3 features x 20 units)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-29T01:00:14.393Z
Comments.Text: Note that, in general, implementation-related questions (including questions about software libraries) are off-topic here. Please, have a look at https://ai.stackexchange.com/help/on-topic for more info.
Comments.UserDisplayName: nbro
Tags: <neural-networks><keras><hidden-layers><dense-layers><batch-learning>
Title: Why does the output shape of a Dense layer contain a batch size?
UserDisplayName: jaksnak
ThreadText: # Why does the output shape of a Dense layer contain a batch size?
  
  (Asked by: username_0 on 2020-08-26)
  
  I understand that the batch size is the number of examples you pass into the neural network (NN). If the batch size is 10, it means you feed the NN 10 examples at once.
  
  Assuming I have an NN with a single [`Dense`](https://keras.io/api/layers/core_layers/dense/) layer. This `Dense` layer of `20 units` has an input shape `(10, 3)`. This means that I am feeding the NN 10 examples at once, with every example being represented by 3 values. This `Dense` layer will have an output shape of `(10, 20)`.
  
  I understand that the 20 in the 2nd dimension comes from the number of units in the `Dense` layer. However, what does the `10 (Batch Size)` in the first dimension mean? Does this mean that the NN learns 10 separate sets of weights (with each set of weights corresponding to one example, and one set of weights being a matrix of 60 values:3 features x 20 units)?
  
  **Tags:** neural-networks, keras, hidden-layers, dense-layers, batch-learning
  
  > **Comments:**
  > **username_1**: Note that, in general, implementation-related questions (including questions about software libraries) are off-topic here. Please, have a look at https://ai.stackexchange.com/help/on-topic for more info.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-08-26. Score: 1 | Accepted answer
  
  The Dense layers outputs 20 values per example. And since you have 10 examples in the batch the output is `(10, 20)` (one set of 20 values per example in the batch).
  The nn doesn't learn 10 separate sets of weights. Each set of 20 values is computed with the same weight (and bias if you have any). So if say example 2 and 5 had the same input values, they'll always have the same output values.
  
  > **Comments:**
  > **username_0**: Yes I understand where you are coming from. There will be 20 different values per example because its features (lets call them x1,x2 and x3) are different. Then, there will be 10 examples because of mini batch gradient descent -  the average of the gradients of all 10 examples will be taken into account for the weights update. Is this correct?  
  > **username_2**: No. The gradients of the weights would be the sum (instead of average) of the gradients for each example.  
  
  ---
  
  