Answers.Body: <p>Check <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense" rel="nofollow noreferrer">the documentation for <code>Dense</code></a> layer:</p>
  <blockquote>
  <p><strong>Note:</strong> If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot). For example, if input has dimensions (batch_size, d0, d1), then we create a kernel with shape (d1, units), and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1) (there are batch_size * d0 such sub-tensors). The output in this case will have shape (batch_size, d0, units).</p>
  </blockquote>
  <p>That is what happening in your first case - for input dimensions <code>(4,1)</code> you've got <code>d0=4</code> and <code>d1=1</code>. So it creates a kernel of shape <code>(1,32)</code> that gets applied along the axis of dimension 4. That's why your output shape is <code>(4,32)</code> and you've got 32 weights + 32 biases = 64 parameters.</p>
  <p>In second case you've got a &quot;standard&quot; 32 * 4 fully-connected weight matrix + 32 biases = 160.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-12T12:28:25.220Z
Answers.LastActivityDate: 2021-04-12T12:28:25.220Z
Answers.UserDisplayName: Kostya
Body: <p>I am a bit confused about how the number of parameters are calculated in Dense model for the Kera/Tensorflow.</p>
  <p>For example, in the figure below I thought that both the statements were the same, but I found a different number of parameters for both.  In particular, I am talking about <code>model.add(Dense(...))</code> command.</p>
  <p><a href="https://i.sstatic.net/w8aip.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/w8aip.png" alt="The models in consideration" /></a></p>
  
Tags: <neural-networks><tensorflow><keras>
Title: Number of parameters in Keras/Tensorflow Dense layers
UserDisplayName: SJa
ThreadText: # Number of parameters in Keras/Tensorflow Dense layers
  
  (Asked by: username_0 on 2021-04-12)
  
  I am a bit confused about how the number of parameters are calculated in Dense model for the Kera/Tensorflow.
  
  For example, in the figure below I thought that both the statements were the same, but I found a different number of parameters for both. In particular, I am talking about `model.add(Dense(...))` command.
  
  [![The models in consideration](https://i.sstatic.net/w8aip.png)](https://i.sstatic.net/w8aip.png)
  
  **Tags:** neural-networks, tensorflow, keras
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-04-12. Score: 2 | Accepted answer
  
  Check [the documentation for `Dense`](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense) layer:
  
  > **Note:** If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot). For example, if input has dimensions (batch\_size, d0, d1), then we create a kernel with shape (d1, units), and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1) (there are batch\_size \* d0 such sub-tensors). The output in this case will have shape (batch\_size, d0, units).
  
  That is what happening in your first case - for input dimensions `(4,1)` you've got `d0=4` and `d1=1`. So it creates a kernel of shape `(1,32)` that gets applied along the axis of dimension 4. That's why your output shape is `(4,32)` and you've got 32 weights + 32 biases = 64 parameters.
  
  In second case you've got a "standard" 32 \* 4 fully-connected weight matrix + 32 biases = 160.
  
  
  ---
  
  