Answers.Body: <p>I've most often heard of the metric, parameters.  I.e. The number of tunable parameters in a model.  For a fully connected layer this is:<br />
  parameters = ((c*p)+1)*c<br />
  Where c is the number of neurons in the previous layer and p is the number of neurons in the current.  Do this for each layer and add.  Convolutional layers, GRUs, LRUs, etc will have their own formula.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-25T14:05:24.910Z
Answers.LastActivityDate: 2023-08-25T14:05:24.910Z
Answers.UserDisplayName: foreverska
Answers.Body: <p>There are a few ways to define &quot;size&quot; of a neural network. The parameter count was previously mentioned, but depending on the circumstances other values may be more important.</p>
  <p>FLOPs may be a good metric (the number of floating point operations during an inference or during training). This can inform your hardware choices as this determines time to train or infer.</p>
  <p>File size (in MB or GB) may also be valuable. Can you load a particular model into your VRAM? How much disk space do I need?</p>
  <p>Size is fairly vague so there are different metrics that can describe it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-25T19:01:49.443Z
Answers.LastActivityDate: 2023-08-25T19:01:49.443Z
Answers.UserDisplayName: Dom DJ
Body: <p>For any type of ANN, is there a common formula which can be used to calculate size of a neural network?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-26T03:17:04.453Z
Comments.Text: What data do you have to calculate it? If you only have the inputs and outputs, it will be extremely hard to tell if it is a complex network or a very well-trained network.
Comments.UserDisplayName: Jake
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-26T10:24:05.667Z
Comments.Text: This is to decide the size of a given NN. Since there are ways to compress NN, is is important to know the size a NN to decide the amount of compression
Comments.UserDisplayName: Indika Alahapperuma
Tags: <neural-networks><math>
Title: Size of a neural network
UserDisplayName: Indika Alahapperuma
ThreadText: # Size of a neural network
  
  (Asked by: username_0 on 2023-08-25)
  
  For any type of ANN, is there a common formula which can be used to calculate size of a neural network?
  
  **Tags:** neural-networks, math
  
  > **Comments:**
  > **username_1**: What data do you have to calculate it? If you only have the inputs and outputs, it will be extremely hard to tell if it is a complex network or a very well-trained network.  
  > **username_0**: This is to decide the size of a given NN. Since there are ways to compress NN, is is important to know the size a NN to decide the amount of compression  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2023-08-25. Score: 1
  
  I've most often heard of the metric, parameters. I.e. The number of tunable parameters in a model. For a fully connected layer this is:  
  parameters = ((c\*p)+1)\*c  
  Where c is the number of neurons in the previous layer and p is the number of neurons in the current. Do this for each layer and add. Convolutional layers, GRUs, LRUs, etc will have their own formula.
  
  
  ---
  
  ### Answer by username_3 on 2023-08-25. Score: 0
  
  There are a few ways to define "size" of a neural network. The parameter count was previously mentioned, but depending on the circumstances other values may be more important.
  
  FLOPs may be a good metric (the number of floating point operations during an inference or during training). This can inform your hardware choices as this determines time to train or infer.
  
  File size (in MB or GB) may also be valuable. Can you load a particular model into your VRAM? How much disk space do I need?
  
  Size is fairly vague so there are different metrics that can describe it.
  
  
  ---
  
  