Answers.Body: <p>The width of a neural network <em>layer</em> is an agreed upon term.</p>
  <p>According to Lou et al., in the paper <a href="https://proceedings.neurips.cc/paper/2017/file/32cbf687880eb1674a07bf717761dd3a-Paper.pdf" rel="nofollow noreferrer">The Expressive Power of Neural Networks: A View from the Width</a> (page 4), <strong>the width of a neural network is the width of the widest layer of the neural network</strong>.</p>
  <blockquote>
  <p>The architecture of neural networks often specified by the width and the depth of the networks. The depth <span class="math-container">$h$</span> of a network is defined as its number of layers (including output layer but excluding input
  layer); while <strong>the width <span class="math-container">$d_m$</span> of a network is defined to be the maximal number of nodes in a layer</strong><br />
  <sub>(emphasis added)</sub></p>
  </blockquote>
  <p>So, I would caution you to be careful with how you use the phrase &quot;the width of a neural network&quot;, due to interpretability and scale, and the fact that neural networks often contain layers with varying numbers of neurons, depending on the layer.</p>
  <p>From this <a href="https://en.wikipedia.org/wiki/Large_width_limits_of_neural_networks" rel="nofollow noreferrer">Wikipedia page</a> on &quot;Large width limits of neural networks&quot;:</p>
  <blockquote>
  <p>The number of neurons in a layer is called the layer width.</p>
  </blockquote>
  <p>From a nice <a href="https://machinelearningmastery.com/how-to-configure-the-number-of-layers-and-nodes-in-a-neural-network/" rel="nofollow noreferrer">machine learning resource page</a></p>
  <blockquote>
  <p>Finally, there are terms used to describe the shape and capability of a neural network; for example:</p>
  <ul>
  <li>Size: The number of nodes in the model.</li>
  <li>Width: The number of nodes in a specific layer.</li>
  </ul>
  </blockquote>
  <blockquote>
  <p>A node [is] also called a neuron or Perceptron</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-24T15:35:56.993Z
Answers.Text: I edited your answer to, hopefully, improve its clarity. I would remove the part "From this wikipedia page ..." until the end of your answer because the OP already knows that we can talk about "the width of a layer". That's a pretty much standard definition, so there's no need to say that again. He's also aware of what "depth" means.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-23T14:54:15.037Z
Answers.LastActivityDate: 2022-05-24T16:27:54.147Z
Answers.LastEditDate: 2022-05-24T16:27:54.147Z
Answers.UserDisplayName: jros
Body: <p>The depth of a neural network is equal to <em>the total number of layers in the neural network (excluding the input layer by convention)</em>. A neural network with &quot;many layers&quot; is called a <em>deep</em> neural network.</p>
  <p>On the other hand, the width is the name of a property of a layer in a neural network: it is equal to the number of neurons in that particular layer. So, it may be apt to use the phrase &quot;the width of a layer in a neural network&quot;.</p>
  <p>But, is it valid to use the phrase &quot;width of a neural network&quot;?</p>
  <p>I got this doubt because the phrase &quot;wide neural network&quot; is widely used. The phrase gives the impression that the width is a property of a neural network. So, I am wondering whether the width of a neural network has a definition. For example, say, the width of a neural network is the number of neurons in the widest layer of that neural network.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-22T21:36:32.937Z
Comments.Text: I changed a little bit your post. I think your question was more what I put in the title, i.e. you were wondering if there is a widely accepted definition of the "width of a neural network". Feel free to change your post again.
Comments.UserDisplayName: nbro
Tags: <neural-networks><deep-learning><terminology>
Title: Is there a widely accepted definition of the width of a neural network?
UserDisplayName: hanugm
ThreadText: # Is there a widely accepted definition of the width of a neural network?
  
  (Asked by: username_0 on 2021-09-22)
  
  The depth of a neural network is equal to *the total number of layers in the neural network (excluding the input layer by convention)*. A neural network with "many layers" is called a *deep* neural network.
  
  On the other hand, the width is the name of a property of a layer in a neural network: it is equal to the number of neurons in that particular layer. So, it may be apt to use the phrase "the width of a layer in a neural network".
  
  But, is it valid to use the phrase "width of a neural network"?
  
  I got this doubt because the phrase "wide neural network" is widely used. The phrase gives the impression that the width is a property of a neural network. So, I am wondering whether the width of a neural network has a definition. For example, say, the width of a neural network is the number of neurons in the widest layer of that neural network.
  
  **Tags:** neural-networks, deep-learning, terminology
  > **Comments:**
  > **username_1**: I changed a little bit your post. I think your question was more what I put in the title, i.e. you were wondering if there is a widely accepted definition of the "width of a neural network". Feel free to change your post again.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-09-23. Score: 5
  
  The width of a neural network *layer* is an agreed upon term.
  
  According to Lou et al., in the paper [The Expressive Power of Neural Networks: A View from the Width](https://proceedings.neurips.cc/paper/2017/file/32cbf687880eb1674a07bf717761dd3a-Paper.pdf) (page 4), **the width of a neural network is the width of the widest layer of the neural network**.
  
  > The architecture of neural networks often specified by the width and the depth of the networks. The depth $h$ of a network is defined as its number of layers (including output layer but excluding input
  > layer); while **the width $d\_m$ of a network is defined to be the maximal number of nodes in a layer**  
  > (emphasis added)
  
  So, I would caution you to be careful with how you use the phrase "the width of a neural network", due to interpretability and scale, and the fact that neural networks often contain layers with varying numbers of neurons, depending on the layer.
  
  From this [Wikipedia page](https://en.wikipedia.org/wiki/Large_width_limits_of_neural_networks) on "Large width limits of neural networks":
  
  > The number of neurons in a layer is called the layer width.
  
  From a nice [machine learning resource page](https://machinelearningmastery.com/how-to-configure-the-number-of-layers-and-nodes-in-a-neural-network/)
  
  > Finally, there are terms used to describe the shape and capability of a neural network; for example:
  >
  > * Size: The number of nodes in the model.
  > * Width: The number of nodes in a specific layer.
  
  > A node [is] also called a neuron or Perceptron
  
  > **Comments:**
  > **username_1**: I edited your answer to, hopefully, improve its clarity. I would remove the part "From this wikipedia page ..." until the end of your answer because the OP already knows that we can talk about "the width of a layer". That's a pretty much standard definition, so there's no need to say that again. He's also aware of what "depth" means.  
  
  ---
  
  