<p>The width of a neural network <em>layer</em> is an agreed upon term.</p>
<p>According to Lou et al., in the paper <a href="https://proceedings.neurips.cc/paper/2017/file/32cbf687880eb1674a07bf717761dd3a-Paper.pdf" rel="nofollow noreferrer">The Expressive Power of Neural Networks: A View from the Width</a> (page 4), <strong>the width of a neural network is the width of the widest layer of the neural network</strong>.</p>
<blockquote>
<p>The architecture of neural networks often specified by the width and the depth of the networks. The depth <span class="math-container">$h$</span> of a network is defined as its number of layers (including output layer but excluding input
layer); while <strong>the width <span class="math-container">$d_m$</span> of a network is defined to be the maximal number of nodes in a layer</strong><br />
<sub>(emphasis added)</sub></p>
</blockquote>
<p>So, I would caution you to be careful with how you use the phrase &quot;the width of a neural network&quot;, due to interpretability and scale, and the fact that neural networks often contain layers with varying numbers of neurons, depending on the layer.</p>
<p>From this <a href="https://en.wikipedia.org/wiki/Large_width_limits_of_neural_networks" rel="nofollow noreferrer">Wikipedia page</a> on &quot;Large width limits of neural networks&quot;:</p>
<blockquote>
<p>The number of neurons in a layer is called the layer width.</p>
</blockquote>
<p>From a nice <a href="https://machinelearningmastery.com/how-to-configure-the-number-of-layers-and-nodes-in-a-neural-network/" rel="nofollow noreferrer">machine learning resource page</a></p>
<blockquote>
<p>Finally, there are terms used to describe the shape and capability of a neural network; for example:</p>
<ul>
<li>Size: The number of nodes in the model.</li>
<li>Width: The number of nodes in a specific layer.</li>
</ul>
</blockquote>
<blockquote>
<p>A node [is] also called a neuron or Perceptron</p>
</blockquote>

CC BY-SA 4.0
2022-05-24T15:35:56.993Z
I edited your answer to, hopefully, improve its clarity. I would remove the part "From this wikipedia page ..." until the end of your answer because the OP already knows that we can talk about "the width of a layer". That's a pretty much standard definition, so there's no need to say that again. He's also aware of what "depth" means.
nbro
CC BY-SA 4.0
2021-09-23T14:54:15.037Z
2022-05-24T16:27:54.147Z
2022-05-24T16:27:54.147Z
jros
<p>The depth of a neural network is equal to <em>the total number of layers in the neural network (excluding the input layer by convention)</em>. A neural network with &quot;many layers&quot; is called a <em>deep</em> neural network.</p>
<p>On the other hand, the width is the name of a property of a layer in a neural network: it is equal to the number of neurons in that particular layer. So, it may be apt to use the phrase &quot;the width of a layer in a neural network&quot;.</p>
<p>But, is it valid to use the phrase &quot;width of a neural network&quot;?</p>
<p>I got this doubt because the phrase &quot;wide neural network&quot; is widely used. The phrase gives the impression that the width is a property of a neural network. So, I am wondering whether the width of a neural network has a definition. For example, say, the width of a neural network is the number of neurons in the widest layer of that neural network.</p>

CC BY-SA 4.0
2022-05-22T21:36:32.937Z
I changed a little bit your post. I think your question was more what I put in the title, i.e. you were wondering if there is a widely accepted definition of the "width of a neural network". Feel free to change your post again.
nbro
<neural-networks><deep-learning><terminology>
Is there a widely accepted definition of the width of a neural network?
hanugm
# Is there a widely accepted definition of the width of a neural network?

(Asked by: username_0 on 2021-09-22)

The depth of a neural network is equal to *the total number of layers in the neural network (excluding the input layer by convention)*. A neural network with "many layers" is called a *deep* neural network.

On the other hand, the width is the name of a property of a layer in a neural network: it is equal to the number of neurons in that particular layer. So, it may be apt to use the phrase "the width of a layer in a neural network".

But, is it valid to use the phrase "width of a neural network"?

I got this doubt because the phrase "wide neural network" is widely used. The phrase gives the impression that the width is a property of a neural network. So, I am wondering whether the width of a neural network has a definition. For example, say, the width of a neural network is the number of neurons in the widest layer of that neural network.

**Tags:** neural-networks, deep-learning, terminology

> **Comments:**
> **username_1**: I changed a little bit your post. I think your question was more what I put in the title, i.e. you were wondering if there is a widely accepted definition of the "width of a neural network". Feel free to change your post again.  

---

## 1 answers

### Answer by username_2 on 2021-09-23. Score: 5

The width of a neural network *layer* is an agreed upon term.

According to Lou et al., in the paper [The Expressive Power of Neural Networks: A View from the Width](https://proceedings.neurips.cc/paper/2017/file/32cbf687880eb1674a07bf717761dd3a-Paper.pdf) (page 4), **the width of a neural network is the width of the widest layer of the neural network**.

> The architecture of neural networks often specified by the width and the depth of the networks. The depth $h$ of a network is defined as its number of layers (including output layer but excluding input
> layer); while **the width $d\_m$ of a network is defined to be the maximal number of nodes in a layer**  
> (emphasis added)

So, I would caution you to be careful with how you use the phrase "the width of a neural network", due to interpretability and scale, and the fact that neural networks often contain layers with varying numbers of neurons, depending on the layer.

From this [Wikipedia page](https://en.wikipedia.org/wiki/Large_width_limits_of_neural_networks) on "Large width limits of neural networks":

> The number of neurons in a layer is called the layer width.

From a nice [machine learning resource page](https://machinelearningmastery.com/how-to-configure-the-number-of-layers-and-nodes-in-a-neural-network/)

> Finally, there are terms used to describe the shape and capability of a neural network; for example:
>
> * Size: The number of nodes in the model.
> * Width: The number of nodes in a specific layer.

> A node [is] also called a neuron or Perceptron

> **Comments:**
> **username_1**: I edited your answer to, hopefully, improve its clarity. I would remove the part "From this wikipedia page ..." until the end of your answer because the OP already knows that we can talk about "the width of a layer". That's a pretty much standard definition, so there's no need to say that again. He's also aware of what "depth" means.  

---

