<p>I also found the WRN-<span class="math-container">$n$</span>-<span class="math-container">$k$</span> notation confusing, but I think I can explain it:</p>
<ul>
<li><p><span class="math-container">$n$</span> is the <em>total</em> number of convolutions in the model.  So to understand the architecture associated with each <span class="math-container">$n$</span>, we have to carefully account for every convolution:</p>
<ul>
<li>Each model starts with a 3x3 convolution (<code>conv1</code> in Table 1)</li>
<li>There are 3 groups of residual blocks, each the same size.</li>
<li>Each residual block has 2 convolutions, assuming we're talking about the <span class="math-container">$B(3,3)$</span> blocks that were found to work the best.</li>
<li>Each group has skip connection, which adds another 1x1 convolution.</li>
</ul>
<p>Putting this all together, we can calculate <span class="math-container">$n$</span> as a function of the number of residual blocks in each group, which I'll denote <span class="math-container">$b$</span>:</p>
<p><span class="math-container">$$
  n = 1 + 3 (2b + 1)
  $$</span>
<span class="math-container">$$
  b = \frac{\frac{n - 1}{3} - 1}{2}
  $$</span></p>
<p>Here are the actual number of residual blocks per group for each of the architectures discussed in the paper:</p>
<ul>
<li>WRN-40-<span class="math-container">$k$</span>: 6</li>
<li>WRN-28-<span class="math-container">$k$</span>: 4</li>
<li>WRN-22-<span class="math-container">$k$</span>: 3</li>
<li>WRN-16-<span class="math-container">$k$</span>: 2</li>
</ul>
</li>
<li><p><span class="math-container">$k$</span> is the factor by which the number of input channels to each residual block is multiplied.  I found this pretty self-explanatory, so I don't really know what else to say about it, but let me know in the comments if there's anything unclear.</p>
</li>
</ul>

CC BY-SA 4.0
2024-05-17T19:43:13.737Z
The way the WideResnet authors calculate the depth is actually inconsistent with how Kaiming He et al. do it in [this](https://arxiv.org/pdf/1512.03385) or [this paper](http://arxiv.org/abs/1603.05027), but your answer is correct. :-)
Hermi
CC BY-SA 4.0
2024-01-08T15:49:52.933Z
2024-01-08T15:49:52.933Z
Kale Kundert
<p>I am a little confused about the WRN-d-k notation from <a href="https://arxiv.org/abs/1605.07146" rel="nofollow noreferrer">Wide Residual Networks</a>. To quote the paper,</p>
<blockquote>
<p>In the rest of the paper we use the following notation: WRN-n-k denotes
a residual network that has a total number of convolutional layers n and a widening factor k (for example, network with 40 layers and k = 2 times wider than original would be denoted as WRN-40-2).</p>
</blockquote>
<p>I need to implement WRN-28-10 as a baseline since it's one of the best-performing models in <a href="https://robustbench.github.io/" rel="nofollow noreferrer">RobustBench</a>. According to the definition, I need to take an &quot;original&quot; ResNet-28 and widen its layers by a factor of 10. However, there are only 5 variants of ResNet defined in the <a href="https://arxiv.org/abs/1512.03385" rel="nofollow noreferrer">ResNet paper</a>: ResNet-[18,34,50,101,152].</p>
<p>How exactly are ResNet-28 and WRN-28-10 defined?</p>

<neural-networks><deep-learning><terminology><papers><residual-networks>
Exact definition of WRN-d-k (Wide ResNet)
nalzok
# Exact definition of WRN-d-k (Wide ResNet)

(Asked by: username_0 on 2022-08-16)

I am a little confused about the WRN-d-k notation from [Wide Residual Networks](https://arxiv.org/abs/1605.07146). To quote the paper,

> In the rest of the paper we use the following notation: WRN-n-k denotes
> a residual network that has a total number of convolutional layers n and a widening factor k (for example, network with 40 layers and k = 2 times wider than original would be denoted as WRN-40-2).

I need to implement WRN-28-10 as a baseline since it's one of the best-performing models in [RobustBench](https://robustbench.github.io/). According to the definition, I need to take an "original" ResNet-28 and widen its layers by a factor of 10. However, there are only 5 variants of ResNet defined in the [ResNet paper](https://arxiv.org/abs/1512.03385): ResNet-[18,34,50,101,152].

How exactly are ResNet-28 and WRN-28-10 defined?

**Tags:** neural-networks, deep-learning, terminology, papers, residual-networks


---

## 1 answers

### Answer by username_1 on 2024-01-08. Score: 2

I also found the WRN-$n$-$k$ notation confusing, but I think I can explain it:

* $n$ is the *total* number of convolutions in the model. So to understand the architecture associated with each $n$, we have to carefully account for every convolution:

  + Each model starts with a 3x3 convolution (`conv1` in Table 1)
  + There are 3 groups of residual blocks, each the same size.
  + Each residual block has 2 convolutions, assuming we're talking about the $B(3,3)$ blocks that were found to work the best.
  + Each group has skip connection, which adds another 1x1 convolution.

  Putting this all together, we can calculate $n$ as a function of the number of residual blocks in each group, which I'll denote $b$:

  $$
  n = 1 + 3 (2b + 1)
  $$
  $$
  b = \frac{\frac{n - 1}{3} - 1}{2}
  $$

  Here are the actual number of residual blocks per group for each of the architectures discussed in the paper:

  + WRN-40-$k$: 6
  + WRN-28-$k$: 4
  + WRN-22-$k$: 3
  + WRN-16-$k$: 2
* $k$ is the factor by which the number of input channels to each residual block is multiplied. I found this pretty self-explanatory, so I don't really know what else to say about it, but let me know in the comments if there's anything unclear.

> **Comments:**
> **username_2**: The way the WideResnet authors calculate the depth is actually inconsistent with how Kaiming He et al. do it in [this](https://arxiv.org/pdf/1512.03385) or [this paper](http://arxiv.org/abs/1603.05027), but your answer is correct. :-)  

---

