Answers.Body: <p>I also found the WRN-<span class="math-container">$n$</span>-<span class="math-container">$k$</span> notation confusing, but I think I can explain it:</p>
  <ul>
  <li><p><span class="math-container">$n$</span> is the <em>total</em> number of convolutions in the model.  So to understand the architecture associated with each <span class="math-container">$n$</span>, we have to carefully account for every convolution:</p>
  <ul>
  <li>Each model starts with a 3x3 convolution (<code>conv1</code> in Table 1)</li>
  <li>There are 3 groups of residual blocks, each the same size.</li>
  <li>Each residual block has 2 convolutions, assuming we're talking about the <span class="math-container">$B(3,3)$</span> blocks that were found to work the best.</li>
  <li>Each group has skip connection, which adds another 1x1 convolution.</li>
  </ul>
  <p>Putting this all together, we can calculate <span class="math-container">$n$</span> as a function of the number of residual blocks in each group, which I'll denote <span class="math-container">$b$</span>:</p>
  <p><span class="math-container">$$
    n = 1 + 3 (2b + 1)
    $$</span>
  <span class="math-container">$$
    b = \frac{\frac{n - 1}{3} - 1}{2}
    $$</span></p>
  <p>Here are the actual number of residual blocks per group for each of the architectures discussed in the paper:</p>
  <ul>
  <li>WRN-40-<span class="math-container">$k$</span>: 6</li>
  <li>WRN-28-<span class="math-container">$k$</span>: 4</li>
  <li>WRN-22-<span class="math-container">$k$</span>: 3</li>
  <li>WRN-16-<span class="math-container">$k$</span>: 2</li>
  </ul>
  </li>
  <li><p><span class="math-container">$k$</span> is the factor by which the number of input channels to each residual block is multiplied.  I found this pretty self-explanatory, so I don't really know what else to say about it, but let me know in the comments if there's anything unclear.</p>
  </li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-17T19:43:13.737Z
Answers.Text: The way the WideResnet authors calculate the depth is actually inconsistent with how Kaiming He et al. do it in [this](https://arxiv.org/pdf/1512.03385) or [this paper](http://arxiv.org/abs/1603.05027), but your answer is correct. :-)
Answers.UserDisplayName: Hermi
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-08T15:49:52.933Z
Answers.LastActivityDate: 2024-01-08T15:49:52.933Z
Answers.UserDisplayName: Kale Kundert
Body: <p>I am a little confused about the WRN-d-k notation from <a href="https://arxiv.org/abs/1605.07146" rel="nofollow noreferrer">Wide Residual Networks</a>. To quote the paper,</p>
  <blockquote>
  <p>In the rest of the paper we use the following notation: WRN-n-k denotes
  a residual network that has a total number of convolutional layers n and a widening factor k (for example, network with 40 layers and k = 2 times wider than original would be denoted as WRN-40-2).</p>
  </blockquote>
  <p>I need to implement WRN-28-10 as a baseline since it's one of the best-performing models in <a href="https://robustbench.github.io/" rel="nofollow noreferrer">RobustBench</a>. According to the definition, I need to take an &quot;original&quot; ResNet-28 and widen its layers by a factor of 10. However, there are only 5 variants of ResNet defined in the <a href="https://arxiv.org/abs/1512.03385" rel="nofollow noreferrer">ResNet paper</a>: ResNet-[18,34,50,101,152].</p>
  <p>How exactly are ResNet-28 and WRN-28-10 defined?</p>
  
Tags: <neural-networks><deep-learning><terminology><papers><residual-networks>
Title: Exact definition of WRN-d-k (Wide ResNet)
UserDisplayName: nalzok
ThreadText: # Exact definition of WRN-d-k (Wide ResNet)
  
  (Asked by: username_0 on 2022-08-16)
  
  I am a little confused about the WRN-d-k notation from [Wide Residual Networks](https://arxiv.org/abs/1605.07146). To quote the paper,
  
  > In the rest of the paper we use the following notation: WRN-n-k denotes
  > a residual network that has a total number of convolutional layers n and a widening factor k (for example, network with 40 layers and k = 2 times wider than original would be denoted as WRN-40-2).
  
  I need to implement WRN-28-10 as a baseline since it's one of the best-performing models in [RobustBench](https://robustbench.github.io/). According to the definition, I need to take an "original" ResNet-28 and widen its layers by a factor of 10. However, there are only 5 variants of ResNet defined in the [ResNet paper](https://arxiv.org/abs/1512.03385): ResNet-[18,34,50,101,152].
  
  How exactly are ResNet-28 and WRN-28-10 defined?
  
  **Tags:** neural-networks, deep-learning, terminology, papers, residual-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-01-08. Score: 2
  
  I also found the WRN-$n$-$k$ notation confusing, but I think I can explain it:
  
  * $n$ is the *total* number of convolutions in the model. So to understand the architecture associated with each $n$, we have to carefully account for every convolution:
  
    + Each model starts with a 3x3 convolution (`conv1` in Table 1)
    + There are 3 groups of residual blocks, each the same size.
    + Each residual block has 2 convolutions, assuming we're talking about the $B(3,3)$ blocks that were found to work the best.
    + Each group has skip connection, which adds another 1x1 convolution.
  
    Putting this all together, we can calculate $n$ as a function of the number of residual blocks in each group, which I'll denote $b$:
  
    $$
    n = 1 + 3 (2b + 1)
    $$
    $$
    b = \frac{\frac{n - 1}{3} - 1}{2}
    $$
  
    Here are the actual number of residual blocks per group for each of the architectures discussed in the paper:
  
    + WRN-40-$k$: 6
    + WRN-28-$k$: 4
    + WRN-22-$k$: 3
    + WRN-16-$k$: 2
  * $k$ is the factor by which the number of input channels to each residual block is multiplied. I found this pretty self-explanatory, so I don't really know what else to say about it, but let me know in the comments if there's anything unclear.
  
  > **Comments:**
  > **username_2**: The way the WideResnet authors calculate the depth is actually inconsistent with how Kaiming He et al. do it in [this](https://arxiv.org/pdf/1512.03385) or [this paper](http://arxiv.org/abs/1603.05027), but your answer is correct. :-)  
  
  ---
  
  