Body: <p>I am training a model to generate images.</p>
  <p>The model contains 5+5 layers:</p>
  <pre><code>Conv2D -&gt; Upsample -&gt; Conv2D -&gt; Upsample -&gt; Conv2D -&gt; Upsample -&gt; Conv2D -&gt; Upsample -&gt; Conv2D -&gt; Upsample
  </code></pre>
  <p>I am modifying it as</p>
  <pre><code>Conv2D -&gt; BatchNorm -&gt; Upsample -&gt; Conv2D -&gt; BatchNorm -&gt; Upsample -&gt; Conv2D -&gt; BatchNorm -&gt; Upsample -&gt; Conv2D -&gt; BatchNorm -&gt; Upsample -&gt; Conv2D -&gt; BatchNorm -&gt; Upsample
  </code></pre>
  <p>I am applying the batch normalization layers just before upsampling as shown above and hence I am not getting the results that are at least comparable to the results by the model without any batch normalization layer.</p>
  <p>Is my placement of the batch normalization layer wrong? If yes, then why?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-29T12:14:29.673Z
Comments.Text: I don't know if it makes any difference, but it may be a good idea to tell us which specific model you're using.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-30T22:30:01.887Z
Comments.Text: @nbro Okay, I will update today.
Comments.UserDisplayName: hanugm
Tags: <deep-learning><convolutional-neural-networks><generative-model><batch-normalization>
Title: Why batch normalization before upsampling is giving worse results?
UserDisplayName: hanugm
ThreadText: # Why batch normalization before upsampling is giving worse results?
  
  (Asked by: username_0 on 2021-11-27)
  
  I am training a model to generate images.
  The model contains 5+5 layers:
  Conv2D -> Upsample -> Conv2D -> Upsample -> Conv2D -> Upsample -> Conv2D -> Upsample -> Conv2D -> Upsample
  
  I am modifying it as
  Conv2D -> BatchNorm -> Upsample -> Conv2D -> BatchNorm -> Upsample -> Conv2D -> BatchNorm -> Upsample -> Conv2D -> BatchNorm -> Upsample -> Conv2D -> BatchNorm -> Upsample
  
  I am applying the batch normalization layers just before upsampling as shown above and hence I am not getting the results that are at least comparable to the results by the model without any batch normalization layer.
  Is my placement of the batch normalization layer wrong? If yes, then why?
  
  **Tags:** deep-learning, convolutional-neural-networks, generative-model, batch-normalization
  > **Comments:**
  > **username_1**: I don't know if it makes any difference, but it may be a good idea to tell us which specific model you're using.  
  > **username_0**: @nbro Okay, I will update today.  
  
  ---
  
  ## 0 answers
  
  