Answers.Body: <p>Why are you using 3D convolution/pooling if you input a 2D image ?</p>
  <p>The expected input size should be <strong>BxCxDxHxW</strong> for a 3D convolution, not  <strong>BxCxHxW</strong> . In your case, <strong>C=1</strong> because you said you only have one channel. <strong>B</strong> is the batch dimension, which is usually added by the dataloader. And I suppose that you have <strong>H=W=512</strong>. But you have no 3rd coordinate (the depth <strong>D</strong> of the image, which should be a cube if you work in 3D imagery).</p>
  <p>The 3D pooling is trying to do pooling on the channel dimension, which has value 1 since you only have one channel. Hence it returns an error because you cannot downscale more than one pixel...</p>
  <p>You can use the Upsample function that is compatible with 3D images, instead of Resize :
  <a href="https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html" rel="nofollow noreferrer">https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html</a></p>
  <p>If you use 2D images then you need to use 2D convolutions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-08T19:50:23.497Z
Answers.Text: I realized that my dataloader was skipping the depth, so I fixed it and it is running fine now. Though I have trouble with loss function. my dice score is coming out to be negative, but I will post the another question for that.
Answers.UserDisplayName: user1631306
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-08T10:51:31.833Z
Answers.LastActivityDate: 2024-02-08T11:00:22.587Z
Answers.LastEditDate: 2024-02-08T11:00:22.587Z
Answers.UserDisplayName: Lelouch
Body: <p>I wrote simple 3D-Unet arch in pytorch to do segmentation on 3D images.</p>
  <pre><code>class UNet3D(nn.Module):
  def __init__(self, in_channels, out_channels):
      super(UNet3D, self).__init__()
  
      # Encoder
      self.encoder = nn.Sequential(
          nn.Conv3d(in_channels, 64, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.Conv3d(64, 64, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.MaxPool3d(kernel_size=(2,2,2), stride=2)
      )
  
      # Middle
      self.middle = nn.Sequential(
          nn.Conv3d(64, 128, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.Conv3d(128, 128, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.MaxPool3d(kernel_size=(2,2,2), stride=2)
      )
  
      # Decoder
      self.decoder = nn.Sequential(
          nn.ConvTranspose3d(128, 64, kernel_size=(2,2,2), stride=2),
          nn.ReLU(inplace=True),
          nn.ConvTranspose3d(64, 64, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.ConvTranspose3d(64, out_channels, kernel_size=(2,2,2), stride=2)
      )
  
  def forward(self, x):
      # Encoder
      print(x.shape)
      print(type(x))
      x1 = self.encoder(x)
  
      # Middle
      x2 = self.middle(x1)
      print(x2.shape)
  
      # Decoder
      x3 = self.decoder(x2)
  
      return x3
  </code></pre>
  <p>My images are one channel with rectangle size, so I am resizing them to 512</p>
  <pre><code>transform = transforms.Compose([ transforms.ToTensor(), transforms.Resize((512, 512))])
  </code></pre>
  <p>my architecture summary seems good to me</p>
  <pre><code>----------------------------------------------------------------
      Layer (type)               Output Shape         Param #
  ================================================================
          Conv3d-1    [-1, 64, 128, 128, 128]           1,792
            ReLU-2    [-1, 64, 128, 128, 128]               0
          Conv3d-3    [-1, 64, 128, 128, 128]         110,656
            ReLU-4    [-1, 64, 128, 128, 128]               0
       MaxPool3d-5       [-1, 64, 64, 64, 64]               0
          Conv3d-6      [-1, 128, 64, 64, 64]         221,312
            ReLU-7      [-1, 128, 64, 64, 64]               0
          Conv3d-8      [-1, 128, 64, 64, 64]         442,496
            ReLU-9      [-1, 128, 64, 64, 64]               0
      MaxPool3d-10      [-1, 128, 32, 32, 32]               0
    ConvTranspose3d-11       [-1, 64, 64, 64, 64]          65,600
           ReLU-12       [-1, 64, 64, 64, 64]               0
    ConvTranspose3d-13       [-1, 64, 64, 64, 64]         110,656
           ReLU-14       [-1, 64, 64, 64, 64]               0
    ConvTranspose3d-15     [-1, 1, 128, 128, 128]             513
  ================================================================
  </code></pre>
  <p>but I am getting following erro when I try to train the model</p>
  <blockquote>
  <p>Given input size: (64x1x512x512). Calculated output size:
  (64x0x256x256). Output size is too small</p>
  </blockquote>
  <p>Any idea how to fix it and what is going on</p>
  <p>PS. the error comes from maxpooling3d layer from encoder part</p>
  
Tags: <pytorch><u-net><3d-convolution>
Title: 3D Unet gives "output size is too small" error
UserDisplayName: user1631306
ThreadText: # 3D Unet gives "output size is too small" error
  
  (Asked by: username_0 on 2024-02-07)
  
  I wrote simple 3D-Unet arch in pytorch to do segmentation on 3D images.
  class UNet3D(nn.Module):
  def __init__(self, in_channels, out_channels):
      super(UNet3D, self).__init__()
  
      # Encoder
      self.encoder = nn.Sequential(
          nn.Conv3d(in_channels, 64, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.Conv3d(64, 64, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.MaxPool3d(kernel_size=(2,2,2), stride=2)
      )
  
      # Middle
      self.middle = nn.Sequential(
          nn.Conv3d(64, 128, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.Conv3d(128, 128, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.MaxPool3d(kernel_size=(2,2,2), stride=2)
      )
  
      # Decoder
      self.decoder = nn.Sequential(
          nn.ConvTranspose3d(128, 64, kernel_size=(2,2,2), stride=2),
          nn.ReLU(inplace=True),
          nn.ConvTranspose3d(64, 64, kernel_size=(3,3,3), padding=1),
          nn.ReLU(inplace=True),
          nn.ConvTranspose3d(64, out_channels, kernel_size=(2,2,2), stride=2)
      )
  
  def forward(self, x):
      # Encoder
      print(x.shape)
      print(type(x))
      x1 = self.encoder(x)
  
      # Middle
      x2 = self.middle(x1)
      print(x2.shape)
  
      # Decoder
      x3 = self.decoder(x2)
  
      return x3
  
  My images are one channel with rectangle size, so I am resizing them to 512
  transform = transforms.Compose([ transforms.ToTensor(), transforms.Resize((512, 512))])
  
  my architecture summary seems good to me
  ----------------------------------------------------------------
      Layer (type)               Output Shape         Param #
  ================================================================
          Conv3d-1    [-1, 64, 128, 128, 128]           1,792
            ReLU-2    [-1, 64, 128, 128, 128]               0
          Conv3d-3    [-1, 64, 128, 128, 128]         110,656
            ReLU-4    [-1, 64, 128, 128, 128]               0
       MaxPool3d-5       [-1, 64, 64, 64, 64]               0
          Conv3d-6      [-1, 128, 64, 64, 64]         221,312
            ReLU-7      [-1, 128, 64, 64, 64]               0
          Conv3d-8      [-1, 128, 64, 64, 64]         442,496
            ReLU-9      [-1, 128, 64, 64, 64]               0
      MaxPool3d-10      [-1, 128, 32, 32, 32]               0
    ConvTranspose3d-11       [-1, 64, 64, 64, 64]          65,600
           ReLU-12       [-1, 64, 64, 64, 64]               0
    ConvTranspose3d-13       [-1, 64, 64, 64, 64]         110,656
           ReLU-14       [-1, 64, 64, 64, 64]               0
    ConvTranspose3d-15     [-1, 1, 128, 128, 128]             513
  ================================================================
  
  but I am getting following erro when I try to train the model
  
  Given input size: (64x1x512x512). Calculated output size:
  (64x0x256x256). Output size is too small
  
  Any idea how to fix it and what is going on
  PS. the error comes from maxpooling3d layer from encoder part
  
  **Tags:** pytorch, u-net, 3d-convolution
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-02-08. Score: 1 | Accepted answer
  
  Why are you using 3D convolution/pooling if you input a 2D image ?
  
  The expected input size should be **BxCxDxHxW** for a 3D convolution, not **BxCxHxW** . In your case, **C=1** because you said you only have one channel. **B** is the batch dimension, which is usually added by the dataloader. And I suppose that you have **H=W=512**. But you have no 3rd coordinate (the depth **D** of the image, which should be a cube if you work in 3D imagery).
  
  The 3D pooling is trying to do pooling on the channel dimension, which has value 1 since you only have one channel. Hence it returns an error because you cannot downscale more than one pixel...
  
  You can use the Upsample function that is compatible with 3D images, instead of Resize :
  <https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html>
  
  If you use 2D images then you need to use 2D convolutions.
  
  > **Comments:**
  > **username_0**: I realized that my dataloader was skipping the depth, so I fixed it and it is running fine now. Though I have trouble with loss function. my dice score is coming out to be negative, but I will post the another question for that.  
  
  ---
  
  