Why is the memory usage for ConvTranspose3d so high?
import torch

from utils.memory_profiling import format_memsize, tensor_size

torch.backends.cudnn.benchmark = True

x = torch.rand(1, 128, 8, 270, 480)
conv = torch.nn.ConvTranspose3d(128, 64, kernel_size=5, stride=2, padding=2, output_padding=1)

x = x.to('cuda:0')
conv = conv.to('cuda:0')
y = conv(x)

Raises:
Traceback (most recent call last):
  File "bug.py", line 15, in <module>
    conv(x)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/conv.py", line 895, in forward
    output_padding, self.groups, self.dilation)
RuntimeError: CUDA out of memory. Tried to allocate 30.90 GiB (GPU 0; 10.92 GiB total capacity; 2.48 GiB already allocated; 5.83 GiB free; 1023.50 KiB cached)

x has shape (1, 128, 8, 270, 480) and should be 506.2 MiB
y should have shape (1, 3, 8, 540, 960) and should be 189.8 MiB
So why does cuda try to allocate 30.90 GiB?
This happens regardless of benchmark and cudnn flags (supposing I am setting them correctly).
I am using pytorch 1.0.0, though it also occured in 0.4.4 on a GeForce GTX 1080 Ti.