Is every input example guaranteed to have 224x224x4x9x3 (5419008) elements? If not, you probably need to adjust the spatial extent (e.g., with something like torchvision.transforms — Torchvision 0.8.1 documentation (pytorch.org) 1 or  AdaptiveAvgPool2d — PyTorch 1.8.1 documentation 1). You might also need to be careful about rearranging your axes with reshape as it does not actually permute the axes (i.e., it does not change data layout).