Hi everyone!
I have a neural network that starts with some convolutional layers, then an LSTM layer and finally some deconvolutional layers. Kind of encoder-decoder architecture with LSTM in the middle.
A sample in my dataset is a sequence of 4 images with shape [4, 3, H, W]. So, when I want to use batches, with batch_size=8 for example, the resulting tensor would have shape [8, 4, 3, H, W].
The problem is that Conv2d layer does not accept 5-dimensional tensors, it expects 4-dimensional tensors.
I’ve tried to reshape the tensor like this: x.view(-1, 3, H, W), resulting this shape: [32, 3, H, W]. But this is not correct for me, as sequences are not related to each other, since they are picked randomly. So sequence 1 (4 contiguous images) in the batch is not the previous to sequence 2 (other 4 contiguous images) in this batch.
How can I feed my network with this 5-dimensional tensor in order to keep the batch information and also the sequence information?