Just for reference, here is the sequence of shapes when I pass my model through my network
torch.Size([1, 3, 128, 128])
torch.Size([1, 32, 126, 126])
torch.Size([1, 64, 62, 62])
torch.Size([1, 128, 30, 30])
torch.Size([1, 115200])
torch.Size([1, 100])
torch.Size([1, 115200])
torch.Size([1, 128, 30, 30])
torch.Size([1, 64, 62, 62])
torch.Size([1, 32, 126, 126])
torch.Size([1, 3, 128, 128])