I think you need to either keep the batch and channel dimension or combine those two, but you shouldn’t combine the batch, height, and width dimensions.  So your resulting tensor should be (100, 1024), then you do tensor.reshape(H, W, batch_num, C).permute(2, 3, 0, 1).  You’ll also have to pay attention to how you permute the tensor before your operation.  Honestly, I would try to have your resulting tensor be of dimensions (1024, 100) or (2, 512, 100) and not the transpose, but it’s not 100% necessary to do so.  There really is not enough information here to see exactly what you are doing.  If you could provide some code, or mathematical equations, it would be easier to help you.