I am using a resnet to do phone recognition from spectrograms. My input is a tensor of shape N C H W, where N is the batch size. But since the speech has different length in time, this causes H to be different in a batch, and I cannot crop the input because they are speech data… Right now, I have to pad zeros within a batch. I am wondering is there a similar way as packed padded sequence in RNN to let all modules in the network know that these  zeros are useless, do not compute the gradient of any parts resulted from them?