Sorry
you are right
the temporal dimension is variable as
CNN Shape==========> torch.Size([16, 128, 40, 21])
CNN Shape==========> torch.Size([16, 128, 40, 28])

This is what I get output from the CNN.
Now how do we handle this variable length in RNNs (this is on the fly training)