You can return a list instead of a tensor if your dimensions are missmatching.
Obviously torch.stack requires both tensors to share dimensionality.
You can pad one of them up to the size of the other but list seems simpler.
Returning a list you will get a a list of list of frames