Say you have a set of D frames in every video of size  h x w, so for every batch you first fetch the frames ,assuming that each frame is RGB image, each frame is of size h x w x 3, now stack the frames to get the shape D x h x w x 3, now permute the along the axis namely T (D x h x w x 3) -> T.permute(3,0,1,2) , the batch size will be set by the dataloader itself.