I am facing the similar problem. However, I am more interested in knowing how to make my dataloader efficient.
Ideally, I want to read multiple temporal blocks for a video, perform transformations like random crop, rescale, extract I3D features for them, concat them in a tensor and return with other information like text embeddings for question, multiple answers.
For now, I am preprocessing my videos and extracting features beforehand, which does not allow me to use multiple random crops and not a good way. The video frames are 60GB size in total. Any tips?
I am using Ubuntu 16.04, 24gb RAM, 245gb/959gb disk space is free right now, 1 Titan Xp GPU, Python 3.6.4, Pytorch 0.4.1, cuda 8.0.61, cudnn 7102.
Dataset is on local disk.