Thanks for your return. My take, the bottleneck is the disk search time.
Here is what I do more precisely. I burst videos to frames first. Then in data_loader get() function, I pick 32 frames for each video, load each frame by opencv and aggregate them into a single tensor (which is 3x32x224x224 adn 32 is seq length). I’ve not used line_profiler yet but as I observed  iotop read speed is limited by 5MB/sec with 8 workers (optimal number of workers) on a drive whose theoretical best is 350MB/sec. I think, it is all about disk search latency (it is HDD unfortunately).
Now we are working on a binary file format which can be used by Pytorch and we are able to increase the load speed to 34 MB/sec. If you are interested, we can discuss more about this. I think it would be a good extension for Pytorch too.
Please let me know, if you have any comment.