I’m interested in exactly the same thing.
However, if this is possible, and you’re in a multiprocessed environment, I’m afraid you’re gonna have big spikes of GPU memory (every process has to load at least 1GB of cuda kernels to use torch GPU, and that’s not even talking about the space to store the frames you’re gonna be manipulating). Ignore if you’re not though.
Anyways, I’m currently looking at this GitHub - osai-ai/tensor-stream: A library for real-time video stream decoding to CUDA memory 8. I haven’t tested it yet. If it’s what you’re looking for, please share your results. Or if you found another way, please keep me posted.
PS : I suggest you could change the name of the topic to be more on point with what you are describing. What about “Video loading and preprocessing fully on GPU” or something like that ?