Thanks Felix,
I did explore the option of using tensor-stream as well but ended up giving up on the idea as it doesn’t appear to be all that efficient GPU memory wise. The saving made from not moving the frames from GPU to system RAM back and forth actually would be loss in some of the image manipulations I am having to make to them which, it seems are faster on the CPU and more memory consuming.