Are you seeing the same increased memory usage, if you replace cv2.VideoCapture and vidcap.read() with random numpy arrays? This would help to narrow down, which part of the code is responsible for the increased memory usage.