Hi,
Have you fixed this ? I have the same problem, very slow when using caffe2 backend on GPU. And I think it’s because the input is on CPU .numpy(), so the question can be how to move the input on gpu in order to use it with caffe2 backend.
with a similar code I have this result about inference speed:
pytorch CPU (3s) > onnx/caffe2 backend GPU (600ms) > onnx/caffe2 backend CPU (200ms)> pytorch GPU (50ms).
I would expect onnx/caffe2 backend GPU to have a better speed but it is not the case.