I’m having an issue using pycuda (for TensorRT) and pytorch together. When I move a “random_tensor” to the gpu the below script fails. If the “random_tensor” is left on the cpu this script completes without error. Is it something to do with cuda contexts clashing between pycuda and pytorch?
I can include more code if necessary.
import tensorrt as trt
import torch
import pycuda.driver as cudadriver
import pycuda.autoinit as cudacontext

random_tensor = torch.ones(1)
sample_tensor = torch.randint(0, 255, size=(3,32,32))
engine = UnifiedTensorRTEngine(
        tensorrt_file="./dummy.trt",
        output_shape=(1,10)
    )
use_gpu = True

if use_gpu:
    random_tensor = random_tensor.cuda()          #FAILS
else:
    pass                                          #PASSES

print(engine(sample_tensor))

Here’s the error that results when I move the tensor to gpu:
[TensorRT] ERROR: ../rtExt/cuda/cudaFusedConvActRunner.cpp (313) - Cuda Error in executeFused: 400 (invalid resource handle)
[TensorRT] ERROR: FAILED_EXECUTION: std::exception

And the output is all zeros:
tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])

Here’s the source code for pycuda.autoinit that sets up the default cuda context:
from __future__ import absolute_import
import pycuda.driver as cuda

# Initialize CUDA
cuda.init()

from pycuda.tools import make_default_context
global context
context = make_default_context()
device = context.get_device()

def _finish_up():
    global context
    context.pop()
    context = None

    from pycuda.tools import clear_context_caches
    clear_context_caches()

import atexit
atexit.register(_finish_up)


My Environment:
Package Versions:
            numpy==1.19.0
            Pillow==5.4.1
            torch==1.5.0
            torchvision==0.6.0
            onnx=1.7.0
            onnxruntime=1.2.0
            pycuda==2019.1.2
            tensorrt==7.0.0.11