I use TensorRT for inference and implemented the required plugins from the PyTorch souce code. You can find my source code for Tensor RT below. If you use ONNX Runtime for inference, I presume you will have to make and register a custom plugin for that. Unfortunately I don’t have any experience with ONNX other than using the graphs as an intermediate representation for Tensor RT to parse so I can’t really help with that.
As an aside (as an improvement on my previous solution), you can ensure correct typing and keyword arguments with the function defined below.
@parse_args('v', 'v', 'i', 'i', 'b')
def grid_sample_op(g, input1, input2, mode, padding_mode, align_corners):
    return g.op("torch::grid_sampler", input1, input2, interpolation_mode_i=mode,
                padding_mode_i=padding_mode, align_corners_i=align_corners)


  
      
      github.com
  
  
    

5had3z/CerberusNet 25

master/runtime/cerberus_net/trt_plugins

  The aim of this project is to generate a pointcloud with motion vectors and semantic segmentation of stereoscopic image sequences. Targeting real-time deployment on an nVIDIA Xavier. - 5had3z/Cerbe...