If I remember correctly TensorRT was breaking API so I think that you need to use TensorRT 7:

  

      github.com/tensorflow/tensorflow
  

  
    
  
	  
  

  
    
      Compilation fails on Ubuntu 20.04 when using TensorRT 8.  9
    

    
      
        opened 
        
          
        
        May 13, 2021
      
      


      
        
          
          feranick
        
      
    

    
        stat:awaiting tensorflower
        type:bug
        subtype: ubuntu/linux
        comp:gpu:tensorrt
        TF 2.5
    
  


  
    **System information**
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04…): Ubuntu 20.04
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
- TensorFlow installed from (source or binary): binary
- TensorFlow version: 2.4.1, 2.5, etc
- Python version: 3.8
- Installed using virtualenv? pip? conda?: no, built from source
- Bazel version (if compiling from source): 3.1 (for TF 2.4.1), 3.7.2 (for TF 2.5.0-rcx)
- GCC/Compiler version (if compiling from source): gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
- CUDA/cuDNN version: Cuda 11.1, cudnn8 (8.0.5.39-1+cuda11.1) or Cuda-11-2, libcudnn 8.1.1, 8.2, 
- GPU model and memory: GTX-1080ti
- TensorRT (crucial): 8.0.0-1+cuda11.0, or 8.0.0-1+cuda11.3

**Describe the problem**
When compiling with support for TensorRT 8 (via libnvinfer8), compilation fails (log is below). 

**Provide the exact sequence of commands / steps that you executed before running into the problem**
When configuring the build, make sure you build with TensorRT support, and make sure TensorRT version 8 is selected. Build TF as usual. Compilation will fail. 

If you install  TensorRT version 7 manually (from debs available for Ubuntu 18.04), compilation will complete just fine.

**Any other info / logs**
Relevant error: 
`C++ compilation of rule '//tensorflow/compiler/tf2tensorrt:tensorrt_stub' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command`

`In file included from bazel-out/k8-opt/bin/external/local_config_tensorrt/_virtual_includes/tensorrt_headers/third_party/tensorrt/NvInfer.h:54,
                 from tensorflow/compiler/tf2tensorrt/stub/nvinfer_stub.cc:17:
bazel-out/k8-opt/bin/external/local_config_tensorrt/_virtual_includes/tensorrt_headers/third_party/tensorrt/NvInferRuntime.h:2264:51: note: from previous declaration 'nvinfer1::IPluginRegistry* getPluginRegistry() noexcept'
 2264 | extern "C" TENSORRTAPI nvinfer1::IPluginRegistry* getPluginRegistry() noexcept;`

Full log here: 
[gesdm-tf2.5.0rc3-error.txt](https://github.com/tensorflow/tensorflow/files/6469944/gesdm-tf2.5.0rc3-error.txt)
  

  

  
    
    
  

  


But Nvidia has a DRAFT PR for TensorRT 8.2 that you can follow at:

  

      github.com/tensorflow/tensorflow
  

  
    
  
    
  

  
    
      Add Support for TensorRT 8.2 23
    

    
      tensorflow:master ← jhalakp-nvidia:trt-8.2
    

    
      
        opened 
        
          
        
        Oct 12, 2021
      
      

      
        
          
          jhalakp-nvidia
        
      

      
        
          +282
          -42
        
      
    
  


  
    1. Add stub files for TensorRT 8.2
2. Fix accuracy regression with MobilenetV2 …i.e. set dynamic range for (-6, 6)
3. Limit GPU memory utilization for TFTRT Integration tests
4. Fix incorrect usage of TRT version to string API
5. TensorRT model_tests update
- Add separate perf tolerance for FP32, FP16, and INT8
- Add separate accuracy tolerance for FP32, FP16, and INT8
- Add functionality to dump histograms for better accuracy validation 

CC: @DEKHTIARJonathan, @tfeher , and @bixia1