Hi I soveld the same problem , you can go
  

      github.com/triton-inference-server/server
  

  
    
  
	  
  

  
    
      PyTorch execute failure: isTensor(); Expected Tensor but got GenericList 38
    

    
      
        opened 
        
          
        
        Sep 10, 2021
      
      

        
          closed 
        
          
        
        Sep 10, 2021
      
        

      
        
          
          rexlow
        
      
    

    
    
  


  
    **Description**
I've built a custom Yolov5 model and traced it via a [export sc…ript](https://github.com/ultralytics/yolov5/blob/master/export.py) from UltraLytics. Testing it via their [inference script](https://github.com/ultralytics/yolov5/blob/master/detect.py) and my Python native code works. Now I attempt to deploy it with Triton.

While deploying it Triton did not throw any error, it is when making a client call only I get this error.

A similar error can be found in #2594 , not sure why is this closed?? Perhaps @CoderHam can have a look at this issue? I'm passing a batched numpy array not tensor.

```
PyTorch execute failure: isTensor() INTERNAL ASSERT FAILED at "/tmp/tritonbuild/pytorch/build/include/torch/ATen/core/ivalue_inl.h":157, please report a bug to PyTorch. Expected Tensor but got GenericList
Exception raised from toTensor at /tmp/tritonbuild/pytorch/build/include/torch/ATen/core/ivalue_inl.h:157 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6c (0x7f56f28405cc in /opt/tritonserver/backends/pytorch/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0xfa (0x7f56f2806d4e in /opt/tritonserver/backends/pytorch/libc10.so)
frame #2: <unknown function> + 0xf558 (0x7f56f2d9a558 in /opt/tritonserver/backends/pytorch/libtriton_pytorch.so)
frame #3: <unknown function> + 0x155a2 (0x7f56f2da05a2 in /opt/tritonserver/backends/pytorch/libtriton_pytorch.so)
frame #4: TRITONBACKEND_ModelInstanceExecute + 0x411 (0x7f56f2da1a91 in /opt/tritonserver/backends/pytorch/libtriton_pytorch.so)
frame #5: <unknown function> + 0x2f2fd7 (0x7f573ca89fd7 in /opt/tritonserver/bin/../lib/libtritonserver.so)
frame #6: <unknown function> + 0xfdfe0 (0x7f573c894fe0 in /opt/tritonserver/bin/../lib/libtritonserver.so)
frame #7: <unknown function> + 0xd6d84 (0x7f573c2d1d84 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
frame #8: <unknown function> + 0x9609 (0x7f573c76c609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0)
frame #9: clone + 0x43 (0x7f573bfbf293 in /usr/lib/x86_64-linux-gnu/libc.so.6)
```

**Triton Information**
Triton docker version 21.03

**Config file**
```
name: "my_model"
platform: "pytorch_libtorch"
max_batch_size : 128
input {
    name: "input__0"
    data_type: TYPE_FP32
    format: FORMAT_NCHW
    dims: [ 3, 640, 640 ]
}
output {
    name: "output__0"
    data_type: TYPE_FP32
    dims: [ 20 ]
    label_filename: "labels.txt"
}
instance_group {
  count: 1
  kind: KIND_GPU
  gpus: 0
}
dynamic_batching {
  preferred_batch_size: [ 1, 2, 4, 8, 16 ]
  max_queue_delay_microseconds: 100
}
```

**Python Client**

```
image = cv2.resize(img, (640, 640), interpolation = cv2.INTER_CUBIC)
image = image.astype(np.float32)
image /= 255. # [640, 640, 3]
image = np.transpose(image, (2, 0, 1)) # [3, 640, 640]
data = np.expand_dims(image, 0) # [1, 3, 640, 640]

input_1 = tchttp.InferInput("input__0", (1, 3, 640, 640), "FP32")
input_1.set_data_from_numpy(data)
outputs = [tchttp.InferRequestedOutput(name) for name in ["output__0"]]
response = triton_client.infer(
  mmodel_name='my_model', 
  inputs=[input_1],
  outputs=outputs,
  model_version='1')
logits = response.as_numpy('output__0')
```
  

  

  
    
    
  

  



Otherwise I also try to create a wrapper code around the my model . just use split、cat、squeeze、narrow function to make multiple inputs combined into one

like this
    class model():
    ...
    def forward(self,inputs):
        image, points = torch.split(inputs, 1)
        points = points.squeeze(1).narrow(1, 0, 1).squeeze(1)



inference model

    x0=torch.ones(1, 3, 28, 28).to(device)
    x1=torch.ones(1, 28, 28).to(device)
    x1 = x1.unsqueeze(1).expand(1, 3, 2, 2)
    inputs = torch.cat([x0, x1])
    
    model(inputs)