Using a model serialized as torchscript.
While inference, getting the following error at model’s forward method.
Doing type conversion here. Can’t use tensor.type(torch.float16). Leads to a different error. Hence creating a new tensor. But can’t infer using this approach either.
out = self.decoder(x).clone().detach()
cast = torch.tensor(out , dtype=torch.float16)
               ~~~~~~~~~~~~ <--- HERE
return cast
RuntimeError: Cannot input a tensor of dimension other than 0 as a scalar argument