Seems like you are using the trainer. Can you do print(trainer.args.device), what does that give you? Normally, the trainer should automatically select the TPU if it is available.

  
      github.com
  
  
    huggingface/transformers/blob/3212b8850d20f5adf6242074193854cf1f808c86/src/transformers/training_args.py#L258-L261 1

elif is_torch_tpu_available():
    device = xm.xla_device()
    n_gpu = 0
elif self.local_rank == -1:



  
  
    
    
  
  


So inputs should be converted to XLA tensors automatically.

  
      github.com
  
  
    huggingface/transformers/blob/3212b8850d20f5adf6242074193854cf1f808c86/src/transformers/trainer.py#L658-L660 1

for k, v in inputs.items():
    if isinstance(v, torch.Tensor):
        inputs[k] = v.to(self.args.device)



  
  
    
    
  
  


Can you also print that final dict that you return?