The code is working now. I removed to_tensor() and is running fine. I see now that the statement was redundant. Although that is the procedure suggested in the Hugging Face official documentation (Fine-tuning a pretrained model — transformers 4.11.3 documentation 2) and TensorFlow wasn’t throwing an error until now. Anyway, I’m glad that is running now.
This solution was suggested here: AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'to_tensor' - Stack Overflow 44