Thank you @sgugger Apparently the ‘input’ is in floats. The task I am doing is token classification, so I have input_ids, which are all ints, and corresponding labels, which are also ints. Maybe model by default would initialize the input as floats, or the trainer? I am not sure. I have found some other issue with the data itself, so I am fixing it now. I may share more information later. Thanks again.