The error points to pandas and the self.rotations.iloc[idx, 0] call in particular.
Could you print out idx and use this last idx which causes this error to call the function manually, i.e.:
ants_dataset.rotations.iloc[idx, 0]