You seem to be on the correct path, could you tell us more about the index error you encountered? What did the stack trace look like? Also could you try briefly with another model than roberta (like bert for instance) and report if the error disappears?