The config is part of the logs of this script, so it’s logical you have it displayed, so using another model solves your issue. There is a weird thing in the positional embeddings of roberta, you might have to add 2 to max_position_embeddings for it to work without indexing error.