It seems you might be loading a state_dict, which isn’t compatible to the current model definition.
I would recommend to post this question in the HuggingFace discussion board 2 as you would find more experts knowing this model there.