Breaking change in upcoming PyTorch version for weight norm and loading pretrained models
Probably to be fixed around here: https://github.com/huggingface/transformers/blob/bbf3090848cf0ceff98f9465691e9ecce63684a1/src/transformers/modeling_utils.py#L3016

See this issue on PyTorch:
https://github.com/pytorch/pytorch/issues/102999#issuecomment-1623975562


Hi, what's your `transformers` version?

If you use a dev version with the commit of  this PR #https://github.com/huggingface/transformers/pull/24030 included, it should be fine. Let me know if not, thanks.