Is your question solved？What is the solution to this problem? The reason why the pre-training model failed to load  have nothing to do with the Transformers version. I also hive this question, when i use  pre-training model ,named ’ uer/chinese_roberta_L-8_H-512’. my Transformers version is 3.4.0.