Thanks a lot for your comments!
Yes I’m explicitly loading the config file, which I believe is almost the same in both tf and pytorch.
I’ve tried to use the pytorch conversion script to load the model in pytorch and then convert the pytorch model to tensorflow model, but I can’t find any scripts to convert the pytorch model to tensorflow one…
However, I’ve found a workaround, which is a temper fix: Since the may problem is the mismatch of tensor names, I load the google checkpoint file and rename tensors to match those in transformers. Fortunately the mismatches are easily located and only need sightly modification.