Hi! I’m the TF maintainer at Hugging Face. I think what you want should be possible, and I agree that it’s annoying that you need PyTorch to use load_tf_weights_in_bert.
I’ve got a deadline to complete right now, but I’m going to try to look at this early next week (Monday/Tuesday). Hopefully I can get the load to work, and then I’ll share how I did it, and possibly update our load_tf_weights_in_bert function if necessary.