Hi @thomwolf,
Initially I started from bert base on the tensorflow hub, https://tfhub.dev/google/bert_uncased_L-12_H-768_A-12/ 5 , dropped the layers after NSP-Dense, added two layers,  fine-tuned to my data, and saved the fine-tuned model with its weights.   I did this using the keras-bert library in tensorflow.    Ideally, I would like to bring the resulting fine-tuned tensorflow-keras model into transformers as a pytorch model, because I want to use Jesse Vig’s bertviz visualisation tool, which is using pytorch.
If that is not possible (and it doesn’t look promising), then I would like to use a transformers bert-base-uncased TFBert model, amend and tune it using tensorflow, then convert the result to pytorch.
Would you expect either of these to be possible?   If so, do I need to use the cli, or just the from_tf=True parameter?