Thanks for clarification - I see in the docs 1 that one can indeed point from_pretrained a TF checkpoint file:

A path or url to a tensorflow index checkpoint file (e.g, ./tf_model/model.ckpt.index ). In this case, from_tf should be set to True and a configuration object should be provided as config argument. This loading path is slower than converting the TensorFlow checkpoint in a PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.

It seems that what you’re missing is the from_tf=True argument, so maybe something like the following works:
config = BertConfig.from_pretrained("path/to/your/bert/directory")
model = TFBertModel.from_pretrained("path/to/bert_model.ckpt.index", config=config, from_tf=True)

I’m not sure whether the config should be loaded with from_pretrained or from_json_file but maybe you can test both to see which one works