Ah now I understand the source of your OSError: it seems that from_pretrained expects your model to be serialised as tf_model.h5 instead of the TensorFlow checkpoint format. You can see this in the source code here: transformers/modeling_tf_utils.py at master · huggingface/transformers · GitHub 6
If you’re happy to use the PyTorch API, it seems that it might be possible to load .ckpt files with from_pretrained: transformers/modeling_utils.py at master · huggingface/transformers · GitHub 2
In that case you could try something like
config = BertConfig.from_pretrained("path/to/your/bert/directory")
model = BertModel.from_pretrained("path/to/bert_model.ckpt.index", config=config, from_tf=True)