Hi,Thanks for your reply @lewtun
“It doesn’ work” means there can not load ckpt.
When I use model.load_weights(), there is different output of the model when I run the same code. I think the reason is random init instead of load ckpt.
When I use transformers to load ckpt(also add .index suffix), It show error message like this
2021-06-02 10:10:12.841564: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
Traceback (most recent call last):
  File "C:\anaconda\envs\tf2\lib\site-packages\transformers\modeling_tf_utils.py", line 1271, in from_pretrained
    missing_keys, unexpected_keys = load_tf_weights(model, resolved_archive_file, load_weight_prefix)
  File "C:\\anaconda\envs\tf2\lib\site-packages\transformers\modeling_tf_utils.py", line 467, in load_tf_weights
    with h5py.File(resolved_archive_file, "r") as f:
  File "C:\anaconda\envs\tf2\lib\site-packages\h5py\_hl\files.py", line 408, in __init__
swmr=swmr)
  File "C:\anaconda\envs\tf2\lib\site-packages\h5py\_hl\files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py\h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/", line 13, in <module>
    bert = transformers.TFBertModel.from_pretrained('./bert/bert_model.ckpt.index', config=bert_config)
  File "C:\anaconda\envs\tf2\lib\site-packages\transformers\modeling_tf_utils.py", line 1274, in from_pretrained
    "Unable to load weights from h5 file. "
OSError: Unable to load weights from h5 file. If you tried to load a TF 2.0 model from a PyTorch checkpoint, please set from_pt=True.

code like this:
import tensorflow as tf
import transformers

for gpu in tf.config.experimental.list_physical_devices('GPU'):
    tf.config.experimental.set_memory_growth(gpu, True)

tokenizer = transformers.BertTokenizer.from_pretrained('bert-base-cased')
bert_config = transformers.BertConfig.from_json_file('./bert/bert_config.json')
bert = transformers.TFBertModel.from_pretrained('./bert/bert_model.ckpt.index', config=bert_config)

inputs = tokenizer("Hello, my dog is cute", return_tensors="tf")
outputs = bert(inputs)

last_hidden_states = outputs.last_hidden_state
print(last_hidden_states)