The output of TFAutoModel-save_pretrained and keras-ModelCheckpoint do not equal.
### Describe the bug

```
history = model.fit(
    tf_train_dataset, validation_split=0.01,
    epochs=int(training_args.num_train_epochs),
    callbacks=callbacks,
)

model.save_pretrained(checkpoint_local)

```
output: `h5` file


```
callbacks = [tf.keras.callbacks.ModelCheckpoint(checkpoint_local)]

history = model.fit(
    tf_train_dataset, validation_split=0.01,
    epochs=int(training_args.num_train_epochs),
    callbacks=callbacks,
)
```
output: `pb` file and `assets` and `variables`


### System info

```shell
transformers = 4.26

python = 3.8
```

Hi @guotong1988 , I think this issue is more related to the `transformers` library so I'm transferring the issue to the corresponding repo. I'll let @sgugger @ydshieh comment about the issue itself.<|||||>@guotong1988 

It's not clear to me what question you have in mind. Do you mean one output `h5` file and another one output `pb` file (and others), and you think both of these 2 methods should output the same (set of) file(s)? Or you mean other thing(s)?
<|||||>Sorry for the late response.

Yes! @ydshieh Thank you!

These 2 methods should output the same. 

`h5` file is preferred.

In fact, I need to output the model file during training, while using the `callbacks`.<|||||>I refer the code here https://github.com/huggingface/transformers/blob/main/examples/tensorflow/language-modeling/run_clm.py#L587<|||||>@guotong1988 These are two different methods of saving models to different formats. It's normal that they don't give the same format. If you need a `.h5` file as well as other files (like configuration file, tokenizers from `transformers`), you can always add a line `model.save_pretrained(checkpoint_local)` in your script/notebook.<|||||>Thank you.

How can I put `model.save_pretrained` into `callbacks`?

So that I can save the model for each epoch.<|||||>There is [PushToHubCallback](https://huggingface.co/docs/transformers/main/en/main_classes/keras_callbacks#transformers.PushToHubCallback).

The goal of this callback is to save and push to the Hub - I am not sure if we can only save but not to push though.
It might be great if you also push the checkpoints to the Hub. If you don't want to push but just save, I will cc @Rocketknight1 :-)
<|||||>Yes, I don't want to push but just save.<|||||>@guotong1988 If you want to proceed quickly, you can modify the code of the class `PushToHubCallback` to remove the part that pushes the checkpoints.