Fix ESM models buffers
# What does this PR do?

Apparently the keys are important to be loaded from the state dict even if we create them at init.
_The documentation is not available anymore as the PR was closed or merged._<|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24576). All of your documentation changes will be reflected on that endpoint.