[ESM] fix `accelerate` tests for esmfold
# What does this PR do?

FIxes slow tests that were not passing for `ESM`. In fact I was running `RUN_SLOW=1 pytest tests/models/esm/test_modeling_esm.py` and forgot to run `RUN_SLOW=1 pytest tests/models/esm/test_modeling_esmfold.py`

cc @sgugger @Rocketknight1 
_The documentation is not available anymore as the PR was closed or merged._