Make Nezha tests slow
# What does this PR do?

The Nezha model tests are fairly slow (see below) so this PR marks them as such. @sijunhe if you have an idea on how to make them faster, it's more than welcome!

Current times on main:
```
46.62s call     tests/models/longt5/test_modeling_longt5.py::LongT5TGlobalModelTest::test_export_to_onnx
38.86s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_attention_outputs
35.41s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_save_load_fast_init_to_base
35.37s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_save_load_fast_init_from_base
28.36s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_head_pruning_integration
26.86s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_hidden_states_output
26.62s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_feed_forward_chunking
26.54s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_head_pruning_save_load_from_pretrained
26.08s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_save_load
25.02s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_correct_missing_keys
22.04s call     tests/models/detr/test_modeling_detr.py::DetrModelTest::test_attention_outputs
20.56s call     tests/models/detr/test_modeling_detr.py::DetrModelTest::test_save_load_fast_init_from_base
19.49s call     tests/models/longt5/test_modeling_longt5.py::LongT5ModelTest::test_export_to_onnx
18.63s call     tests/models/detr/test_modeling_detr.py::DetrModelTest::test_save_load_fast_init_to_base
18.27s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_torch_fx_output_loss
18.20s call     tests/models/flava/test_modeling_flava.py::FlavaImageCodebookTest::test_save_load
17.45s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_torch_fx
17.15s call     tests/models/flava/test_modeling_flava.py::FlavaImageCodebookTest::test_feed_forward_chunking
16.16s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_tie_model_weights
16.05s call     tests/models/detr/test_modeling_detr.py::DetrModelTest::test_save_load
15.34s call     tests/models/flava/test_modeling_flava.py::FlavaImageCodebookTest::test_determinism
14.88s call     tests/models/detr/test_modeling_detr.py::DetrModelTest::test_feed_forward_chunking
14.87s call     tests/models/mobilevit/test_modeling_mobilevit.py::MobileViTModelTest::test_save_load_fast_init_from_base
14.62s call     tests/models/detr/test_modeling_detr.py::DetrModelTest::test_hidden_states_output
14.45s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_headmasking
14.20s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_model_outputs_equivalence
14.03s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_initialization
13.81s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_inputs_embeds
13.80s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_head_pruning
13.58s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_determinism
13.36s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_head_pruning_save_load_from_config_init
13.16s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_forward_signature
13.05s call     tests/models/data2vec/test_modeling_data2vec_audio.py::Data2VecAudioModelTest::test_mask_time_prob_ctc
13.00s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_gradient_checkpointing_backward_compatibility
12.99s call     tests/models/nezha/test_modeling_nezha.py::NezhaModelTest::test_gradient_checkpointing_enable_disable
```
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18103). All of your documentation changes will be reflected on that endpoint.<|||||>Hi @sgugger. I am not sure why the Nezha tests are slow. The tests that you listed here are not the integration test with the full model, but the model tests with test config (which is be pretty small and the same size as the regular bert test). I can try to decrease the test config size to see if it helps.<|||||>Found the issue and here is the fix #18106