CI with layers=2
# What does this PR do?

Running a (sub) set of 24315 tests (given by test fetcher) - only tests in `test_modeling_xxx.py`.
(for a full run like nightly run, it doesn't seem change anything about running time - need more investigation)

Running time:

- num_layers = mixed (2, 3, 4, 5, 6) - currently `main`
  - torch: 16m
  - tf: : 8m
  - flax: 11m30

- num_layers = 2
  - torch: 12m30
  - tf: 8m (not sure nothing change)
  - flax: 8m30
_The documentation is not available anymore as the PR was closed or merged._