[Mask2Former] Remove SwinConfig
# What does this PR do?

This PR removes what was probably a leftover from the Mask2Former PR, the model works without requiring those lines of code.

Fixes #24244
_The documentation is not available anymore as the PR was closed or merged._<|||||>gently ping @amyeroberts to see if we could merge 🤗 