Fix `check_config_attributes`: check all configuration classes
# What does this PR do?

As @NielsRogge points out to me, the `check_config_attributes` doesn't check all configuration classes (to make sure all `__init__` arguments are really used).

This is because `CONFIG_MAPPING` doesn't contain all configuration classes, in particular, the vision/text config classes for models like Blip2 or CLIP.

This PR fixes this issue, and remove some unused arguments/attributes in some configuration classes.
_The documentation is not available anymore as the PR was closed or merged._