Fix custom config loading for clip model
# What does this PR do?

Fixes # (issue)
In the clip model, CLIPTextTransformer and CLIPVisionTransformer loads the original clip config file eventhough custom parameters are given in the new config file.


The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18382). All of your documentation changes will be reflected on that endpoint.<|||||>Hey @avinashsai, I don't understand what you're trying to do: two lines above your changes is a `self.config = config`, so you're working with the same object.

Could you shed some light on what doesn't work so I can help you out? Thanks :)<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.