TF: use the correct config with `(...)EncoderDecoder` models
# What does this PR do?

Fixes #18071 

Modifies `unpack_inputs` to ignore the config file for `(...)EncoderDecoder` models, mimicking the behavior in PT. If we don't ignore it, then unset options will get set with the config's default (`False` for most of them), causing the inner models to ignore their own config files.

⚠️ I've added a corresponding test for the `EncoderDecoder` models. I then noticed that other `(...)EncoderFecoder` tests have copy/pasted their own `EncoderDecoderMixin`, so I've left the other classes for a follow-up PR with the following question: should a common `EncoderDecoderMixin` be defined and shared across `(...)EncoderDecoder` tests, or should I add a similar test to all other classes individually?
_The documentation is not available anymore as the PR was closed or merged._<|||||>I believe they are -- going to give it a go afterwards if @ydshieh also agrees :)<|||||>I have limited connection at this moment in the mountain, so feel free to merge if you prefer. Regarding the common mixin, good for me. I see there are a few little things to address, like the input names (input_ids, pixel values etc).
Would be nice if you do this refactorization after merging my PR about the PT/TF equivalence tests, or incoperate the change in it 🙏 

Thank you for the fix, @gante<|||||>@ydshieh can I have a review plz 🙏 <|||||>@ydshieh rebased with main and reran tests -- all working 👍 