Update `test_batched_inference_image_captioning_conditioned`
# What does this PR do?

The test `tests/models/pix2struct/test_modeling_pix2struct.py::Pix2StructIntegrationTest::test_batched_inference_image_captioning_conditioned` starts to fail on CI run of `April 27` which includes the merged PR #23023.

@younesbelkada Could you double check if the changes in this PR are reasonable? Thank you.


_The documentation is not available anymore as the PR was closed or merged._<|||||>@younesbelkada If you can take over this PR to avoid `generate weird output right after`, it would be really nice :-)<|||||>The test should be now fixed, the generated text produces different output than before, probably due to https://github.com/huggingface/transformers/pull/23051 that now made the model using a causal attention mask on the text decoder (which was not the case before)<|||||>Thanks a lot!<|||||>> I think that we should educate users that for text-conditioned generation we should never add special tokens to the tokenizer - as introduced in https://github.com/huggingface/transformers/pull/23004

@younesbelkada The best place to do this I think is in the example docstring for the model, as this is what a lot of users will reference, and it currently doesn't do that. Could you open a PR to update this? <|||||>> @younesbelkada The best place to do this I think is in the example docstring for the model, as this is what a lot of users will reference, and it currently doesn't do that. Could you open a PR to update this?

Sure yes will do! 

> Changes look fine - my only concern is that the generations appear to have become worse. @younesbelkada @ydshieh do we have any other generation samples to make sure the model is behaving as expected?

Yes! I was relieved since we do have the tests `test_batched_inference_image_captioning` & `test_inference_image_captioning` that still pass --> meaning that the un-conditional text generation seem to be unaffected!<|||||>> > @younesbelkada The best place to do this I think is in the example docstring for the model, as this is what a lot of users will reference, and it currently doesn't do that. Could you open a PR to update this?
> 
> Sure yes will do!

I am going to merge this PR and leave @amyeroberts 's suggestion for @younesbelkada in a separate PR. Thank you for the review and the refine of this PR. 