BLIP-2 batch generate error
### System Info

`transformers` version: 4.27.0.dev0

### Who can help?

@NielsRogge  

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The following error happens when running BLIP-2 model's generate() with num_beams>1 and input batch_size>1

<img width="892" alt="Screenshot 2023-02-13 at 6 53 28 pm" src="https://user-images.githubusercontent.com/13638455/218443029-8bc45f00-4884-4785-95a9-228aa08d1266.png">


### Expected behavior

The model should be able to do batch generate.
Can confirm that:
1. The issue only happens with `batch_size>1`
2. The issue happens with both `num_beams>1` and `num_return_sequences>1` (they both rely on input replication, which is my suspicion)
3. https://github.com/huggingface/transformers/pull/21580, which addresses some BLIP2 `.generate()` issues does not fix this issue<|||||>I think that [this](https://github.com/salesforce/LAVIS/blob/3ac397aa075c3e60b9521b012dda3660e3e35f1e/lavis/models/blip2_models/blip2_opt.py#L213) is not incorporated in our current implementation. It seems the authors only defined this for OPT but not for T5.<|||||>@NielsRogge yeah, that's the fix 👍 However, I'm generalizing the PR to correctly expand any model input, as it is a bit limited at the moment (it is expanding tensors with specific names, as opposed to all tensors that might be used as model input).<|||||>FYI, I have tried to do repeat_interleave for the inputs_embeds, but that results in another error.
T5 does not seem need this because of the encoder-decoder architecture.<|||||>@LiJunnan1992 that's correct, only non-encoder inputs need the expansion. In a nutshell, in the generation loop, we have a 1:1 input-output row correspondence, so we need to expand the model inputs before the loop accordingly.

T5-BLIP2 sends `inputs_embeds` to the (text) encoder, whereas OPT-BLIP2 has no (text) encoder at all. In the former, the encoder outputs need to be expanded, whereas in the latter the `inputs_embeds` need the expansion treatment.

#21624 takes care of all those cases for any model input in `model_kwargs`, which should future-proof `.generate()`<|||||>@LiJunnan1992 if you install `transformers` from `main`, it should be working 🙏 <|||||>@gante I can verify that this is working now, thanks! When will I be able to pip install this version?<|||||>@LiJunnan1992 we aim at monthly releases, so 1-2 weeks from now :)