Request for feature for setting batch size in pipeline when inference
```
from transformers import pipeline

from transformers import AutoModelWithLMHead, AutoTokenizer
model = AutoModelWithLMHead.from_pretrained("Helsinki-NLP/opus-mt-en-fr")
tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-fr")
nlp = pipeline("translation_en_to_fr", model=model, tokenizer=tokenizer, device=0)

nlp(ds.df_train.sample(32)['content'].tolist(), max_length=300)
```
I am suing pipeline instance for inference on trunks of sentences.
When the size of the trunk data is small, like the 32, it is ok to fit into the GPU memory.

However, when increasing the size of this input,  memory error comes out:

> RuntimeError: CUDA out of memory.

Is there any way to set the batch size inside the `nlp()` to automatically to fit into the GPU to make trunks of inferences?
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.