Batched pipeline
Hi, 

Is there a way to run batches with QuestionAnsweringPipeline rather than just one example?

Thanks.
Yes, building off of the model and example [here](https://huggingface.co/twmkn9/albert-base-v2-squad2): 

```
from transformers.pipelines import pipeline

model_name = "twmkn9/albert-base-v2-squad2"
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
    'question': 'Why is model conversion important?',
    'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'}, {
    'question': 'What is the name of the repository ?',
    'context': 'Pipeline have been included in the huggingface/transformers repository. '
    }
res = nlp(QA_input, handle_impossible_answer=True)
print(res)
# [{'score': 0.2479676753282547, 'start': 59, 'end': 132, 'answer': 'gives freedom to the user and let people easily switch between frameworks.'}, {'score': 0.5168691277503967, 'start': 35, 'end': 71, 'answer': 'huggingface/transformers repository.'}]


```

<|||||>Hi. 

I used your example for testing. It seems like even though I put multiple question-context pairs in as input, it really is just doing a one-by-one prediction on them in the background. 
So for 1 example the inference time is: 0.56 sec
For 2 examples the inference time is: 1.05 sec
For 16 examples it is: 8.4 sec., etc..

Is there a way to do batch inference with the model to save some time ? (I use 12 GB gpu, transformers 2.4.0 or 3.2.0)
 
Thanks!<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Possible duplicate of #3007<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.<|||||>still an open point. highly required, any information on the progress?<|||||>This is implemented in recent versions: https://huggingface.co/docs/transformers/master/en/main_classes/pipelines#pipeline-batching

cc @Narsil <|||||>For the sake of completion: 
```python
pipe = pipeline('question-answering', model=model_name, tokenizer=model_name)
questions = [{"question": "Who am I ?", "context": "There is something about me"}, .... ]
for answer in pipe(questions, batch_size=16):
    print(answer)
````