Pipelines: batch size
I'm using a pipeline with feature extraction and I'm guessing (based on the fact that it runs fine on the cpu but dies with out of memory on gpu) that the `batch_size` parameter that I pass in is ignored. Can pipeline be used with a batch size and what's the right parameter to use for that?

This is how I use the feature extraction:
```
# use pipelines and feature extraction
    feature_extractor = pipeline(
            task="feature-extraction",
            model=model_args.model_name_or_path,
            config = config,
            tokenizer = tokenizer,
            framework="pt",
            batch_size=data_args.batch_size,
            truncation=True,
        )

....
outputs = feature_extractor(inputs = predict_inputs, truncation=True)
```

@Narsil has been really helpful with pipelines, perhaps he knows the answer?

Hi,

What's your batch size ?

Do you mind sharing the version of transformers you are running ?
Also what's the model and the predict inputs ?

Batch_size is implemented for this pipeline, getting OOM, means probably that the batch_size is just too big, try setting it at 1 first probably to check if that fixes the issue.

Ideally when you share such an issue if you can provide a reproducible script + an actual error output it helps us tremendously in diagnosing our issue.

Cheers<|||||>I used 16 as the batch size, I installed directly from git yesterday so pretty up to date. When I manually split in the same batch size (so really send the inputs in batches of 16 from the start to the feature extractor), it works, this is what made me suspect that batching may not work. 

There is not much for the error, the job dies with the gpu spitting an OOM error. 

I also noticed another performance issue and I have a snippet of the script there where I manually batched the samples. See #14333 <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>I had a similar issue where the batch_size wasn't changing regardless of what argument I entered. I fixed this by changing the batch and batch_eval values in the config object that you pass into  input_pipeline.get_data_from_tfds()<|||||>Hi @CallumJMac ,

batching doesn't work with TF (yet). I was convinced a warning was in place but I fail to see it now. 

Could that be the issue ?
Also could you share your solution, it could help setting up automated batching on the pipelines for TF. If you're willing to do a PR on it, I would be glad to help too !

Cheers.