In my experience it doesn’t help.
Also, when you look at the code, you can see that once it encode all the given examples, it iterate over them one by one (transformers.pipelines — transformers 4.0.0 documentation):
all_answers = []
for features, example in zip(features_list, examples):
    model_input_names = self.tokenizer.model_input_names + ["input_ids"]
    fw_args = {k: [feature.__dict__[k] for feature in features] for k in model_input_names}

    # Manage tensor allocation on correct device
    with self.device_placement():
        if self.framework == "tf":
            fw_args = {k: tf.constant(v) for (k, v) in fw_args.items()}
            start, end = self.model(fw_args)[:2]
            start, end = start.numpy(), end.numpy()
        else:
            with torch.no_grad():
                # Retrieve the score for the context tokens only (removing question tokens)
                fw_args = {k: torch.tensor(v, device=self.device) for (k, v) in fw_args.items()}
                start, end = self.model(**fw_args)[:2]
                start, end = start.cpu().numpy(), end.cpu().numpy()