That’s true we recently tried to uniformize many aspect of transformers. Yet, for pipeline it raises three concerns:
1. Performances
If you look at the current implementation of pipelines, we rely on a framework-specific part for inference through the model and then convert everything needed to Numpy array in order to post-process and extract the higher-level information from the model (QA span extraction for instance).
This works well, but it implies some internal memory-copy and layout change to go from one framework to another. This is even more true if a tensor lives on the GPU.
Also, some NumPy operators doesn’t support batching whereas PyTorch & TensorFlow alternatives do support it. It would be much more efficent to leverage those operators for pipelines.
2. Tracing
Another point we would like to give acces to the user is ensuring an easier way to trace/export a model.
Currently, with the current implementation of pipelines mixin (PyTorch/Tensorflow) & Numpy operators we cannot really export an entire model from the actual Neural Network to the post processing steps.
TorchScript, TensorFlow Graph or ONNX all rely on the operations supported by the exporting framework to generate a static graph representation. Having NumPy operators in the middle breaks this compatibility and prevent us to export monolothic block which, hopefully, should work out of the box for a wide variety of tasks.
3. Training
Last but not least, as we would like to introduce training capability to the pipelines, we need to provide some closer integration with frameworks.
Again, PyTorch & TensorFlow creates a graph representation of the network during the forward pass. This graph is created by using some internal mechanisms specific to each framework (i.e. grad_fn for PyTorch for instance) tightly coupled with the operators from that specific framework.
All the operations within the network requiring gradients need to be included in the graph for the chain-rule to apply and autograd to compute the actual gradients for each of them. This doesn’t work if you put NumPy specific operators in the middle.