Native integration with pytorch/serve
First off, many thanks to @julien-c for sharing my [blog post](https://medium.com/analytics-vidhya/deploy-huggingface-s-bert-to-production-with-pytorch-serve-27b068026d18) for an example custom handler integration of a BERT sequence classifier model with [pytorch/serve](https://github.com/pytorch/serve/).

# 🚀 Feature request
Provide scripts to automatically generate a [pytorch/serve](https://github.com/pytorch/serve/) custom handler for transformers models to be able to serve models after training. 

Example usage could be:
generate_serving_handler --output-filename transformers_ner_handler.py --handler-type "ner" /path/to/checkpoint_directory 

## Motivation
There is already an increasing need to serve transformers models with pytorch/serve, see for example:
https://github.com/pytorch/serve/issues/249
https://github.com/pytorch/serve/issues/267

My understanding of the goal of transformers is to be able to support NLP users starting with model prototyping, ranging over model training and evaluation all the way into their productive system. Native integration with a serving framework seems like a logical step given this understanding.

There is ongoing work in `pytorch/serve` to showcase how users can write custom handlers for their transformers models. 
However, handler code will have to differ quite significantly for different transformers applications (NER, QA, Summarization, Classification) which will put a heavy burden on the user. 

Technically, this feature could also be integrated on the `pytorch/serve` side, but I'm wondering if it would not be easier to maintain here as one could more easily test that new integrated models can still be served with the handler code. 


## Your contribution
I would be happy to support and submit a PR - for simple sentence classification I have handler code ready, other applications would have to be designed and integrated.
That would be awesome – and might be of interest to @mfuntowicz among others.

We'll help you if needed.<|||||>Awesome! Thanks for the upvotes and interest, very appreciated
@julien-c, thanks for your support of the idea. 

I will create a PR and get started on a basic first version and cross-reference it here. 
@all: Feel free to chime in on the PR to help shape the interface :)<|||||>@MFreidank I was curious if you had an update here! I've implemented a version of a PyTorch serve handler that operates on the  https://huggingface.co/transformers/main_classes/pipelines.html abstraction for a currently unreleased project, as opposed to a distinct model or tokenizer. I think it's a nice abstraction for this use-case, what do you think? <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
