Recommended way for creating distillBERT container and serving 
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
As per documentation, I am supposed to load distilbert as below.

question_answering_model = torch.hub.load('huggingface/pytorch-transformers', 'modelForQuestionAnswering', 'distilbert-base-uncased-distilled-squad')
question_answering_tokenizer = torch.hub.load('huggingface/pytorch-transformers', 'tokenizer', 'distilbert-base-uncased-distilled-squad')

I'm using google Cloud Run. It bring up the container (and hence the model) only upon request. This causes download and load delay.

How to pre-download the model and serve? I am looking for dockerfile step where it could install the weights file and other files needed for the model. This way, I am hoping that my dynamic delays are reduced and I get inference must faster.

Please let me know if such a thing is possible.

thanks
Ishwar




You can find the s3 URL of models here for distilbert: https://github.com/huggingface/transformers/blob/master/transformers/configuration_distilbert.py

If you build the docker on your machine, first download model files on your machine.
Then just add those files to your container through Dockerfile.

If you want your Docker build to download from s3, you can install `aws-cli` in Dockerfile and run `aws s3 cli`. But it will make it slower.

Naturally a model in your docker will make it a bit fatter.<|||||>I think the path has changed slightly. I found the file in "src" folder under master.  https://github.com/huggingface/transformers/blob/master/src/transformers/configuration_distilbert.py <|||||>About downloading model files, in configuration_distilbert.py, I only found https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-uncased-distilled-squad-config.json file path. It just gives config. It is not a weights file / pickle file. Please suggest the path of files which I can download and make part of the local folder. Thanks.<|||||>Links to pre-trained models are available in the beginning of each `modeling_xxx.py` file, e.g. for [BERT](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L34-L56). Put this with a configuration in your folder and you can load them locally.

You could also use the `save_pretrained` method to automatically create a folder that can be used with `from_pretrained`.