I created an example with a inference.py included in a model.tar.gz. You can find the whole repository here: GitHub - philschmid/sample-custom-inference-sagemaker-huggingface 3
You can find the inference.py here: sample-custom-inference-sagemaker-huggingface/inference.py at master · philschmid/sample-custom-inference-sagemaker-huggingface · GitHub 1
The structure of the archive is
code/
    inference.py
pytorch_model.bin
config.json
tokenizer.json
....