I am trying to download the Hugging Face distilbert model, trying to save to S3. The model itself does not have a deploy method. So I am saving to S3, instantiating it and trying to deploy. May I know if this will work with Sagemaker. What I am doing wrong. Here are the steps:
model_name = ‘distilbert-base-uncased-distilled-squad’ model = DistilBertForQuestionAnswering.from_pretrained(model_name) tokenizer = DistilBertTokenizerFast.from_pretrained(model_name)
the below works and gives output context = “xxxx” question = “yyy?”
nlp = pipeline(‘question-answering’, model=model, tokenizer=tokenizer)
nlp({ ‘question’: ‘What organization is the IPCC a part of?’, ‘context’: context })
save the model to local folder model.save_pretrained(’./scripts/mymodel’)
zip the model file. with tarfile.open("./scripts/mymodel/model.tar.gz", “w:gz”) as tar: tar.add("./scripts/mymodel/pytorch_model.bin") tar.add("./scripts/mymodel/config.json")
upload the zipped file to S3 sagemaker.Session().upload_data(bucket=sagemaker_session_bucket, path=’./scripts/mymodel/model.tar.gz’, key_prefix=‘model’)
instantiating the saved model
bertmodel = PyTorchModel(entry_point=‘inference.py’, source_dir=‘scripts’, model_data=‘s3://’+sagemaker_session_bucket+’/model/model.tar.gz’, role=sagemaker.get_execution_role(), framework_version=‘1.5’, py_version=‘py3’)
the below does not work nlp = pipeline(‘question-answering’, model=bertmodel, tokenizer=tokenizer)
nlp({ ‘question’: ‘What organization is the IPCC a part of?’, ‘context’: context })
error recd - AttributeError: ‘PyTorchModel’ object has no attribute ‘config’
I am able to deploy the predictor predictor = bertmodel.deploy(initial_instance_count=1, instance_type=‘ml.m5.xlarge’)