Hello @TomKc,
You need to create a model.tar.gz including your tokenizer and model files. Instructions can be found here: Deploy models to Amazon SageMaker 1 or below.
{repository} must include those files.
cd {repository}
tar zcvf model.tar.gz *
aws s3 cp model.tar.gz <s3://{my-s3-path}>

and then you can use the env parameter to provide your task.
huggingface_model = HuggingFaceModel(
   model_data="s3://{my-s3-path}/model.tar.gz",
   role=role,
   transformers_version="4.12",
   pytorch_version="1.9",
   py_version='py38', 
   env = {"HF_TASK":"text-classification"}
)