I’m using bert-large-cnn for a summarization task. I have been truncating the input text in order to avoid exceeding the maximum sequence length:
tokenizer = BartTokenizer.from_pretrained("facebook/bart-large-cnn")
model = BartForConditionalGeneration.from_pretrained(
    "facebook/bart-large-cnn"
)

inputs = tokenizer(
    input_text, return_tensors="pt", max_length=1024, truncation=True
)

outputs = model.generate(
    inputs["input_ids"],
    max_length=300,
    min_length=100,
    length_penalty=2.0,
    num_beams=4,
    early_stopping=True,
)

summary = " ".join(
    tokenizer.decode(
        g, skip_special_tokens=True, clean_up_tokenization_spaces=False
    )
    for g in outputs
)

How can I replicate this in SageMaker? I don’t see a way to pass configuration values to the tokenizer when calling predict on an instance of HuggingFaceModel? Here is my code so far:
env = {
    "HF_MODEL_ID": "facebook/bart-large-cnn",
    "HF_TASK": "summarization",
}

huggingface_model = HuggingFaceModel(
    env=env,
    role=role,
    transformers_version="4.6",
    pytorch_version="1.7",
    py_version="py36",
)

data = {
    "inputs": input_text,
    "parameters": {
        "max_length": 300,
        "min_length": 100,
        "length_penalty": 2.0,
        "num_beams": 4,
    }
}

result = predictor.predict(data)

Thank you!