Hi -
I am trying to use the sagemaker Huggingface estimator to fine-tune a model for summarization using the run_summarization.py 4 entry_point. I have created a Sagemaker Studio notebook based on the code from the summarization example notebook 3 provided in the Sagemaker examples.
I would like to use my own data to train the model and so have added the following code to make train and validation datasets I have uploaded to s3 storage available to the estimator:
Define s3 locations:
training_input_path = "s3://sagemaker-eu-central-1-88888888888/train_20210607.csv"
test_input_path = "s3://sagemaker-eu-central-1-88888888888/val_20210607.csv"

Define file locations in hyperparameters:
hyperparameters={
    ...,
    'train_file': '/opt/ml/input/data/train_20210607.csv',
    'validation_file': '/opt/ml/input/data/val_20210607.csv',
    'text_column': 'document',
    'summary_column': 'summary',
    ...
}

Ensure data is loaded when starting training job:
huggingface_estimator.fit({'train': training_input_path, 'test': test_input_path})
It looks from the log that the data has been loaded into the expected directory:
SM_HP_VALIDATION_FILE=/opt/ml/input/data/val_20210607.csv
SM_HP_TRAIN_FILE=/opt/ml/input/data/train_20210607.csv

However, after the run_summarization.py script is run, I get the following error:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/input/data/train_20210607.csv'
Apologies if I am missing something obvious, but would be great if anyone could let me know how I should be referencing my data so that it can be used by the run_summarization.py script?
Thank you!
Ben