Good day! Thank you very much for reading this question.
I am working on private dataset in local storage and I want to mimic the program that loads dataset with load_dataset(). In order not to modify the training loop, I would like to convert my private dataset into the exact format the online dataset is stored; so that after loading the dataset, it will have exact same behavior, i.e. having a DatasetDict object with 3 splits (train, validation and test) with feature ‘translation’ which contains two key value pairs in each row with key name as language code and value as sentence). The behavior is shown as below.
Would you please help me with
(1) the folder structure, naming of the files, and the data format
(2) how to call load_dataset so that it will return a DatasetDict with same behavior as below.
raw_datasets = load_dataset("wmt17", "de-en")
print(raw_datasets)
'''
DatasetDict({
    train: Dataset({
        features: ['translation'],
        num_rows: 5906184
    })
    validation: Dataset({
        features: ['translation'],
        num_rows: 2999
    })
    test: Dataset({
        features: ['translation'],
        num_rows: 3004
    })
})
'''
print(raw_datasets["train"][0])
'''
{'translation': {'de': 'Wiederaufnahme der Sitzungsperiode',
  'en': 'Resumption of the session'}}
'''

Thank you!