Thanks for the reply. I’ll look into using this mode of dataset generation. For now, I’ve taken the easy way out and parsed my data into a form that the run_translation.py script in the examples understands. I’ve saved my data in the jsonl format.
For others hoping to use the same method, look into df.to_dict(orient=‘records’) and json dumping.