Great work!
There are only two possible sets of keys to get from run_eval.py since
score_fn = calculate_bleu_score if "translation" in args.task else calculate_rouge
You shouldn’t hard code the possible tasks any more than that IMO.