If you do want to automate a little more how you generate synthetic data sets, you can check out this video here on data set preparation with LLMs. Tip number six is always use a validation data set. If you don't have one, you can just split off 10 to 20% of your training data set. You want to be checking your training loss as you progress along the process. Make sure it's not too bumpy and your learning rate is not too high or your batch size or virtual batch size is too small.