Thanks for replying. I rotate and filp my data to generate other 7 kinds of data. After I search on albumentations, I see that it provides many kinds of augmentations I don’t familiar with. Is there any method you recommend to apply to my MR dataset?
Also, I recently try to degraded my HR images in frequency domain by cutting off three quarters of high frequency region and get better results than before(guassian blur+bicubic down-sample). I think maybe features of LR images generated this way are more similar to real LR images.
I check if my model overfit my training data by calculating average  psnr on my validation data every half of epoch. It saves the model when psnr is higher than the highest psnr before it. And I will lower the learning rate if the average psnr doesn’t get higher for maybe 3 or 5 epoch. Then it early stops training when val psnr doesn’t get higher for maybe 10 epoch. Finally I get the model with best psnr score on validation data. This sounds reasonable to me but I wonder if this is a proper way to manage learning rate and early stop to prevent overfitting.