I am training a multiclass image segmentation model using U-Net and R2U-Net architectures from Image_Segmentation/network.py at master · LeeJunHyun/Image_Segmentation · GitHub
Using Unet I get very good convergence of Validation and Training Cross Entropy Loss

Whereas R2-UNet shows an erroneous validation loss whereas training loss converges nicely.

I am training both the models using the same batch size, Learning Rate, epochs. Is this a case of overfitting? How can I overcome this, will regularization/LR scheduling/ SGD/ Gradient Clipping help in convergence.
On a more general note how should I debug such results, is it just by trying out different things and finding out which gives the best result? Would like to know if there is mathematically intuitive approach to debugging.
Thank you in advance!