I assume you have trained longer than 4 epochs, because this loss curve does not look like overfitting to me. Was the curve identical for both ResNets? If yes, maybe the problem is somewhere else.
Some good examples of curves here: How to use Learning Curves to Diagnose Machine Learning Model Performance
If you are 100% certain its not a different issue I would attempt more augmentation. It could also be an issue how your dataset train/val/test splits are selected. Maybe the test split is unrepresentative compared to the train/val splits.