If the posted curves represent the losses, I would recommend to make sure your model is training and the loss decreases.
The augmented images might be “harder examples”, which might create a higher loss, but it seems that your model is not learning anything at all at the moment.