The scaling of RandomResizedCrop could be too aggressive for your model.
If I’m not mistaken, it worked well using the default setup in Inception models, but you might need to reduce the scaling for your use case.