I have 20,580 images about dogs downloaded from http://vision.stanford.edu/aditya86/ImageNetDogs/ 21
In the dataset, I have 120 classes, and each class has images between 120 and 180.And I use 16,418 (about 80% of the dataset) as the train dataset.
During my training, the train accuracy could be 0.96, but the val accuracy is only 0.7(after 50 epochs). It is obviously that overfitting happened.
So I tried seveal ways to improve the problem:


Reduce the size of my network.I switched from using a pretrained(Imagenet) Resnet50 to Resnet18, but it improved a little.


Data augmentation.For the train dataset, I used random horizontal flip,random brightness,random shift and random ratation. In the end, I resized the image to 384384,but it does not help too much with overfitting(Because the size of images are different,some are 180120,some are 500*400, and so on).


Add weight decay.I tried 1e-5,5e-4,1e-4,1e-3 weight_decay,and 1e-5 and 1e-4 could improve a little.The train accuracy is 0.85,and the val accuracy is 0.65(after 7 epochs).


I am confused about how to prevent overfitting. I even doubt if the dataset is suitable for classfication.
Hope your advice.
Thanks.