Hi All,
I trained a dataset (grey-scale ultrasound images. with 15 classes) on vgg-16 (using batch norm version) and resnet-34.
vgg-16 gives me a validation accuracy of 92% where  as I can only hit 83% with resnet-34 .
I handled overfitting in both architectures with dropout in FC layer and regularization in optimizer.
I don’t know why there would be minimal overfitting with vgg and not resnet . The resnet model train loss is 0.02 vs valid loss 0.67. Moreover resnet model doesn’t seem to improve beyond this loss range. I have tried hyper-parameter tuning on weight decay, learning rate, momentum, dropout.
The purpose of this exercise to improve our predictions on the dataset. Hence I started trying resnet-34 after vgg-16.
I would love some suggestions on

How to improve resnet-34 performance and address its overfitting.
Any other architecture suggestions that would work for this type of dataset.