Hello again,
After some work, i’ve manage to get the model over 35% top1 accuracy on test and validation with overfitting showing up at past 18% of dataset. I didn’t want to discount “making network smaller” before i found the next stepping stone, as this is a generic intuition of what works in neural networks.
What worked for me was an autoencoder, tranied on the same training data and put at the begining of final network while being set to frozen. So i guess i took a chapter from transfer learning.
Coming back to original question, i guess i’m now looking for other building blocks that people might have employed in their succeses breaking barriers in their nets.
Regards,
AL