not that dramatic…I update all my conda package to latest, and introduce some data augmentation and it reach 68.3%, then I try ‘He init’ it gives me 70.2% as top-1 accuracy, which is quite close to 22-28% top-1 error reported by almost all github repos…
the benefit of "he init’’ to me is like the model converges more stable, because my previous training the model is quite unstable and even with warm up the accuracy gets stuck…