Hello Alban!



 albanD:

But Adam has to handle all the 6.8M parameters so it’s not surprising to me to see it being so high in runtime.


Just for my education:
Would I be correct in deducing that in this situation a single SGD
optimizer step would run much faster than the Adam step?
(I’m leaving aside the question of whether Adam is a better
optimizer that would train the network faster.)
Thanks.
K. Frank