I would try lowering it more like 1e-3, 1e-4 just to make sure. Also what is the data you’re training on? Have you tried running it through a very simple dataset like mnist to make sure it’s in this part of the code