Thanks, I was hopeful as I was getting exact same network outputs when commenting out optimizer.step().
If I get different grads for simple loss function like cross entropy, is there any way to solve that problem?