Solved by albanD in post #4 
              
              
                For Adam, if any “momentum-like” term is non-zero, then you will update the weights even with a gradient of 0. So this is expected.