Based on the posted images it looks as if the very first loss value is already different before the parameter update was performed. Could you compare the initial parameters as well as the input and loss value before comparing the parameter updates?