Hi,
Please avoid using .data. Not sure why you use it here as w1.grad does not require gradient. If you want a copy, you need .clone().
These gradients are expected to be close but not equal I’m afraid.
Floating point operations are not exact and can differ based on their order. Here when you backward every time or if you sum the result and then do one backward, you will perform all operations in different orders.