Hi,
The two W0 there are actually a single Tensor that is used twice.
So just like any other case where you re-use a Tensor, the mathematical result is to accumulate the contributions for each use of it.
If you use something like torchviz (https://github.com/szagoruyko/pytorchviz/ 45) to print the computational graph, you will see that both evaluations point to a single circle for each weights.