Hi,
Your problem is a bit different from the classical neural net as you have one weight per sample that you just try to hardcore over-fit to it.
In your current code, even though you have a single loss which is the mean, since each element before the mean is computed completely independently from each (different input, different parameter, etc), then the gradients of the mean will be the same as if you called backward on each element of the sum one by one (dividing by the number of elements because of the mean).
So you do any of these (and mix) because the different part of your model are independent 
Note that in general, it is better to sum the losses and do a single backward (speed wise).