All the “code a simple neural net” articles that I have seen to date just use output-targets as the initial gradients and don’t explain the correspondance to mse loss.
Averaging over the batch size avoids having to adjust the learning rate to the batch size. Otherwise big batch -> big sum(loss) -> big update, and big infrequent updates aren’t always a great idea.