Personally, I would expect that with reduction='mean' you would have to take the mean over the gradients you get from individual backpropagation.
If you use VGG, I’m assuming this is VGG without batch norm?
Best regards
Thomas