No theoretical proof, just empirical observation. Here’s an illustration with the toy problem above, as I increase the dimensionality. I don’t think it is just a numerical precision issue, since the discrepancies are >1e-6 as you can see below.
dim=1,  torch.max(torch.abs(gradient1 - gradient2)) = 1.49011611938e-08
dim=10,  torch.max(torch.abs(gradient1 - gradient2)) = 1.49011611938e-08
dim=50,  torch.max(torch.abs(gradient1 - gradient2)) = 4.76837158203e-07
dim=100,  torch.max(torch.abs(gradient1 - gradient2)) = 9.53674316406e-07
dim=200,  torch.max(torch.abs(gradient1 - gradient2)) = 1.90734863281e-06
dim=500,  torch.max(torch.abs(gradient1 - gradient2)) = 3.81469726562e-06
dim=1000,  torch.max(torch.abs(gradient1 - gradient2)) = 1.90734863281e-06

Unfortunately it does matter significantly. In my experiments on real datasets, the accuracies are off by 1-2% after just a few thousand rounds of training.