Hi dear all
I am using pytorch to generate adversarial samples.
When back propagating the error to input pattern elements in .eval() mode, I observe by changing the batch size from 1 to larger, the gradient values varies slightly.
Does anyone have any idea that this is common or I made a mistake.
two typical gradients are:
Batchsize>1
-0.012502569
-0.06345608
0.024056617
0.00014444088
0.030317063
-0.049938474
0.0059496835
Batchsize=1
-0.012502571
-0.06345608
0.024056613
0.00014443748
0.030317055
-0.049938496
0.005949676
For reproducibility, I have used the following commands:
torch.backends.cudnn.deterministic = True
torch.manual_seed(1)
np.random.seed(1)