Hi,
I have reproduced ResNeXt with pytorch on CIFAR 86 and results are always slightly below the original torch implementation 66 .
I have been in contact with the author of the paper and he thinks it might be due to differences with the original ResNet implementation, such as applying weight decay to the biases.
I have gone through the pytorch implementation of the weight decay and I have now some doubts I would like to ask:

Is weight decay affecting the batchnorm biases by default?
Taking a look into the current implementation in sgd.py 127, could anyone explain what is line 85 doing?

d_p.add_(weight_decay, p.data)
Thank you in advance!