From the paper 1 (page 6):

During training, their loss gets added to the total loss of the network with a discount weight (the losses of the auxiliary classifiers were weighted by 0.3).
~ https://github.com/BVLC/caffe/pull/1367 4

Can we do similar to that in PyTorch?