@fmassa, so because of PyTorch’s autograd functionality, we do not need to worry about L1 regularization during the backward pass, i.e. applying the derivative of the L1 regularization term to the gradient of the output? That will be handled by the autograd variables?