Ok I got it (I didn’t quite understood how torch.autograd.grad worked).
Basically, for every layer that has its requires_grad = True, I can perform:
torch.autograd.grad(reg, layer)

ant it returns the gradient of the regularization on the layer. Thanks a lot.
Just an auxiliary question though: do we agree that this does not update the .grad value of the layer and that I can still call .backward() like normal afterwards?