I wonder if torch.nn.utils.clip_grad_norm will clip gradient of loss+L2 penalty or just loss ?
Here is my code:
            optimizer = Adam(model.parameters(),t_lr,weight_decay=options['reg'])
                 ...
                 ...
            optimizer.zero_grad()
            output=model(input_var)
            loss=criterion(output,target_var)
            loss.backward()
            torch.nn.utils.clip_grad_norm(model.parameters(),options['clip_gradient_norm'])
            optimizer.step()