Solved by marcman411 in post #2 
              
              
                Actually it seems the answer is in the code I linked to: 
For a 2-norm: 
        for p in model.parameters():
            param_norm = p.grad.data.norm(2)
            total_norm += param_norm.item() ** 2
        total_norm = total_norm ** (1. / 2)