It is safer to detach the grad:
for p in model.parameters():
    param_norm = p.grad.detach().data.norm(2)
    total_norm += param_norm.item() ** 2
total_norm = total_norm ** 0.5

Another tensor-style way is:
parameters = [p for p in model.parameters() if p.grad is not None and p.requires_grad]
if len(parameters) == 0:
    total_norm = 0.0
else:
    device = parameters[0].grad.device
    total_norm = torch.norm(torch.stack([torch.norm(p.grad.detach(), norm_type).to(device) for p in parameters]), 2.0).item()