I’m a PyTorch beginner, and I’m curious how to mimic this 7 functionality in PyTorch. My current approach defines a function and registers it with register_backward_hook:
def normalize_grad(module, grad_input, grad_output):
    # both grad input and grad output are of size one, but we'll do it in general anyway
    in_tup = []
    for gi, go in zip(grad_input, grad_output):
        gin = gi.div(torch.norm(gi, p=1) + 1e-8).mul(module.strength).add(go)
        in_tup.append(gin)
    return tuple(in_tup)

I’m not sure if this is correct, because it is very dramatically changing my results.