The gradient norm clipping wouldn’t work, since multiplying a +/-Inf gradient with the scale factor won’t change the gradient (used here).
While clipping the gradient with values would work for +/-Inf values, unfortunately the +/-Inf loss might create NaN gradients as seen here:
model = nn.Linear(1, 1)
x = torch.randn(1, 1)
out = model(x)

loss = torch.log(out - out)
print(loss)
> tensor([[-inf]], grad_fn=<LogBackward>)

loss.backward()
print(model.weight.grad)
> tensor([[nan]])

I think the right approach would be to avoid creating the invalid loss values and e.g. add a small eps value to the loss calculation.