Hello,
When I try to backpropagate on a tensor full of -inf and I have a torch.logsumexp , the gradients of that becomes nan. Like the code below:
>>> a = torch.nn.Parameter(torch.tensor([-float("inf"), -float("inf"), -float("inf")]))
>>> b = 2 + a
>>> torch.logsumexp(a, dim=0)
tensor(-inf, grad_fn=<LogsumexpBackward>)
>>> torch.logsumexp(a, dim=0).backward()
>>> a.grad
tensor([nan, nan, nan])

But even if one of the elements in the tensor is non -inf the gradients are propagated properly:
>>> a = torch.nn.Parameter(torch.tensor([1.0, -float("inf"), -float("inf")]))
>>> b = 2 + a
>>> torch.logsumexp(a, dim=0)
tensor(1., grad_fn=<LogsumexpBackward>)
>>> torch.logsumexp(a, dim=0).backward()
>>> a.grad
tensor([1., 0., 0.])


How do I ensure that even in case of all -inf my gradients should be zero?