Usually this is taken care of.
What happens internally is that the backward of autograd Functions returns None for inputs that the result cannot be differentiated by.
To see it in action:
x = Variable(torch.randn(10), requires_grad=True)
y,l = x.max(0)
l.backward(retain_graph=True)
print(x.grad) # zero because dl/dx does not make sense
y.backward()
print(x.grad) # one at the maximum

Best regards
Thomas