You cannot mathematically differentiate that.
The typical “pretend” solution is to have the backward act as if you passed the probability vector, e.g.
pdist = torch.randn(5,4).softmax(1).requires_grad_()
out = torch.zeros_like(pdist)
out.scatter_(1, pdist.multinomial(2), torch.ones(()))
out_with_soft_backward = pdist + (out - pdist).detach()

Best regards
Thomas