Weird. My implementation of the Inverse square root linear unit works fine.
class ISRLU(torch.autograd.Function):
    @staticmethod
    def forward(ctx, tensor, alpha=1):
        negatives = torch.min(tensor, torch.Tensor([0]))
        nisr = torch.rsqrt(1. + alpha * (negatives ** 2))
        return tensor * nisr

Just guessing: Are you using pytorch 0.3.0 ?