The following codes give the same results. Does it mean torch.clamp is totally the same as torch.nn.functional.hardtanh both in training and test phase?
x1 = torch.randn(10)
x2 = x1.detach().clone()

x1.requires_grad_()
x2.requires_grad_()

y1 = torch.nn.functional.hardtanh(x1)
y2 = torch.clamp(x2, -1.0, 1.0)

l1 = y1.sum()
l2 = y2.sum()

l1.backward()
l2.backward()

assert (x1.grad - x2.grad).abs().sum() == .0
assert l1 == l2