Hi There,
For this purpose, I think that using Leaky ReLu on the loss and then back propagation will help you as it traces f(x) = x for x > 0 and f(x) = negative slope * x for x < 0. Here keeping negative slope between (0, 1) will help you.
Be sure to check out torch.nn.LeakyReLU.
All the best.