Solved by crowsonkb in post #2 
              
              
                The sigmoid function is a soft version of the step function, so you can do: 
diff = y_pred - y_true
diff2 = diff**2
mask = torch.sigmoid(diff)
losses = diff2 * (1 - mask) + diff2 * mask * 2
loss = losses.mean()

This penalizes (y_pred - y_true) > 0 approximately twice as much as the reverse. If we s…