Based on your description it seems you want to use a step function, which would have a zero gradient everywhere (except exactly as x==value), so it won’t be useful for your loss and gradient calculation.
If this would work, you could directly optimize the model predictions without e.g. using sigmoid or softmax.