I think the easy way is to approach this as [binary] classification problem, with added probability calibration 1 after training.
If you want to also utilize the magnitude of change of doctor scores, perhaps KLDivLoss could be used (map value to “soft” targets like (0.9, 0.1) by some ad-hoc formula, calibrating doctors’ scores).