The rules are that you should use the training set.
The next question then is what to do when the test set a larger depth somewhere. I’d probably just clamp it down to 1. In a good training set, I would expect the training set to max out the depth reported by the camera somewhere.
Best regards
Thomas