Interesting, naively I would have expected the V100 to be potentially “worse” in convergence @fp16 because of the possibility of reduced precision reductions involving tensor cores.
Can you provide some more details about the model architecture (it sounds like if mAP is the metric then it would be a conv-heavy object detection model)? In this case I would not expect the use of mixed-precision to be the reason for slower convergence, but it might be useful to double check what happens in single precision if you can spare the extra compute time.
As for comparing results such as model weights and outputs, you might consider taking a look at : Reproducibility — PyTorch 1.10.0 documentation 1 to see if reproducibility can be improved, but as you correctly note there won’t be any guarantees across cards with different hardware architectures, especially in half-precision layers.