Investigating the Impact of Randomness on Reproducibility in Computer Vision: A Study on Applications in Civil Engineering and Medicine
<|reference_start|>Investigating the Impact of Randomness on Reproducibility in Computer Vision: A Study on Applications in Civil Engineering and Medicine: Reproducibility is essential for scientific research. However, in computer vision, achieving consistent results is challenging due to various factors. One influential, yet often unrecognized, factor is CUDA-induced randomness. Despite CUDA's advantages for accelerating algorithm execution on GPUs, if not controlled, its behavior across multiple executions remains non-deterministic. While reproducibility issues in ML being researched, the implications of CUDA-induced randomness in application are yet to be understood. Our investigation focuses on this randomness across one standard benchmark dataset and two real-world datasets in an isolated environment. Our results show that CUDA-induced randomness can account for differences up to 4.77% in performance scores. We find that managing this variability for reproducibility may entail increased runtime or reduce performance, but that disadvantages are not as significant as reported in previous studies.<|reference_end|>
@article{eryılmaz2024investigating,
    title={Investigating the Impact of Randomness on Reproducibility in Computer
  Vision: A Study on Applications in Civil Engineering and Medicine},
    author={Bahad{i}r Ery{i}lmaz, Osman Alperen Korac{s}, J"org Schl"otterer,
  Christin Seifert},
    journal={arXiv preprint arXiv:2410.02806},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.02806},
    primaryClass={cs.CV cs.AI}
}