Pragmatic auditing: a pilot-driven approach for auditing Machine Learning systems
<|reference_start|>Pragmatic auditing: a pilot-driven approach for auditing Machine Learning systems: The growing adoption and deployment of Machine Learning (ML) systems came with its share of ethical incidents and societal concerns. It also unveiled the necessity to properly audit these systems in light of ethical principles. For such a novel type of algorithmic auditing to become standard practice, two main prerequisites need to be available: A lifecycle model that is tailored towards transparency and accountability, and a principled risk assessment procedure that allows the proper scoping of the audit. Aiming to make a pragmatic step towards a wider adoption of ML auditing, we present a respective procedure that extends the AI-HLEG guidelines published by the European Commission. Our audit procedure is based on an ML lifecycle model that explicitly focuses on documentation, accountability, and quality assurance; and serves as a common ground for alignment between the auditors and the audited organisation. We describe two pilots conducted on real-world use cases from two different organisations and discuss the shortcomings of ML algorithmic auditing as well as future directions thereof.<|reference_end|>
@article{benbouzid2024pragmatic,
    title={Pragmatic auditing: a pilot-driven approach for auditing Machine
  Learning systems},
    author={Djalel Benbouzid, Christiane Plociennik, Laura Lucaj, Mihai Maftei,
  Iris Merget, Aljoscha Burchardt, Marc P. Hauer, Abdeldjallil Naceri, Patrick
  van der Smagt},
    journal={arXiv preprint arXiv:2405.13191},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13191},
    primaryClass={cs.LG}
}