The Problems with Proxies: Making Data Work Visible through Requester Practices
<|reference_start|>The Problems with Proxies: Making Data Work Visible through Requester Practices: Fairness in AI and ML systems is increasingly linked to the proper treatment and recognition of data workers involved in training dataset development. Yet, those who collect and annotate the data, and thus have the most intimate knowledge of its development, are often excluded from critical discussions. This exclusion prevents data annotators, who are domain experts, from contributing effectively to dataset contextualization. Our investigation into the hiring and engagement practices of 52 data work requesters on platforms like Amazon Mechanical Turk reveals a gap: requesters frequently hold naive or unchallenged notions of worker identities and capabilities and rely on ad-hoc qualification tasks that fail to respect the workers' expertise. These practices not only undermine the quality of data but also the ethical standards of AI development. To rectify these issues, we advocate for policy changes to enhance how data annotation tasks are designed and managed and to ensure data workers are treated with the respect they deserve.<|reference_end|>
@article{rothschild2024the,
    title={The Problems with Proxies: Making Data Work Visible through Requester
  Practices},
    author={Annabel Rothschild, Ding Wang, Niveditha Jayakumar Vilvanathan, Lauren
  Wilcox, Carl DiSalvo, and Betsy DiSalvo},
    journal={arXiv preprint arXiv:2408.11667},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.11667},
    primaryClass={cs.HC}
}