AIMS.au: A Dataset for the Analysis of Modern Slavery Countermeasures in Corporate Statements
Adriana Eufrosina Bora Pierre-Luc St-Charles Mirko Bronzi Arsene Fansi Tchango Bruno Rousseau Kerrie Mengersen
Despite over a decade of legislative efforts to address modern slavery in the supply chains of large corporations, the effectiveness of government oversight remains hampered by the challenge of scrutinizing thousands of statements annually. While Large Language Models (LLMs) can be considered a well established solution for the automatic analysis and summarization of documents, recognizing concrete modern slavery countermeasures taken by companies and differentiating those from vague claims remains a challenging task. To help evaluate and fine-tune LLMs for the assessment of corporate statements, we introduce a dataset composed of 5,731 modern slavery statements taken from the Australian Modern Slavery Register and annotated at the sentence level. This paper details the construction steps for the dataset that include the careful design of annotation specifications, the selection and preprocessing of statements, and the creation of high-quality annotation subsets for effective model evaluations. To demonstrate our dataset's utility, we propose a machine learning methodology for the detection of sentences relevant to mandatory reporting requirements set by the Australian Modern Slavery Act. We then follow this methodology to benchmark modern language models under zero-shot and supervised learning settings.
natural language processing, modern slavery, corporate statements, benchmark, text extraction, large language models
We provide a new dataset of modern slavery statements annotated at the sentence level to train and evaluate language models for relevant text detection and extraction.