title: Toxicity of the Commons: Curating Open-Source Pre-Training Data
summary: Open-source large language models are becoming increasingly available and
  popular among researchers and practitioners. While significant progress has
  been made on open-weight models, open training data is a practice yet to be
  adopted by the leading open-weight models creators. At the same time, there
  researchers are working to make language models safer. We propose a data
  curation pipeline to reduce harmful outputs by models trained on public domain
  data. There are unique challenges to working with public domain data, as these
  sources differ from web text in both form and content. Many sources are
  historical documents and are the result of Optical Character Recognition (OCR).
  Consequently, current state-of-the-art approaches to toxicity filtering are
  often infeasible or inappropriate for open data models. In this paper, we
  introduce a new fully open-source pipeline for open-data toxicity filtering.
  Our contributions are threefold. We create a custom training dataset,
  ToxicCommons, which is composed of texts which have been classified across five
  different dimensions (racial/origin-based, gender/sex-based, religious,
  ability-based discrimination, and violence). We use this dataset to train a
  custom classifier, Celadon, that can be used to detect toxic content in open
  data more efficiently at a larger scale. Finally, we describe the balanced
  approach to content filtration that optimizes safety filtering with respect to
  the filtered data available for training.
paper_authors._id: 6722e83247de1a565f91c768
paper_authors.name: Catherine Arnett
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-31T14:06:04.601Z
paper_authors._id: 6384d424ac61472e5e92d403
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6384d424ac61472e5e92d403/RfEyLD3FmEobLcpNE9pNu.png
paper_authors.fullname: Catherine Arnett
paper_authors.type: user
paper_authors.user: catherinearnett
paper_authors._id: 6722e83247de1a565f91c769
paper_authors.name: Eliot Jones
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-10-31T02:18:38.592Z
paper_authors._id: 66214936126ebc0deb501a87
paper_authors.avatarUrl: /avatars/e038e84f58c8577425df327a06b801d0.svg
paper_authors.fullname: Eliot Jones
paper_authors.type: user
paper_authors.user: eliotj
paper_authors._id: 6722e83247de1a565f91c76a
paper_authors.name: Ivan P. Yamshchikov
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-10-31T15:30:07.401Z
paper_authors._id: 6723a215f60c084a7dd016b8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/5z5Ap4AFQhCkMJz3oOXCU.png
paper_authors.fullname: Prof. Ivan Yamshchikov
paper_authors.type: user
paper_authors.user: ivan-the-bearable
paper_authors._id: 6722e83247de1a565f91c76b
paper_authors.name: Pierre-Carl Langlais
paper_title: Toxicity of the Commons: Curating Open-Source Pre-Training Data
paper_summary: Open-source large language models are becoming increasingly available and
  popular among researchers and practitioners. While significant progress has
  been made on open-weight models, open training data is a practice yet to be
  adopted by the leading open-weight models creators. At the same time, there
  researchers are working to make language models safer. We propose a data
  curation pipeline to reduce harmful outputs by models trained on public domain
  data. There are unique challenges to working with public domain data, as these
  sources differ from web text in both form and content. Many sources are
  historical documents and are the result of Optical Character Recognition (OCR).
  Consequently, current state-of-the-art approaches to toxicity filtering are
  often infeasible or inappropriate for open data models. In this paper, we
  introduce a new fully open-source pipeline for open-data toxicity filtering.
  Our contributions are threefold. We create a custom training dataset,
  ToxicCommons, which is composed of texts which have been classified across five
  different dimensions (racial/origin-based, gender/sex-based, religious,
  ability-based discrimination, and violence). We use this dataset to train a
  custom classifier, Celadon, that can be used to detect toxic content in open
  data more efficiently at a larger scale. Finally, we describe the balanced
  approach to content filtration that optimizes safety filtering with respect to
  the filtered data available for training.
paper_ai_summary: A new fully open-source pipeline for toxicity filtering is introduced to detect and reduce harmful outputs in language models trained on public domain data.
paper_ai_keywords: OCR toxicity filtering custom training dataset ToxicCommons custom classifier Celadon