title: Data Contamination Report from the 2024 CONDA Shared Task
summary: The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant
  aspects of data contamination in natural language processing, where data
  contamination is understood as situations where evaluation data is included in
  pre-training corpora used to train large scale models, compromising evaluation
  results. The workshop fostered a shared task to collect evidence on data
  contamination in current available datasets and models. The goal of the shared
  task and associated database is to assist the community in understanding the
  extent of the problem and to assist researchers in avoiding reporting
  evaluation results on known contaminated resources. The shared task provides a
  structured, centralized public database for the collection of contamination
  evidence, open to contributions from the community via GitHub pool requests.
  This first compilation paper is based on 566 reported entries over 91
  contaminated sources from a total of 23 contributors. The details of the
  individual contamination events are available in the platform. The platform
  continues to be online, open to contributions from the community.
paper_authors._id: 66aaef50be4c62fe6ab1ed3c
paper_authors.name: Oscar Sainz
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:47:51.431Z
paper_authors._id: 6253d0cc27a8414d3bbea683
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6253d0cc27a8414d3bbea683/fRck2wXEg-PS11ph9u8ZI.jpeg
paper_authors.fullname: Oscar Sainz
paper_authors.type: user
paper_authors.user: OSainz
paper_authors._id: 66aaef50be4c62fe6ab1ed3d
paper_authors.name: Iker García-Ferrero
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:47:57.708Z
paper_authors._id: 614a1ebb8f82f1df64d55126
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1632247447995-noauth.jpeg
paper_authors.fullname: Iker García-Ferrero
paper_authors.type: user
paper_authors.user: Iker
paper_authors._id: 66aaef50be4c62fe6ab1ed3e
paper_authors.name: Alon Jacovi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:48:05.844Z
paper_authors._id: 6596c90dfe0f9826a63654d0
paper_authors.avatarUrl: /avatars/e616fbdff1a345ad22081f5cd019329a.svg
paper_authors.fullname: Alon Jacovi
paper_authors.type: user
paper_authors.user: alonjacovi
paper_authors._id: 66aaef50be4c62fe6ab1ed3f
paper_authors.name: Jon Ander Campos
paper_authors._id: 66aaef50be4c62fe6ab1ed40
paper_authors.name: Yanai Elazar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:48:18.420Z
paper_authors._id: 623ca115a795593324c4353f
paper_authors.avatarUrl: /avatars/bf11fe728df2786d52ed4d2de12b48d3.svg
paper_authors.fullname: Yanai Elazar
paper_authors.type: user
paper_authors.user: yanaiela
paper_authors._id: 66aaef50be4c62fe6ab1ed41
paper_authors.name: Eneko Agirre
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:48:24.221Z
paper_authors._id: 6243084d5b75d53952f28e83
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6243084d5b75d53952f28e83/_wpjRn8qhPvijjfh-W-_M.png
paper_authors.fullname: Eneko Agirre
paper_authors.type: user
paper_authors.user: eagirre
paper_authors._id: 66aaef50be4c62fe6ab1ed42
paper_authors.name: Yoav Goldberg
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:39:38.610Z
paper_authors._id: 676d5323dd95830fd94d675f
paper_authors.avatarUrl: /avatars/a8622e2a189a13c7b5a99177c3a87c37.svg
paper_authors.fullname: Yoav Goldberg
paper_authors.type: user
paper_authors.user: yoavgoldberg
paper_authors._id: 66aaef50be4c62fe6ab1ed43
paper_authors.name: Wei-Lin Chen
paper_authors._id: 66aaef50be4c62fe6ab1ed44
paper_authors.name: Jenny Chim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:48:58.831Z
paper_authors._id: 5fc8e6e9628d967e353416b3
paper_authors.avatarUrl: /avatars/d913dff9380e711aa4ecf368e3070f88.svg
paper_authors.fullname: Jenny Chim
paper_authors.type: user
paper_authors.user: j-chim
paper_authors._id: 66aaef50be4c62fe6ab1ed45
paper_authors.name: Leshem Choshen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:49:04.733Z
paper_authors._id: 61bf40824b4300d0fb0acf59
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644224872623-61bf40824b4300d0fb0acf59.jpeg
paper_authors.fullname: Leshem Choshen
paper_authors.type: user
paper_authors.user: borgr
paper_authors._id: 66aaef50be4c62fe6ab1ed46
paper_authors.name: Luca D'Amico-Wong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:49:12.178Z
paper_authors._id: 6669cb5c00cf2a0d01b3499f
paper_authors.avatarUrl: /avatars/a20d79c2599bc49c96361618b3e5419b.svg
paper_authors.fullname: Luca D'Amico-Wong
paper_authors.type: user
paper_authors.user: ldamicowong
paper_authors._id: 66aaef50be4c62fe6ab1ed47
paper_authors.name: Melissa Dell
paper_authors._id: 66aaef50be4c62fe6ab1ed48
paper_authors.name: Run-Ze Fan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-01T07:00:30.573Z
paper_authors._id: 616bfc2b40e2f69baa1c7add
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/616bfc2b40e2f69baa1c7add/Os7_qgMei-2lRVelrOG7B.jpeg
paper_authors.fullname: Run-Ze Fan
paper_authors.type: user
paper_authors.user: Vfrz
paper_authors._id: 66aaef50be4c62fe6ab1ed49
paper_authors.name: Shahriar Golchin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:49:25.711Z
paper_authors._id: 6263102f3662a749162d67e6
paper_authors.avatarUrl: /avatars/51c36acbc4d3b76632273ccad5ca8d5c.svg
paper_authors.fullname: Shahriar Golchin
paper_authors.type: user
paper_authors.user: shahriargolchin
paper_authors._id: 66aaef50be4c62fe6ab1ed4a
paper_authors.name: Yucheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:50:18.424Z
paper_authors._id: 63d00710645dd8d34ea9bcc6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63d00710645dd8d34ea9bcc6/E6YsIsXH57OACL-NZ52fB.jpeg
paper_authors.fullname: Yucheng
paper_authors.type: user
paper_authors.user: liyucheng
paper_authors._id: 66aaef50be4c62fe6ab1ed4b
paper_authors.name: Pengfei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:50:24.962Z
paper_authors._id: 6144a0c4ff1146bbd84d9865
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
paper_authors.fullname: Pengfei Liu
paper_authors.type: user
paper_authors.user: Pengfei
paper_authors._id: 66aaef50be4c62fe6ab1ed4c
paper_authors.name: Bhavish Pahwa
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:50:33.243Z
paper_authors._id: 623f2f5828672458f74879b3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/623f2f5828672458f74879b3/E8sWXNEBu1Th1G76XOIqb.jpeg
paper_authors.fullname: Bhavish Pahwa
paper_authors.type: user
paper_authors.user: bpHigh
paper_authors._id: 66aaef50be4c62fe6ab1ed4d
paper_authors.name: Ameya Prabhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:51:17.078Z
paper_authors._id: 6464a0d41683d3c81f51924a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6464a0d41683d3c81f51924a/s7yYVwfUB4WOhVFJS6A6T.jpeg
paper_authors.fullname: Ameya Prabhu
paper_authors.type: user
paper_authors.user: AmeyaPrabhu
paper_authors._id: 66aaef50be4c62fe6ab1ed4e
paper_authors.name: Suryansh Sharma
paper_authors._id: 66aaef50be4c62fe6ab1ed4f
paper_authors.name: Emily Silcock
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:52:01.563Z
paper_authors._id: 61654589b5ec555e8e9c203a
paper_authors.avatarUrl: /avatars/654bdc8c0585ace01aba48633b745250.svg
paper_authors.fullname: Emily Silcock
paper_authors.type: user
paper_authors.user: emilys
paper_authors._id: 66aaef50be4c62fe6ab1ed50
paper_authors.name: Kateryna Solonko
paper_authors._id: 66aaef50be4c62fe6ab1ed51
paper_authors.name: David Stap
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:52:16.267Z
paper_authors._id: 6359028d6ca91ddc7401b757
paper_authors.avatarUrl: /avatars/c45c7c3e8c25eb18cd41afce1d5e9d54.svg
paper_authors.fullname: David Stap
paper_authors.type: user
paper_authors.user: davidstap
paper_authors._id: 66aaef50be4c62fe6ab1ed52
paper_authors.name: Mihai Surdeanu
paper_authors._id: 66aaef50be4c62fe6ab1ed53
paper_authors.name: Yu-Min Tseng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:52:31.038Z
paper_authors._id: 65e0538c527a7da472310059
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65e0538c527a7da472310059/-w7vfNyHOpym7b57WWhhT.jpeg
paper_authors.fullname: Yu-Min Tseng
paper_authors.type: user
paper_authors.user: ymtseng
paper_authors._id: 66aaef50be4c62fe6ab1ed54
paper_authors.name: Vishaal Udandarao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:52:37.230Z
paper_authors._id: 6304da46ce6b12280b1bd575
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6304da46ce6b12280b1bd575/V96ocKW4HOoysAGxuAH1X.jpeg
paper_authors.fullname: Vishaal Udandarao
paper_authors.type: user
paper_authors.user: vishaal27
paper_authors._id: 66aaef50be4c62fe6ab1ed55
paper_authors.name: Zengzhi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:52:43.935Z
paper_authors._id: 62cbeb2d72dfd24b86bdf977
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62cbeb2d72dfd24b86bdf977/UcGYYSBNrCvPM5K9v-sro.png
paper_authors.fullname: Zengzhi Wang
paper_authors.type: user
paper_authors.user: SinclairWang
paper_authors._id: 66aaef50be4c62fe6ab1ed56
paper_authors.name: Ruijie Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:52:50.631Z
paper_authors._id: 6537cdda25f780fed2375a7b
paper_authors.avatarUrl: /avatars/6309fbcbb97cd63957ed7429bbe311d0.svg
paper_authors.fullname: Ruijie Xu
paper_authors.type: user
paper_authors.user: GardenStrayer
paper_authors._id: 66aaef50be4c62fe6ab1ed57
paper_authors.name: Jinglin Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:53:02.497Z
paper_authors._id: 6457edb172b60ae7a37ead0b
paper_authors.avatarUrl: /avatars/85c6f38d27cbb885ac8d151b26f39541.svg
paper_authors.fullname: yangjinglin
paper_authors.type: user
paper_authors.user: zhaoyuxin924
paper_title: Data Contamination Report from the 2024 CONDA Shared Task
paper_summary: The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant
  aspects of data contamination in natural language processing, where data
  contamination is understood as situations where evaluation data is included in
  pre-training corpora used to train large scale models, compromising evaluation
  results. The workshop fostered a shared task to collect evidence on data
  contamination in current available datasets and models. The goal of the shared
  task and associated database is to assist the community in understanding the
  extent of the problem and to assist researchers in avoiding reporting
  evaluation results on known contaminated resources. The shared task provides a
  structured, centralized public database for the collection of contamination
  evidence, open to contributions from the community via GitHub pool requests.
  This first compilation paper is based on 566 reported entries over 91
  contaminated sources from a total of 23 contributors. The details of the
  individual contamination events are available in the platform. The platform
  continues to be online, open to contributions from the community.
paper_ai_summary: A shared task was organized to document and analyze data contamination in NLP datasets and models, aiming to prevent reporting on contaminated resources.
paper_ai_keywords: natural language processing data contamination pre-training corpora evaluation results shared task contamination evidence database GitHub pool requests