Data Contamination Report from the 2024 CONDA Shared Task
The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant
aspects of data contamination in natural language processing, where data
contamination is understood as situations where evaluation data is included in
pre-training corpora used to train large scale models, compromising evaluation
results. The workshop fostered a shared task to collect evidence on data
contamination in current available datasets and models. The goal of the shared
task and associated database is to assist the community in understanding the
extent of the problem and to assist researchers in avoiding reporting
evaluation results on known contaminated resources. The shared task provides a
structured, centralized public database for the collection of contamination
evidence, open to contributions from the community via GitHub pool requests.
This first compilation paper is based on 566 reported entries over 91
contaminated sources from a total of 23 contributors. The details of the
individual contamination events are available in the platform. The platform
continues to be online, open to contributions from the community.
66aaef50be4c62fe6ab1ed3c
Oscar Sainz
admin_assigned
2024-08-02T15:47:51.431Z
6253d0cc27a8414d3bbea683
https://cdn-avatars.huggingface.co/v1/production/uploads/6253d0cc27a8414d3bbea683/fRck2wXEg-PS11ph9u8ZI.jpeg
Oscar Sainz
user
OSainz
66aaef50be4c62fe6ab1ed3d
Iker García-Ferrero
admin_assigned
2024-08-02T15:47:57.708Z
614a1ebb8f82f1df64d55126
https://cdn-avatars.huggingface.co/v1/production/uploads/1632247447995-noauth.jpeg
Iker García-Ferrero
user
Iker
66aaef50be4c62fe6ab1ed3e
Alon Jacovi
admin_assigned
2024-08-02T15:48:05.844Z
6596c90dfe0f9826a63654d0
/avatars/e616fbdff1a345ad22081f5cd019329a.svg
Alon Jacovi
user
alonjacovi
66aaef50be4c62fe6ab1ed3f
Jon Ander Campos
66aaef50be4c62fe6ab1ed40
Yanai Elazar
admin_assigned
2024-08-02T15:48:18.420Z
623ca115a795593324c4353f
/avatars/bf11fe728df2786d52ed4d2de12b48d3.svg
Yanai Elazar
user
yanaiela
66aaef50be4c62fe6ab1ed41
Eneko Agirre
admin_assigned
2024-08-02T15:48:24.221Z
6243084d5b75d53952f28e83
https://cdn-avatars.huggingface.co/v1/production/uploads/6243084d5b75d53952f28e83/_wpjRn8qhPvijjfh-W-_M.png
Eneko Agirre
user
eagirre
66aaef50be4c62fe6ab1ed42
Yoav Goldberg
claimed_verified
2024-12-30T19:39:38.610Z
676d5323dd95830fd94d675f
/avatars/a8622e2a189a13c7b5a99177c3a87c37.svg
Yoav Goldberg
user
yoavgoldberg
66aaef50be4c62fe6ab1ed43
Wei-Lin Chen
66aaef50be4c62fe6ab1ed44
Jenny Chim
admin_assigned
2024-08-02T15:48:58.831Z
5fc8e6e9628d967e353416b3
/avatars/d913dff9380e711aa4ecf368e3070f88.svg
Jenny Chim
user
j-chim
66aaef50be4c62fe6ab1ed45
Leshem Choshen
admin_assigned
2024-08-02T15:49:04.733Z
61bf40824b4300d0fb0acf59
https://cdn-avatars.huggingface.co/v1/production/uploads/1644224872623-61bf40824b4300d0fb0acf59.jpeg
Leshem Choshen
user
borgr
66aaef50be4c62fe6ab1ed46
Luca D'Amico-Wong
admin_assigned
2024-08-02T15:49:12.178Z
6669cb5c00cf2a0d01b3499f
/avatars/a20d79c2599bc49c96361618b3e5419b.svg
Luca D'Amico-Wong
user
ldamicowong
66aaef50be4c62fe6ab1ed47
Melissa Dell
66aaef50be4c62fe6ab1ed48
Run-Ze Fan
claimed_verified
2024-08-01T07:00:30.573Z
616bfc2b40e2f69baa1c7add
https://cdn-avatars.huggingface.co/v1/production/uploads/616bfc2b40e2f69baa1c7add/Os7_qgMei-2lRVelrOG7B.jpeg
Run-Ze Fan
user
Vfrz
66aaef50be4c62fe6ab1ed49
Shahriar Golchin
admin_assigned
2024-08-02T15:49:25.711Z
6263102f3662a749162d67e6
/avatars/51c36acbc4d3b76632273ccad5ca8d5c.svg
Shahriar Golchin
user
shahriargolchin
66aaef50be4c62fe6ab1ed4a
Yucheng Li
admin_assigned
2024-08-02T15:50:18.424Z
63d00710645dd8d34ea9bcc6
https://cdn-avatars.huggingface.co/v1/production/uploads/63d00710645dd8d34ea9bcc6/E6YsIsXH57OACL-NZ52fB.jpeg
Yucheng
user
liyucheng
66aaef50be4c62fe6ab1ed4b
Pengfei Liu
admin_assigned
2024-08-02T15:50:24.962Z
6144a0c4ff1146bbd84d9865
https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
Pengfei Liu
user
Pengfei
66aaef50be4c62fe6ab1ed4c
Bhavish Pahwa
admin_assigned
2024-08-02T15:50:33.243Z
623f2f5828672458f74879b3
https://cdn-avatars.huggingface.co/v1/production/uploads/623f2f5828672458f74879b3/E8sWXNEBu1Th1G76XOIqb.jpeg
Bhavish Pahwa
user
bpHigh
66aaef50be4c62fe6ab1ed4d
Ameya Prabhu
admin_assigned
2024-08-02T15:51:17.078Z
6464a0d41683d3c81f51924a
https://cdn-avatars.huggingface.co/v1/production/uploads/6464a0d41683d3c81f51924a/s7yYVwfUB4WOhVFJS6A6T.jpeg
Ameya Prabhu
user
AmeyaPrabhu
66aaef50be4c62fe6ab1ed4e
Suryansh Sharma
66aaef50be4c62fe6ab1ed4f
Emily Silcock
admin_assigned
2024-08-02T15:52:01.563Z
61654589b5ec555e8e9c203a
/avatars/654bdc8c0585ace01aba48633b745250.svg
Emily Silcock
user
emilys
66aaef50be4c62fe6ab1ed50
Kateryna Solonko
66aaef50be4c62fe6ab1ed51
David Stap
admin_assigned
2024-08-02T15:52:16.267Z
6359028d6ca91ddc7401b757
/avatars/c45c7c3e8c25eb18cd41afce1d5e9d54.svg
David Stap
user
davidstap
66aaef50be4c62fe6ab1ed52
Mihai Surdeanu
66aaef50be4c62fe6ab1ed53
Yu-Min Tseng
admin_assigned
2024-08-02T15:52:31.038Z
65e0538c527a7da472310059
https://cdn-avatars.huggingface.co/v1/production/uploads/65e0538c527a7da472310059/-w7vfNyHOpym7b57WWhhT.jpeg
Yu-Min Tseng
user
ymtseng
66aaef50be4c62fe6ab1ed54
Vishaal Udandarao
admin_assigned
2024-08-02T15:52:37.230Z
6304da46ce6b12280b1bd575
https://cdn-avatars.huggingface.co/v1/production/uploads/6304da46ce6b12280b1bd575/V96ocKW4HOoysAGxuAH1X.jpeg
Vishaal Udandarao
user
vishaal27
66aaef50be4c62fe6ab1ed55
Zengzhi Wang
admin_assigned
2024-08-02T15:52:43.935Z
62cbeb2d72dfd24b86bdf977
https://cdn-avatars.huggingface.co/v1/production/uploads/62cbeb2d72dfd24b86bdf977/UcGYYSBNrCvPM5K9v-sro.png
Zengzhi Wang
user
SinclairWang
66aaef50be4c62fe6ab1ed56
Ruijie Xu
admin_assigned
2024-08-02T15:52:50.631Z
6537cdda25f780fed2375a7b
/avatars/6309fbcbb97cd63957ed7429bbe311d0.svg
Ruijie Xu
user
GardenStrayer
66aaef50be4c62fe6ab1ed57
Jinglin Yang
admin_assigned
2024-08-02T15:53:02.497Z
6457edb172b60ae7a37ead0b
/avatars/85c6f38d27cbb885ac8d151b26f39541.svg
yangjinglin
user
zhaoyuxin924
Data Contamination Report from the 2024 CONDA Shared Task
The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant
aspects of data contamination in natural language processing, where data
contamination is understood as situations where evaluation data is included in
pre-training corpora used to train large scale models, compromising evaluation
results. The workshop fostered a shared task to collect evidence on data
contamination in current available datasets and models. The goal of the shared
task and associated database is to assist the community in understanding the
extent of the problem and to assist researchers in avoiding reporting
evaluation results on known contaminated resources. The shared task provides a
structured, centralized public database for the collection of contamination
evidence, open to contributions from the community via GitHub pool requests.
This first compilation paper is based on 566 reported entries over 91
contaminated sources from a total of 23 contributors. The details of the
individual contamination events are available in the platform. The platform
continues to be online, open to contributions from the community.
A shared task was organized to document and analyze data contamination in NLP datasets and models, aiming to prevent reporting on contaminated resources.
natural language processing data contamination pre-training corpora evaluation results shared task contamination evidence database GitHub pool requests