The Heap: A Contamination-Free Multilingual Code Dataset for Evaluating
  Large Language Models
The recent rise in the popularity of large language models has spurred the
development of extensive code datasets needed to train them. This has left
limited code available for collection and use in the downstream investigation
of specific behaviors, or evaluation of large language models without suffering
from data contamination. To address this problem, we release The Heap, a large
multilingual dataset covering 57 programming languages that has been
deduplicated with respect to other open datasets of code, enabling researchers
to conduct fair evaluations of large language models without significant data
cleaning overhead.
678a632dd03f325c5a5ad954
Jonathan Katzy
claimed_verified
2025-01-17T17:05:55.731Z
643d13b04816d7cb420966dc
/avatars/4f4875e35fa57c2b958cdb5dd6b70c27.svg
Jonathan katzy
user
Jkatzy
678a632dd03f325c5a5ad955
Razvan Mihai Popescu
claimed_verified
2025-01-17T17:05:53.530Z
6446fea4bd9e0e0edf521d49
/avatars/493c16b52e1a9a940b4e02b6cd6c6cc3.svg
Razvan Popescu
user
Razvan27
678a632dd03f325c5a5ad956
Arie van Deursen
678a632dd03f325c5a5ad957
Maliheh Izadi
admin_assigned
2025-01-17T14:25:21.390Z
63dd4b2af37111482526e0e9
/avatars/ffd2a07c61ea6272009df6184cb9dcef.svg
Maliheh Izadi
user
MalihehIzadi
The Heap: A Contamination-Free Multilingual Code Dataset for Evaluating
  Large Language Models
The recent rise in the popularity of large language models has spurred the
development of extensive code datasets needed to train them. This has left
limited code available for collection and use in the downstream investigation
of specific behaviors, or evaluation of large language models without suffering
from data contamination. To address this problem, we release The Heap, a large
multilingual dataset covering 57 programming languages that has been
deduplicated with respect to other open datasets of code, enabling researchers
to conduct fair evaluations of large language models without significant data
cleaning overhead.
A large multilingual code dataset, The Heap, enables fair evaluations of language models by minimizing data contamination and cleaning overhead.
large language models code datasets deduplicated multilingual programming languages fair evaluations data contamination data cleaning