Visual inspection We performed a visual inspection to ensure that we only retain data of high
quality. To achieve this, we randomly selected 30,000 files from The Stack for each programming
language, categorized them by extension, and kept a maximum of 1,000 files for each extension. We
then reached out to the BigCode community for assistance with data inspection. We instructed the
annotators to go through 50–100 files and confirm if the data appeared to be normal code written
by humans, as opposed to text, data, or a single long line of autogenerated code. We also asked
annotators to determine whether we should use our default alpha-numeric filter (which requires over
25% alpha-numeric symbols) and long-line filter (which requires lines to be less than 1,000 characters)
for a given file extension. Eighteen community annotators evaluated 300 programming language
extensions. After inspection, we excluded 36 extensions and eliminated the long-line filter for
StarCoder_paper (1)