Jupyter – scripts We utilize Jupytext5 to convert notebooks to scripts. It is an actively maintained
software that currently supports 31 programming languages. To initiate the conversion process,
Jupytext requires the identification of the specific programming languages within each notebook.
We extracted this information from the metadata of each respective notebook. However, more than
30,000 notebooks lacked any programming language information, making it difficult to convert them
to the script format. To address this issue, we incorporated the use of Guesslang,6 an open-source
library that employs machine learning techniques to identify the programming languages of source
code. By applying a probability threshold greater than or equal to 0.5, we successfully reduced the
number of unidentified notebooks to 6,400 using Guesslang. Ultimately, we amassed 1,432,992
scripts through the utilization of Jupytext. The distribution of programming languages among these
StarCoder_paper (1)