S TA R C O D E R A N D S TA R C O D E R B A S E : M U LT I - L A N G U A G E E VA L U AT I O N

In this section, we focus primarily on StarCoderBase, and evaluate its performance on a variety of
programming languages and programming tasks, including producing code from natural language
descriptions, documenting code, predicting type annotations, and more. This section also shows that
StarCoder, despite being fine-tuned on Python, remains a very capable multi-language Code LLM
and even outperforms StarCoderBase on some languages.

19

Model
Java
InCoder-6B 0.49
0.62
SantaCoder
0.73
StarCoder

JavaScript Python

0.51
0.60
0.74

0.31
0.44
0.62

Table 14: Performance on single-line fill-in-the-middle on the FIM benchmark by Ben Allal et al.
(2023).
StarCoder_paper (1)