Data.
It is well-known that data quality is critical in the training and responsible development of LLMs
(e.g., Hoffmann et al., 2022; Penedo et al., 2023), and this is also true for code as discussed by Allal et al.

17
CodeLlama2