models has also raised concerns in the scientific community. Some of the best-performing LLMs,
such as Google’s PaLM (Chowdhery et al., 2022) and DeepMind’s Chinchilla (Hoffmann et al., 2022),
were developed in a completely closed manner, restricting model access to researchers within their
respective organizations. Although OpenAI and other AI startups have made their LLMs available to
the general public, they have done so through a paid API service and without sharing all the details
regarding the development process. While API access allows researchers to experiment with these
models, it limits their ability to research LLM safety and alignment (Perez et al., 2022) and inspect
the models’ inner workings (Olsson et al., 2022). Additionally, the high development costs make it
nearly impossible for academic institutions to develop these models from scratch, which has created
anxiety among academic researchers about whether they can meaningfully contribute to new AI
StarCoder_paper (1)