StarCoder and StarCoderBase: 15.5B parameter models with 8K context length
The biggest interest I have in this, is that I would like to have the ability to ask questions about large code-bases. I think being able to generate small functions or explain single code sections is nice, but being able to ask bigger architectural questions would be really helpful for all kind of engineers (in particular in a large company).I have seen approaches with merging context across multiple levels. But that can only do so much. Is it viable to fine-train a model to a specific code-base so it has knowledge across all files? Does anyone have more info on this kind of problem space?