Replit's new Code LLM: Open Source, 77% smaller than Codex, trained in 1 week
Some links:- Repo: https://github.com/replit/ReplitLM/tree/main/replit-code-v1-...- HuggingFace: https://huggingface.co/replit/replit-code-v1-3b- Demo: https://huggingface.co/spaces/replit/replit-code-v1-3b-demo- Early benchmark results: https://twitter.com/amasad/status/1651019556423598081A lot about this project was surprising. We knew it was going to be good, but didn't expect to be this good -- especially surprising was the finetuned performance boost, and the fact that the model is decent at language tasks and reasoning (in some cases much better than much larger general-purpose models).It feels like there is a lot more to do with this model, and I have a suspicion you can even make a half-decent chatbot (at least one focused on code) by finetuning it on conversation (and/or instruction) datasets.Will follow up with a more comprehensive technical report and the UL2R version (fill-in-the-middle support).