The challenges of producing quality code when using AI-based generalistic models
This is one of the things that bugs be about AI code generators. They are pretty helpful when I&#x27;m writing quick generic routines. But they don&#x27;t work for anything slightly novel (I write research code). The other big factor is that the models are trained on GitHub, SO, Reddit, etc and have no real quality filter. That&#x27;s good for breadth, but not for depth and high quality code[0,1]. I&#x27;m already a shitty coder, it&#x27;s not that helpful to have a companion write other shitty code who doesn&#x27;t understand the whole ecosystem we&#x27;re building within nor the abstract idea we&#x27;re trying to build.<p>I hear a lot about how people say they can&#x27;t live without copilot now and I&#x27;m often confused (not new programmers, but I&#x27;m hearing SWEs say this). Is it because we&#x27;re all just writing junk code as fast as possible and say fuckall to technical debt? Am I using the tools wrong? (Not sure how) Or what is going on? I really just hear these two opinions &quot;omg it changed my life&quot; and &quot;yeah, I get some use out of it, but just saving me a google search mostly.&quot; Is my sampling just really biased?<p>[0] yes, there&#x27;s lot of high quality code on github, but it is dominated by a lot of shitty code. And high quality code is often high quality because of the complex interconnections between many different routines which is a level of complexity LLMs don&#x27;t yet have and requires quite large context lengths. We&#x27;re getting there but there&#x27;s still good performance degrade as context length increases even if all the text fits within that context.<p>[1] Don&#x27;t get me started on HumanEval as a test set...