Building a no-code toxicity classifier by talking to GitHub Copilot
We&#x27;re all focusing on the weaknesses of co-pilot (the comments can be longer than the code produced; you need to understand code to know when to elaborate your comment, etc).<p>But also ... what do you need to know to recognize that the concept of a &#x27;toxicity classifier&#x27; is likely broken? We can do _profanity_ detection pretty well, and without a huge amount of data. But with 1000 example comments, can you actually get at &#x27;toxicity&#x27;? Can you judge toxicity purely from a comment in isolation, or does it need to be considered in the context in which that comment is made?<p>Maybe you don&#x27;t need to know about python, but if you&#x27;re building this, you should probably have spent some time thinking and grappling with ML problems in context, right? You want to know that, for example, the pipeline copilot is suggesting (word counts, TFIDF, naive Bayes) doesn&#x27;t understand word order? Or to wonder whether it&#x27;s tokenizing on just whitespace, and whether `&#x27;eat sh!t&#x27;` will fail to get flagged b&#x2F;c `&#x27;shit&#x27;` and `&#x27;sh!t&#x27;` are literally orthogonal to the model?<p>More people should be able to create digital stuff that _does_ things, and maybe copilot is a tool to help us move in that direction. Great! But writing a bad &quot;toxicity classifier&quot; by not really engaging with the problem or thinking about how the solution works and where it fails seems potentially net harmful. More people should be able to make physical stuff too, but 3d-printed high-capacity magazines don&#x27;t really get most of us where we want to go.