Yahoo uses AI to detect online abuse
Yahoo has developed a machine learning algorithm to detect aggressive and abusive comments online. The algorithm was able to identify hateful speech successfully in 90% of cases, having been trained using over one million comments from Yahoo's New and Finance sites. If a post is flagged as abusive by the algorithm, it is either automatically deleted or forwarded on to a human moderator to decide. The algorithm does however struggle to identify sarcastic comments and newer ways to express hate.

algorithm iStock

Yahoo has created an abuse-detecting algorithm that can accurately identify whether online comments contain hate speech or not. In 90 per cent of test cases Yahoo's algorithm was able to correctly identify that a comment was abusive – a level of accuracy that the study claims outperforms other "state-of-the-art" deep learning approaches.

The company used a combination of machine learning and crowdsourced abuse detection to create an algorithm that trawled the comment sections of Yahoo News and Finance to sniff out abuse. As part of its project, Yahoo is releasing the first publicly available curated database of online hate speech.

Currently, most abusive language detectors work by searching user comments for hateful words or phrases. If a comment contains a particular keyword common in hateful posts then it's flagged as abusive by the algorithm and automatically deleted or forwarded to a moderator to take a look at.

But keyword-based systems – similar to a spam filters – aren't great at catching more subtle types of hate speech. Trolls might deliberately obscure abusive words so they sneak past filters or posts might be hateful without ever containing a particular abusive keyword. Existing algorithms can also mislabel sarcastic comments as abusive, fail to recognise new ways of expressing hate, or ignore a troll's post because it was written in grammatically correct English.