Facebook: Improvement in AI for hate speech
Facebook flagged and removed 90 percent of hate speech. But how accurate are these numbers?
Facebook’s algorithms for detecting hate speech are hard at work as it flagged almost 90 percent of the hate speech it removed in the first quarter. The question that needs to be asked is how accurate are the numbers?
Facebook and hate speech
Facebook’s latest quarterly report says the company removed 9.6 million pieces of content it deemed hate speech in the first quarter of 2020 — up from 5.7 million in the fourth quarter of 2019. The total was a record, topping the 7 million removed in the third quarter of 2019.
Facebook software detected 88.8 percent of the 9.6 million posts removed in the first quarter before users reported them. This is an improvement in the number of posts removed for breaching its rules on hate speech. This is partly due to better detection by the automated hate speech sniffers that was developed by Facebooks artificial intelligence experts.
How accurate is this?
The accuracy of the system is questionable as Facebook can’t estimate the total volume of hate speech posted by its 1.7 billion daily users.
Facebook chief technology officer Mike Schroepfer does not provide clarity on the accuracy of the numbers but says, “Our language models have gotten bigger and more accurate and nuanced. They’re able to catch things that are less obvious.”
Schroepfer don’t specify how accurate the automated hate speech sniffer systems are, only that Facebook tests systems extensively before they are deployed, in part so that they do not incorrectly penalise innocent content.
Also, the system is unable to indicate how much hate speech slips through its algorithmic net. The company’s quarterly reports estimates the incidence of some types of content banned under Facebook’s rules, but not hate speech. The Facebook hate speech section says Facebook is still developing a global metric.
The missing numbers hide the true size of the social network’s hate speech problem. The 9.6 million posts removed for hate speech appear small in comparison to Facebook’s huge network of users.
The content flagged by Facebook’s algorithms is reviewed the same way as posts reported by users. The process determines whether to remove the content or add a warning, and it can involve human reviewers or software alone.
Facebook’s recent transparency drive now also includes a new panel of outside experts with the power to overturn the company’s moderation decisions.
Facebook technology is not perfect but the company needs to disclose the numbers if it is to succeed in its transparency efforts.
This content has been created as part of our freelancer relief programme. We are supporting journalists and freelance writers impacted by the economic slowdown caused by #lockdownlife.
If you are a freelancer looking to contribute to The South African, read more here.