Social platforms lean on AI moderation, but mistakes keep piling up

In short: Social media companies are using more AI to police posts, but repeated false bans and deletions show humans still need to be involved.

What's going on

Social platforms are leaning more on AI moderation, meaning automated systems that scan posts and decide what to remove or punish. Companies say this helps them act faster and at larger scale. But recent examples show the same speed can also create a lot of mistakes.

On Reddit, moderators of r/AskHistorians say dozens of posts and comments going back about 10 years were automatically removed. One moderator, Dr. Sarah Gilbert, said there was no clear way for them or the original experts to stop it. The moderators believe Reddit’s updated AI tools may have treated links to a history photo website as spam, which then caused legitimate answers to disappear.

Discord also reported a major error. It said its AI system wrongly banned about 8,400 accounts from May to early July after mistaking images with square grids, like chessboards or spreadsheets, for child sexual abuse material. Discord said a bug let the AI skip the required human review step, and it later reinstated the accounts.

Similar complaints have been reported by users on Meta’s Facebook and Instagram, and Tumblr has acknowledged wrongful automated bans affecting under 200 accounts in one afternoon. Researchers also warn that automated moderation can hit marginalized groups harder, for example when people quote hateful language to criticize it.

What to watch

Platforms are trying to add better controls, like Reddit’s Rules Hub, which lets human moderators choose what gets enforced automatically and what gets sent to a review queue (like a waiting line for a human decision). The key question is whether companies will invest in enough human oversight, so AI acts more like a helpful filter and less like an unchecked judge.

Source: Arstechnica