352
Productivity & Workflow352
Software Development247
Automation & Workflow239
Marketing & Growth204
AI Infrastructure & MLOps193
Photography & Imaging166
Writing & Content Creation192
Design & Creative171
Data & Analytics165
Customer Support155
Education & Learning142
Voice & Speech130
Sales & Outreach132
Operations & Admin119
OpenAI launched a new tool to spot child sexual abuse material in images and added warning screens and a 24-hour account suspension in ChatGPT for related attempts.
In short: OpenAI announced new tools to help detect and deter child sexual abuse material (CSAM), including an image checking tool and new warning screens in ChatGPT.
OpenAI said it is releasing a new image classifier, which is software that checks an image and estimates whether it may contain CSAM (illegal sexual images involving children). Think of it like a spam filter, but for images, and with human reviewers making the final call.
OpenAI said the classifier can detect CSAM even in images generated by AI. It will be offered for free to select companies through OpenAI’s API (an API is a way for different software systems to connect), and it gives each image a score to help reviewers decide what needs urgent attention.
OpenAI said it built the tool with the National Center for Missing and Exploited Children, and it is working with the Tech Coalition and ROOST to share access. In early tests with partners, OpenAI said the classifier flagged 48% fewer images while finding the same number of abusive images, which could reduce the workload for review teams.
OpenAI is also adding warning screens in ChatGPT for users who try to create or access content that sexualizes children. The company said ChatGPT will show an alert, point users to specialized help to stop harmful behavior, and suspend the account for 24 hours as a warning. OpenAI said repeat attempts can lead to a permanent ban.
AI tools can be misused to create or spread illegal content, including CSAM. These updates aim to stop some of that misuse earlier, and to help the people who review reports focus on the highest risk material.
Source: TechCrunch AI