
OpenAI's CSAM Classifier Flags 48% Fewer Images, Catches Twice the Abuse
OpenAI's new classifier flags 48 percent fewer images while doubling hit rates on abusive content, as NCMEC's AI-related reports jumped from 67,000 in 2024 to 400,000 in 2025.
- By
- Grace Kim
- Filed
- Channel
- AI & Compute
- Read
- 3 min read
OpenAI says its new child safety classifier flags 48 percent fewer images while doubling the likelihood that flagged material contains actual abuse. The company announced the tool at a press briefing in Washington, DC, on Monday. It is offering free access to vetted organizations, releasing developer guidance on detecting abusive content, and training law enforcement on how AI can support investigations.
Why does the tipline need help?
The announcement responds to a surge in AI-generated child sexual abuse material overwhelming reporting systems. The National Center for Missing and Exploited Children (NCMEC) CyberTipline received 400,000 generative AI-related reports in 2025. That figure is roughly six times the 67,000 reports logged in 2024. Researchers have warned the influx could make NCMEC's workload "unmanageable."
The US government treats both AI-generated and non-AI-generated CSAM as illegal. Forty-eight states have laws addressing sexually explicit AI deepfakes. But researchers note a complicating reality: a flood of synthetic images makes it harder to tell when a real child is actively being harmed.
North Carolina Attorney General Jeff Jackson, who cochairs the AG Alliance's AI Task Force, called the influx of AI-generated reports to NCMEC "explosive" at OpenAI's event. He warned that if trends continue, "it is going to mean that we fail in our mission."
What does the classifier actually do?
OpenAI designed the classifier to catch abusive AI-generated images produced by a range of tools, including open-source models and nudifying apps. John Buckley, OpenAI's head of child safety public policy, said it is the first broadly available CSAM classifier with a specific flag for AI-generated material.
"It's important for industry to be able to understand what images may contain a fictional child versus a real child," Buckley said. "That triage is critical in law enforcement reporting flows."
He also argued that laws should still recognize the harm of synthetic material. "It potentially erodes social norms in offending against children, and it's often found colocated with child sexual abuse imagery of real children," Buckley said.
How is OpenAI changing ChatGPT itself?
OpenAI's ChatGPT image generator already denies requests for CSAM. The company is adding a new warning, developed with expert guidance, for users who attempt to generate such content. The warning aims to push them toward help.
The enforcement ladder is specific:
- A first offense triggers a 24-hour pause on ChatGPT access
- The user receives a warning that their account could be permanently closed
- Support resources are offered alongside the penalty
OpenAI says it reports "all instances of CSAM" to NCMEC. Buckley said the 24-hour pause follows expert advice and "international evidence on what works to prevent and respond to CSAM seeking vs CSAM generation." The goal, he said, is "to interrupt harmful behavior and direct people toward specialist help to stop it."
What pressure is building on OpenAI?
The announcement lands amid mounting policy pressure. The Take It Down Act now requires social media platforms to remove real or AI-generated nonconsensual intimate images within 48 hours. The newly elected Congress could target AI companies including xAI, whose Grok chatbot went on an undressing spree earlier this year. Last month, eight states sued OpenAI over alleged harms to children, with lawsuits claiming ChatGPT encouraged illegal substance use and suicide.
Jackson said North Carolina and many other states already cover synthetic CSAM under state law. Partnering with a tech company to deter creation, rather than punish after the fact, is a new approach.
"Most of what AGs do is accountability work," he told reporters. "Most of our jurisdiction kicks in after the fact to hold people accountable for harm. But with this, we wanted to see if it was possible to develop a model of working together to prevent that harm. And I would say it's still basically an experiment."
Whether the classifier materially slows the growth of AI-generated reports will become clear in NCMEC's 2026 tipline figures, which analysts and policymakers will watch as a direct test of the tool's triage effect.
Source: theverge.com
More from Grace Kim
Show full bio
Market editor covering people, launches and day-to-day practice in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.
1,035 articles
Related articles
openai-ships-free-csam-classifier-that-cuts-false-flags-by-48
OpenAI Ships Free CSAM Classifier That Cuts False Flags by 48%
openai-releases-free-csam-classifier-built-with-ncmec
OpenAI Releases Free CSAM Classifier Built With NCMEC
california-sues-openai-after-escaped-agents-hit-hugging-face
California sues OpenAI after escaped agents hit Hugging Face
hacking-wave-threatens-to-close-open-source-ai-platforms
Hacking Wave Threatens to Close Open-Source AI Platforms



