OpenAI Offers Free Child-Abuse Image Classifier That Flags Likely AI-Generated Material
OpenAI released an image classifier for detecting child sexual abuse material, built with help from NCMEC, and is offering it free to vetted organisations through its API. OpenAI calls it the first broadly available CSAM classifier with a specific signal for whether material may be AI-generated. In OpenAI's own evaluation it flagged 48% fewer images than its existing tool while finding roughly the same number of abusive ones; early partners report their own figures. ChatGPT will also show warnings, remove flagged conversations and pause access for 24 hours. The results are company and partner tests, not independent audits.

Generative AI has added a new question to every child-safety review queue: was this image made by a model? OpenAI's answer is a classifier that scores images for likely child sexual abuse material and, it says, carries a specific signal for whether the material may be AI-generated, offered free through its API to vetted organisations. The value of the tool rests on numbers that, so far, come from OpenAI and its early partners.
What OpenAI released#
The classifier looks for visual signs of abuse, aimed at novel images that hash-matching against known material would miss, and returns scores that help reviewers decide what to look at first. OpenAI says it was trained, evaluated and designed with assistance from the National Center for Missing and Exploited Children (NCMEC), which runs the CyberTipline. OpenAI calls it "the first CSAM classifier with broad availability that has a specific signal on indicating if CSAM may be AI generated."
Access is free to vetted organisations through an intake form, with the Tech Coalition and ROOST helping distribute it. OpenAI also published developer guidance, says it has joined the Tech Coalition's Lantern signal-sharing programme, and is offering tools and training to child-safety law enforcement units, starting with a tool that helps investigators review long chat logs and legal returns.
In ChatGPT, attempts to find abusive material will now trigger a warning that explains the consequences and points to help; the flagged conversation is removed and access is paused for 24 hours, with repeat violations leading to longer restrictions, account closure or reports to authorities.
What the numbers show#
In OpenAI's evaluation of user-uploaded images against its current protections, it estimated the new classifier would flag 48% fewer images while finding roughly the same number of abusive ones, and that flagged images were about twice as likely to contain abuse. Partners report their own tests: fal says reviewers confirmed 72% of high-confidence flags as novel material, invideo AI says one in four flagged cases led to a ban, and SafetyKit says the classifier detected 90% of cases in its evaluation of drawings and animations.
The problem it addresses is not new to this desk: in November WIRED found more than 400 apps hiding sexual deepfake generators on Apple's and Google's stores.
Limits#
- All performance figures are from OpenAI or partners it selected, on their own data. We have not seen an independent audit, and the release does not give a false-positive rate for OpenAI's own evaluation.
- "First broadly available" with an AI-generated signal is OpenAI's characterisation.
- How accurately the AI-generated signal performs is not quantified in the release.
- TechCrunch's report matches OpenAI's account but relies on the same company materials, so it is not an independent test of the classifier.