OpenAI releases tools to detect child sexual abuse material
Summarised from 2 outlets · Updated 16 Nov, 17:13 · Archive
OpenAI has released an image classifier to detect child sexual abuse material and introduced warnings and 24-hour account suspensions in ChatGPT.
OpenAI has released new tools designed to combat child sexual abuse material, addressing growing concerns about AI's misuse. The company developed an image classifier that detects CSAM, including content generated by artificial intelligence. The tool examines visual signs of abuse and assigns a score to help reviewers prioritise cases. OpenAI created the classifier with the National Center for Missing and Exploited Children and is providing access to select companies through its API at no cost.
In early testing, the classifier proved more efficient than existing systems. It flags 48 per cent fewer images for human review while identifying the same number of abusive images, and images it flags are twice as likely to be abuse material. This reduces the burden on human reviewers who currently use digital fingerprints to identify previously known abuse imagery, a method that cannot detect newly created material.
OpenAI is simultaneously strengthening ChatGPT's defences against abuse. The platform will display warning alerts to users attempting to access or create content that sexualises children, directing them towards support services and suspending their accounts for 24 hours. The company said repeated violations will result in permanent account removal. ChatGPT already blocks requests for CSAM and related content.

How it is being reported
- OpenAI child safety tools target AI-generated abuseOpenAI's new classifier flags 48% fewer images for review while detecting similar levels of CSAM, and ChatGPT now suspends users for 24 hours.Fox News · 16 Nov, 17:00
- OpenAI releases a new image classifier and ChatGPT warnings to combat CSAMChatGPT will ban an account trying to access CSAM content for 24 hours as a warningTechCrunch · 16 Nov, 17:00
In this story: OpenAI · National Center for Missing and Exploited Children · ChatGPT · Tech Coalition · ROOST
This summary was written by AI from the headlines and standfirsts above, and states as fact only what at least two outlets report. How we use AI · Report a problem · Corrections log
Comments (0)