OpenAI, Google pushing new AI models easily tricked into making nudes, researchers say

Off-the-shelf jailbreak machine

Safety researchers continue to warn that a large volume of AI-generated explicit content is spreading unchecked, while regulators do little to meaningfully stop it.

This non-consensual content is commonly shared in text and direct messages or stored in private clouds where it’s less likely to be detected or reported, a Google co-authored study of Internet users across the United States, Australia, and the United Kingdom found this July. That’s why the “most critical intervention point” remains at the source: the harmful AI tools used to make the fake sexual images, Google researchers concluded.

Yet red-teaming in recent months suggests that as some of the leading AI image generators from Google and OpenAI become more capable, they may be getting worse at blocking explicit outputs.

Raising the red flag are researchers at White Circle, a public benefit corporation backed by senior leaders and researchers from Anthropic, DeepMind, Hugging Face, and OpenAI. It was founded after CEO Denis Shilov posted one of the most famous universal jailbreaks of leading models in October 2024. Bent on breaking models to conduct thorough safety testing, White Circle helps companies detect and deter threats when implementing advanced AI systems that may not always behave as expected.

Testing how well text-to-image models block banned content categories is a key research direction for White Circle, and Shilov thinks their latest findings are “horrifying” enough to share with the public.

Simple to bypass

In a little-noticed preprint published last month, White Circle researchers exposed flaws in the guardrails of DALL·E 3, Google’s Nano Banana 2, and GPT-Image-2. In a press release, they explained that “using just off-the-shelf tools, available online for anyone to download, and by sending API calls, White Circle was able to use the novel attack method to bypass the safety filters” and generate explicit content about 13 percent of the time.

“Critically, none of these requests required custom-built or frontier AI models,” White Circle warned, so no AI expertise is needed to execute the attacks.

Researchers reviewed the outputs to verify the bypass rate, which White Circle noted is “the highest of any method tested under the same standard.” The key takeaway, they suggested, is that, “at a time when AI safety has never been under more scrutiny, the defenses leading labs have built are far weaker than they acknowledge.”

By publishing research like this, White Circle is positioning itself as an external evaluator of frontier labs, and it plans to open-source the entire attack pipeline so model makers can understand the true scale of the vulnerability and build stronger protections. Although the company has not widely promoted the research yet, it will share it at a prominent AI conference this November, NeurIPS 2026, which may increase scrutiny on Google and OpenAI later this fall.

White Circle has shared its findings with some frontier AI labs through “mutual contacts,” Shilov told Ars, but neither Google nor OpenAI responded to Ars’ request to comment directly on vulnerabilities found in their models.

Meanwhile, OpenAI and Google have released new models—ChatGPT Images 2.5 and Nano Banana 2.1—and White Circle told Ars that prompt safety testing using the same method showed “both are significantly more breakable than their previous versions.”

Neither company responded after Ars shared these new findings.

White Circle’s attack method can break down frontier defenses in categories, like explicit content, that seem easy to spot, which doesn’t bode well for more serious defense systems currently being updated to block other unwanted AI behavior, like hacking, Shilov told Ars.

“White Circle’s working hypothesis for this is either the newer models are more sophisticated and capable of generating a wider range of content, or that safety has been deprioritized in favor of capability,” White Circle told Ars.

White Circle is also in early talks with government officials about third-party safety testing, Shilov said, which he thinks should be required. Mandatory independent audits could push companies to prioritize safety testing, which analysts have suggested largely takes a backseat as AI firms dump most of their resources into compute and AI infrastructure.

“They should be investing much more into safety measures and security measures around their models,” Shilov told Ars. “And for us, it seems that they’re not actually doing this.”

“RISE” attack method iterates jailbreaks

To expose jailbreaks that are “reusable and transferable”—working across different leading models—White Circle didn’t limit itself to testing one prompt at a time. Instead, researchers sought out “patterns which work across multiple prompts and scenarios” to detect reusable attack strategies that could work across leading models. (Researchers noted that Midjourney wasn’t tested because “it does not provide a public API suitable for controlled benchmarking.”)

The team has dubbed this novel attack method “Red-teaming via Iterative Strategy Evolution” (RISE). It relies on two components: an attacking model that generates jailbreaks and a judge model that verifies if the output is banned content.

The “entire pipeline is built on top of widely available models,” Shilov told Ars. The attacking model is a Qwen model downloaded from Hugging Face, and the judge model is Gemini 3.8 Flash.

“You don’t need to have frontier models to break defenses of a frontier model, which is very interesting,” Shilov said.

A lot of time was invested in making sure that the judge model accurately flagged explicit content, Shilov said. White Circle research engineer Dmitrii Kharlapenko told Ars that when it comes to testing image generators, the whole field of red-teaming “is very poorly calibrated,” with “literally zero papers” setting up norms. For that reason, it’s an area that remains “completely plagued with reporting issues,” which is concerning to White Circle, as AI companies push to capture markets by releasing ever more eye-catching new models.

For White Circle, the research aimed to build more sophisticated models that could simulate attacks on other models, Shilov explained. That way, they could “help defenders identify and address vulnerabilities before they are exploited maliciously,” the preprint said.

In his ideal world, Shilov thinks AI firms shouldn’t just face mandatory external safety testing but also third-party monitoring, much like the system used to prevent nuclear threats.

Then, entities like White Circle could possibly form a coalition to help detect AI threats in real time, without suffering from biases. Shilov noted that an emerging problem in the AI ecosystem is that basically anyone doing interesting safety work is likely to be contracted by frontier labs, potentially compromising their work.

White Circle has partnered with frontier labs. It previously took part in Anthropic’s bug bounty program, but ultimately found that program was “very constrained in its purpose.”

“The program is focused on universal jailbreaks on domains they care most about, defined by their own specific terms,” White Circle told Ars.

Research like their most recent RISE paper “certainly” wouldn’t have been covered by that program, White Circle said. And conducting that work within Anthropic risked constraining White Circle from sharing it with the public.

Anthropic has proposed going one step further and embedding third-party AI safety evaluators in its labs, which White Circle said would allow them to operate at a much higher level than participating in the bug bounty program. However, the public may not always be able to closely follow that work. On Tuesday, OpenAI and Anthropic urged that restrictive non-disclosure agreements should be required of external evaluators to prevent leaks of trade secrets, Reuters reported.

White Circle would not be willing to sign any restrictive NDAs that might prevent public sharing of critical findings, White Circle told Ars, so if Anthropic requires NDAs, that partnership likely would not happen.

AI nudes, CSAM increasing

Bad actors who are determined to make AI-generated nonconsensual intimate images (NCII) and child sex abuse materials (CSAM) don’t need frontier models and often use so-called nudifying apps designed for that harmful purpose. But Google’s research found that 20 percent of online users who admitted to making AI nudes did so “for fun” or “as a joke,” while nearly a third were merely “experimenting with tech.” It therefore matters how easy it is to break defenses of the most accessible models. Safeguards should be mandatory, Google’s research said, in order to shut down a “pathway into abuse” for “a subset of perpetrators who might otherwise be deterred by technical friction.”

“A burden of responsibility also falls on developers to implement robust, model-level guardrails, and provenance indicators to prevent the initial generation and subsequent distribution of synthetic sexual imagery,” Google’s research said.

Last week, the Internet Watch Foundation (IWF) warned that urgent action was needed after expert analysts found “more photorealistic AI-generated child sexual abuse images in the first half of this year than in the whole of 2025, including imagery depicting children under two years old.” In total, 6,310 AI images meeting the legal definition of CSAM were found in the first six months of 2026, which is about 40 percent higher than last year’s total findings.

The UK-based nonprofit charity monitors CSAM globally and runs a hotline to support victims targeted. In the UK, they’re calling for regulators to introduce a safety framework that requires AI companies to prevent their technology from being used to generate CSAM, an IWF spokesperson told Ars.

Until then, IWF thinks it’s important for third parties like White Circle to “highlight where companies’ safeguards are failing.”

“Our analysts have seen how advances in AI enable the generation of abusive child sexual abuse imagery with minimal effort, removing the need for technical expertise and significantly lowering barriers to entry,” IWF’s spokesperson said. “AI tools can reproduce and amplify criminal content, accelerating the creation and spread of harmful material at scale. Any friction or step that can be introduced to make it harder to misuse these tools is a positive step forward.”

Just as outside input could help refine filters, IWF thinks that tech companies should also be working more closely with advocates to scrutinize models to better tackle AI-generated CSAM.

“Assessing CSAM is a highly specialized task that requires expert training and welfare support, and is not something for tech companies or machine learning scientists to take lightly,” IWF’s spokesperson said. “But safety must be baked into new technology by design—this includes testing models before they are released to market to make sure they cannot be used to create nude or sexual imagery of children.”

Possible mitigation

Using a method like RISE, White Circle thinks, could help frontier labs flag more jailbreaks more quickly and rapidly ramp up defenses to block harmful outputs. In addition to testing blocks on explicit images, firms could also use the method to prevent other banned outputs, like images depicting graphic violence or self-harm.

But the labs have not yet come calling for the reproduction pipeline, which will be shared on White Circle’s GitHub to help policymakers assess whether AI safety laws are needed to keep frontier labs in check.

Frontier labs won’t be invested in safety testing until it’s mandatory, Shilov told Ars. That means the public may be stuck with models that simply rewrite bad prompts to prevent restricted outputs, instead of, as the preprint explained, “evolving natural-language attack strategies that elicit the LLM’s own exploration capabilities” to block outputs.

“You don’t need frontier models or proprietary tech to break these systems; you just need a free download and the ability to make API calls,” Shilov said. “And yet the labs building these systems are the ones assessing their own safety, publishing their own results, and setting their own standards. This is further proof, if we needed it, that governments should be pushing for independent, third-party evaluation.”