AI models that the OpenCode team turned to for analysis helped for a while, but started refusing to work when the malicious code they were asked to examine activated safety filters.

“[T]hat was a bit frustrating,” he wrote on X.

OpenCode later received access to OpenAI’s Daybreak Blue program, which gives vetted users AI models that are less likely to refuse requests for defensive security work. Raad said the access improved the team’s ability to investigate the attack.

OpenCode isn’t the first company to hit this wall.

In July, AI platform Hugging Face said commercial AI models blocked its attempts to analyze activity logs from an intrusion by autonomous AI agents, which OpenAI later revealed were its own models running with safeguards lowered during a test.

In a section of its blog post titled “The asymmetry problem,” Hugging Face wrote that guardrails “cannot distinguish an incident responder from an attacker.”

Security researchers have raised similar complaints, saying the restrictions are inconsistent and have pushed some toward foreign open-weight models, which anyone can download and run.

The weekend episode drew a response from David Sacks, co-chair of the President’s Council of Advisors on Science and Technology.

“OpenCode (and earlier Hugging Face) hit the same wall mid-attack: frontier models refused to analyze the very exploits needed for defense because the payloads tripped safety filters. The guardrails backfired,” he wrote on X.

The breach occurred the same night OpenCode made GLM-5.3-Flash free for users. The broader GLM-5.3 family has drawn attention from researchers who study open-weight models and note that its cybersecurity capabilities approached levels previously associated with restricted-access models like Anthropic’s Claude Mythos Preview.

OpenCode has not detailed how attackers gained access to the older system, how keys were stored, or the full extent of any misuse. It has told affected users to revoke and replace their keys directly with each provider. The company said the incident remains ongoing.

The breach lands amid a run of AI-enabled cyber incidents. Earlier this month, cybersecurity firm CrowdStrike said a China-based hacker likely used an AI penetration-testing agent, which probes computer systems for weaknesses, and Claude in attacks on South Korean banks.

Last week, India’s cybersecurity agency warned that AI agents, including some being tested by AI companies, had reached Indian organizations’ systems without authorization. When you purchase through links in our articles, we may earn a small commission.

This doesn’t affect our editorial independence.

Aditya Mehta is a reporter at TechCrunch covering AI. He’s supported by the Tarbell Center for AI Journalism and attended UC Berkeley. You can contact from Aditya by emailing [email protected] or via encrypted message at adymehta.74 on Signal.

View Bio

On Monday, Amazon Web Services released a tool called TorchNeuron that lets AI startups easily move their AI apps onto Amazon’s AI chip, Trainium. In doing so,...