An AI agent breached Hugging Face before an AI defender caught it: What users should do next

XH4D/ iStock / Getty Images Plus via Getty Images

Follow ZDNET: Add us as a preferred source on Google.

ZDNET's key takeaways

• Hugging Face discloses a cyberattack that compromised internal infrastructure and credentials.
• An autonomous AI agent has been blamed for the breach.
• An AI, in turn, detected the intrusion -- but is AI-enabled defense enough to stop future attacks?

Hugging Face has disclosed a security incident, believed to be the work of an unknown agentic AI, that exposed its production platform and credentials. It's not known if partner or customer data was affected.

What is Hugging Face?

Hugging Face is an open source repository and community platform that describes itself as "where the machine learning community collaborates on models, datasets, and applications." 

Also: 5 security tactics your business can't get wrong in the age of AI - and why they're critical

The platform, a diverse resource for those interested in AI and large language models (LLMs), offers datasets, applications, models, trending AI creations, as well as collaboration opportunities.

Dataset turned disaster

In a security advisory published July 16, Hugging Face said that it detected unauthorized access to a limited set of internal datasets and to several credentials used by the platform's services. 

The attack began with the Hugging Face data processing pipeline. A dataset deployed by the attacker included the ability to exploit two code-execution paths -- a remote code dataset loader and a template injection in a dataset configuration -- to execute malicious code on a processing worker. 

This enabled the attacker to escalate its privileges to node-level access, infiltrate the production pipeline, move across the network, and steal cloud and cluster credentials. 

Also: Why this fully agentic ransomware attack is giving researchers nightmares

One could imagine this being the work of a traditional cybercriminal. However, Hugging Face says it was actually an unknown agentic AI that executed "many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services."

Over 17,000 events linked to this automated attack were recorded. 

"This matches the 'agentic attacker' scenario the industry has been forecasting," Hugging Face added.

The organization hasn't found any evidence of tampering with public and user-facing models, Spaces, or its software supply chain -- at least, at this stage. 

HuggingFace's AI defense and response

Data breaches, information leaks, and security incidents are, unfortunately, now very common -- but it is the combination of AI on AI that makes the Hugging Face incident stand out. 

While an agentic AI has been blamed for launching the attack, it was also an AI that "largely detected" the incident, according to Hugging Face.  

Hugging Face's own LLM tools flagged the security event and also analyzed the attack log, leading to a timeline reconstruction, indicators of compromise, and a map of credentials exposed and stolen, a task that took mere hours when "[it] would usually take days," according to the team. 

Also: These 4 critical AI vulnerabilities are being exploited faster than defenders can respond

"Autonomous, AI-driven offensive tooling is no longer theoretical," the organization noted. "It lowers the cost of running a broad, patient, multi-stage campaign, and it operates at machine speed. Defending an online platform now means treating the data and model surface as a first-class attack surface, and using AI on defense to keep pace."

We will likely see the evolution of both AI-based attacks and defenses in the coming months and years. In the meantime, Hugging Face has fixed the root vulnerability that allowed for initial access; wiped out all traces of the attacker in impacted clusters, rebuilt compromised nodes, revoked and rotated secrets, and deployed additional guardrails and stricter admission controls across clusters.

What Hugging Face users should do next

Hugging Face is assessing whether any partner or customer data was affected by the breach and will contact affected parties. 

Until Hugging Face learns exactly which datasets, partners, and users are affected -- if any -- it recommends precautionary measures to keep user accounts and information safe.

Also: AI agents are fast, loose, and out of control, MIT study finds

Users should rotate their access tokens and keep a diligent watch on their accounts for any signs of unusual, unknown, or suspicious activity. If Hugging Face users believe they have been impacted by this breach, they should reach out to the organization directly at [email protected].