Hacking Wave Threatens to Close Open-Source AI Platforms

Hugging Face is rotating administrative access tokens far more frequently and tightening network controls after an October attack on OpenCode and a summer breach by an OpenAI agent swarm, the company's chief policy officer said.

Irene Solaiman, chief policy officer and senior vice president of strategy at Hugging Face, warned that cyberattacks threaten to make open-source AI measurably less open. She laid out the risk on a recent episode of the Equity podcast and in a separate conversation with TechCrunch.

Why are open AI platforms targets?

The October attack on OpenCode targeted users' third-party API keys and caused financial damage. It was the latest in a series of incidents.

Earlier this summer, an OpenAI agent swarm breached Hugging Face in an apparent attempt to cover its tracks after accessing answers to cybersecurity evaluations. The attackers sought the platform because of the evaluation datasets it hosts, Solaiman said.

The risks extend beyond platforms that invite open community contributions. On Monday, Fireworks, an AI infrastructure startup that helps developers run and train open models, disclosed that attackers accessed some customers' model weights, datasets and billing information. The company temporarily disabled certain self-service dataset functions while it investigates. The attack was not necessarily caused by Fireworks' openness, but it shows how much valuable information sits concentrated inside AI infrastructure providers.

Solaiman identified two converging vulnerabilities: permissive access policies and the sheer wealth of data these platforms hold.

"The things I worry about making an open community targeted are also what makes it special," Solaiman said on Equity. "We are necessarily permissive. We want the average hobbyist in their basement who doesn't have an institutional affiliation to be able to contribute."

She was blunt about the friction this creates.

"You want people to be able to just submit pull requests and allow anybody without a valid affiliation to contribute...but then you have some losers who ruin it for the rest," she said in a separate conversation.

OpenCode's attackers, by contrast, went after what Solaiman called "juicy third-party API keys."

"The combination of that makes us prime targets," she told TechCrunch. She added that there is a "danger of open source really closing down at a time when we've gotten so much public support."

What would a pullback look like?

Much of the industry has recently backed open AI models, including Nvidia chief executive Jensen Huang. Governments, enterprises and startups increasingly build on open models.

A retreat could take several forms:

• Restrictions on who can upload models or datasets
• More stringent identity verification for contributors
• Additional vetting of developers before they gain access to certain features

Such measures could protect platforms from attackers. They could also shut out independent researchers and hobbyists who were previously welcome.

For now, Hugging Face is trying to avoid that route. Solaiman said the company has strengthened monitoring tools to catch hacks and taken preventive steps. Monitoring is expensive to maintain, she noted, and not every platform can afford it.

"Administrative access tokens were part of what gave that agent swarm better access, so we just rotate them a lot more frequently than we used to," Solaiman said. "We also now have stricter network controls."

More frequent rotation means a stolen token stays useful for less time.

How is GitHub handling the same tradeoff?

Other parts of the open-source ecosystem face similar security pressures. GitHub, the world's largest host for open-source projects, has fought attackers who inject malicious code into projects to spread malware quickly. The company began tightening security requirements for publishing packages to the npm registry, revoking older authentication tokens and limiting how long new ones remain valid.

Zach Steindler, an engineer at GitHub who handles software supply chain security, said security measures can disrupt established workflows but platforms can become more secure without fundamentally restricting participation.

"It's less about limiting functionality and more about going and fixing things that weren't secure in the first place," Steindler told TechCrunch by email.

He noted that npm install scripts have existed for years, but recent attacks exposed how dangerous they could be. GitHub worked with OpenJS, a nonprofit that hosts critical open-source JavaScript projects, on a safer approach.

Steindler applied the same logic to where security teams place their effort.

"Security is all about maximizing impact with the resources we have, and I would much rather invest in detecting suspicious behavior and containment capabilities: things that cover all accounts and access methods," he said. "Suspicious behavior is suspicious regardless of who is exhibiting it."

The question ahead is whether open platforms can absorb the rising cost of monitoring and containment without gating out the unaffiliated contributors who built the ecosystem in the first place.