Nvidia on Monday launched a new security system that it claims can prevent AI agents breaching their testing environments and gain unauthorized access to outside systems, just weeks after the AI chipmaker’s billionaire CEO Jensen Huang downplayed alarm about recent safety breaches and pushed back on the need for an AI slowdown.
Nvidia’s billionaire CEO Jensen Huang has repeatedly downplayed concerns about AI safety and pushed back against calls for a slowdown in development.
AFP via Getty Images
Key Facts
In a statement, the company said its “Open Agent Safety Platform” is a reference system designed to improve AI security during both testing and deployment of new agents.
The announcement said that the “recent security incidents” have highlighted the need for such a system to allow AI companies to “enforce more control over long-running agents.”
While the announcement didn’t specify a particular incident, a Nvidia official told multiple outlets that this new platform could have prevented an OpenAI agent breaching testing protocols in July and hacking Hugging Face without being prompted to do so.
The two-part security system includes “Nvidia OpenShell,” which sets “boundaries for agents” running on CPUs, and “Sentry,” which runs on the company’s own BlueField chips and “quarantines and stops” any attempts by AI agents to go outside their defined boundaries.
The announcement listed several companies like Microsoft, Palantir, Hugging Face, SpaceXAI and CrowdStrike as examples of companies working with Nvidia’s new safety systems, but there was no mention of OpenAI.
The chipmaker said it has collaborated with Anthropic to bring “bring additional layers of security and control” to the Claude-maker’s AI agent stack.
What Has Huang Said About AI Safety Recently?
Huang has come out strongly against a push for more government regulation of AI companies and efforts to coordinate a slowdown in advanced AI development. Earlier this month, he told attendees at Salesforce’s annual Dreamforce event that the choice between AI safety and speed was a “false” one.
“You could definitely have both at the same time.” He then argued: “We don’t need new laws — we don’t need new regulations.” Huang also sided with Trump when the president called the Nvidia CEO live on stage during the All-In Summit in San Francisco and dismissed fears of AI safety as a hoax.
On the call, Trump argued that people opposing AI and data center construction were “playing right into the hands” of skeptics and China and declared, “We’re not going to let that happen.” Huang told the president, “You’re right. We’re not going to let that happen, sir.” On fears that advanced AI could cause human extinction by the end of the decade, Huang told CBS News that there was a “0% chance” of that happening.
Forbes Valuation
According to Forbes’ estimates, Huang has a net worth of $194.9 billion, making him the seventh-richest person in the world. His company, Nvidia, is the world’s most valuable company, with a market capitalization of $5.42 trillion.
