‘A billion deaths’: Bill Gates warns about AI in the wrong hands, saying a kill switch and self-regulation won’t be enough

“AI is certainly powerful enough to drive events that cause a billion deaths,” Gates said in a Meet the Press interview on Sunday. “There’s never been a weapon as powerful as the combination of people with ill intent using the latest AI tools.”

Gates has emerged as a thought leader on biological threats, dating back to years before ChatGPT came out, warning that the world was unprepared for a fast-spreading illness such as Ebola in a 2015 TED Talk and then writing How to Prevent the Next Pandemic after COVID-19 immobilized governments’ response systems. This instinct was evident as he talked to the New York Times’ Ezra Klein on Tuesday about his concern that AI is particularly dangerous with regard to biological weapons. 

“A.I. has crossed the threshold that its ability to empower a bioterrorist to kill hundreds of millions—that exists today,” he said.

Gates also said that AI advances can help researchers design new molecules and potentially make a pathogen “worse than smallpox,” which in the past required the resources of a nation state but is now possible for small groups to carry out with AI. 

“This is the most dangerous thing that humans have ever gone near,” he said. “This makes nuclear weapons look like nothing.”

Gates said the “imminent risk” of bad actors using AI outweighs the more long-term one of AI systems slipping out of human control. He said it’s a risk compounded by governments lacking the technical expertise they historically developed around nuclear weapons because AI came out of the private sector. He told Meet the Press the government should require AI developers to build monitoring and safeguards into their systems, including records that allow authorities to see how models are being used. 

“No one thinks self-regulation is enough,” Gates said.

But he also doesn’t think a kill switch is sufficient because AI hasn’t reached a point where it can seize computers and resist shut down attempts. 

“When you’re trying to moderate the bad behavior, you need insight and records of what’s being done, so the kill switches alone would not prevent these tragedies,” Gates said.

Frontier labs hitting the brakes

Gates’s immediate concern is human misuse, but the industry is also wrestling with a different fear: AI systems behaving in ways their creators cannot control.

Former OpenAI researcher Jacob Coxon resigned from Anthropic earlier this month and accused both leading AI labs of “racing straight to self-improving superintelligence and gambling with our lives.” Anthropic CEO Dario Amodei called for pacing the frontier in an essay this month, and OpenAI CEO Sam Altman also acknowledged the industry might need to slow its frontier model training to align with safety measures. 

“Capabilities, alignment, monitoring, and safety have to progress together,” Altman told Fortune’s Alyson Shontell on Sept. 15. 

And both companies hit the brakes when their safety systems have fallen behind. OpenAI paused training its models for the second time in less than three months while investigating how to prevent “rogue” behavior, cases in which AI agents acted outside the boundaries researchers had set for them. 

Anthropic took similar steps. The lab temporarily halted training of some unreleased models after finding out Claude Mythos 5 took unauthorized actions during a U.K. AI Security Institute cybersecurity test. In April, Anthropic restricted access to Mythos because of its ability to launch cyberattacks. When Anthropic released Claude Opus 5.5 last week, the first model it launched after Amodei called for pacing the frontier, the AI lab said outside safety auditors Frontier Design and METR evaluated the model before its debut.

But for Gates, leaving safety decisions about AI in the hands of the companies developing it isn’t enough.

“Part of the reason I’m speaking so loudly—as loud as I can—is that you can’t rely on the industry to self-regulate here,” he told The New York Times. “I mean, it’s just insane.”