AI leaders including Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have warned that AI poses serious risks, with some researchers fearing it could enable bioweapon development. AI tools can generate dangerous molecules and provide scientific instructions that, combined with accessible biotech and gene-editing tools, create potential biosecurity threats that existing safeguards may not fully contain.
In October 2014, the White House paused funding for gain-of-function research on influenza, SARS, and MERS viruses to assess risks and benefits. The National Science Advisory Board for Biosecurity tasked Gryphon Scientific with conducting comprehensive risk and benefit assessments covering biosafety, laboratory security, information misuse, and research benefits.
The author argues that realistic scenarios for AI causing human extinction lack scientific rigor, often relying on speculation rather than detailed planning. While acknowledging real harms from AI and biosecurity risks deserve serious attention, the author contends that extinction-level scenarios typically require implausibly magical capabilities—like perfect bioweapons or unprecedented AI manipulation—that ignore how defenses and human ingenuity would likely adapt.
The US and China should establish a Strategic AI Dialogue (SAID), a permanent government-to-government mechanism to manage AI risks where both countries are mutually vulnerable, including nuclear command and control, biological risks, frontier-model safety, and financial stability. Led by Treasury Secretary Bessent and Vice Premier He Lifeng, SAID would meet quarterly in a neutral country with continuous working groups, focusing on risk reduction rather than resolving technological competition.