It once felt like science fiction. But artificial intelligence, or AI, is now integrated into our daily lives. It’s in search engines and accessible with a click of a mouse.
The technology holds promise for opening new doors to scientific discoveries. But industry leaders warn that its rapid development may pose some serious risk.
Jacob Coxon, a former researcher for leading AI company Anthropic, sounded the alarm.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” he said in a viral social media post.
The claim prompted AI developers to call for an industry-wide slowdown and push for more regulations.
“At the federal level there’s basically no statutory safeguards,” Rep. Sara Jacobs, D-San Diego, said. “We’ve been able to get some in the National Defense Authorization Act around the use of AI in nuclear weapons deployment, things like that, but congress has not really passed AI safeguards.”
Jacobs represents the 51st District and supports writing AI safety protocols into law. She’s introduced several bills to Congress to standardize AI systems and hold AI companies more accountable.
One bill, called the AI Threat Output and Monitoring Incident Containment Act (ATOMIC Act), would require National Laboratories to evaluate AI systems for vulnerabilities that could lead to potential nuclear security risk.
“I think the risks are very real,” Jacobs said. “A lot of what we’re seeing right now is in the testing phases, is these AI agents not following the rules they were given or finding to deceive the companies in the role … these sort of tasks that they’ve been given to do in this testing environment.”
Coxon warns these defects could lead to existential threats to society. That includes AI’s ability to create biological weapons, influence mental health, and incorporate problematic biases in things like job screenings.
Many notable AI companies have supported efforts to slow the industry’s pace. DeepMind’s Demis Hassabis, OpenAI’s Sam Altman, and SpaceX’s Elon Musk are among them.
In August, more than 100 AI companies signed an open letter to push for better defenses against AI cyberattacks.
Despite the collective caution pouring in from industry leaders, the Trump administration does not support adding more guardrails to AI. He suggests that an industry slowdown would cause the U.S. to lose its competitive edge over China.
