Fears that AI could wipe out humanity within the next decade have changed Nate Soares' retirement planning. The president of the Machine Intelligence Research Institute has stopped contributing to his retirement account entirely.
Here's what to know
Soares discussed his concerns with Nicholas Thompson, CEO of The Atlantic, for Thompson's interview series, "The Most Interesting Thing in AI," according to TechSpot.
"I do not save for retirement right now. I have a 401(k) from when I worked at Google, but I haven't contributed for a decade," Soares told Thompson.
The title of Soares' book, "If Anyone Builds It, Everyone Dies," reflects his concerns about AI.
More background
Soares fears that human survival could be irrelevant to a superintelligent system intent on achieving its goals.
His concern doesn't necessarily depend on such a system hating people. Attempts to disable that AI or develop a competing system could put humans in the way of its objectives.
TCD Picks » Quince Spotlight
💡These best-sellers from Quince deliver affordable, sustainable luxury for all
Soares worries about a system that can run its own automated economy, eliminating its need for human support. He believes that independence would be the turning point, since AI currently depends on people for electricity and computer chips.
Soares isn't sure exactly how a catastrophe might unfold. Among the possibilities are deliberate human extermination and changes machines could make to the planet that would render it uninhabitable for people. Neither scenario is an established outcome.
What's being done?
The Machine Intelligence Research Institute, the nonprofit Soares leads, focuses on AI risks. According to TechSpot, Soares has worked on AI alignment for 12 years.
Soares offers no specific policy proposal, agreement, or technical safeguard for stopping the AI race. Even with his bleak outlook, he said the race toward AI he considers dangerous can still be halted.
"I don't think it's inevitable. I think we could absolutely still stop this race to the very dangerous sort of AI," Soares told Thompson.
Where can I learn more?
Soares isn't the only AI researcher warning that we could lose control of the technology.
• An Anthropic researcher resigned over fears that AI could kill humanity within a decade.
• AI pioneers warned governments that an intelligence explosion could outpace human control over the technology.
• Microsoft's president urged AI kill switches and breach alerts as Trump resisted new rules.
• Anthropic CEO Dario Amodei urged an AI slowdown and outside scrutiny of model safety practices.
• California Rep. Ro Khanna called for an FDA-style AI agency, warning against an AI war.
Get TCD's free newsletters for easy tips, smart advice, and a chance to earn $5,000 toward home upgrades. To see more stories like this one, change your Google preferences here.
