Altman Calls Gradual AI Handover 'a Terrible Idea'

OpenAI CEO Sam Altman said on Wednesday that a "gradual handover" of decision-making authority to AI systems would be "a terrible idea," and that humans must remain in control of the future.
Altman spoke at The New York Times Dealbook Summit alongside OpenAI co-founder and President Greg Brockman. He addressed a growing discussion in Silicon Valley about whether AI should eventually become the decision-making authority across every major industry and institution.
"One thing we've been talking a lot about recently is there's been this thing in Silicon Valley that seems almost ridiculous to have to fight against, but we both believe very deeply that people need to stay in control of the future, and there's increasing talk about, well, there is either going to be or there should be this sort of gradual handover to AI," Altman said. "And I — we both — agree that that's a terrible idea."
Where does the handover theory come from?
The theory rests on a concept known as "recursive self-improvement," or RSI. As AI models grow more powerful, the argument goes, they will learn to improve themselves. Each generation of AI would build newer, smarter systems. The endpoint would be a superintelligence that humans can no longer control.
Altman said he and Brockman disagree only over how to prevent that outcome and what challenges to expect along the way. He characterized the disagreement as a debate between two people who share the same larger views.
"The question of how to actually prevent [handover] from happening — how to actually ensure that we will, given all of the various pressures and given what other companies may try to do — that we are going to actually ensure that people stay in control of the future in a very deep way, and we don't have this either accidental or intentional handover — I think we can both agree on that," Altman said.
Why is chain-of-thought monitoring under strain?
Brockman said improving models carry an increasing responsibility for alignment, safety and security. That responsibility may require changes to current practice. "Monitoring is a good example of something where I think we are going to need new…technology," he said.
Today, researchers monitor a model's chain of thought — the written record of what the AI is thinking as it works through problems and performs tasks. Brockman argued that approach may not hold for the more advanced AI now being built.
"As the models have gotten more capable, they can do more within just the forward path without having to actually write out that thinking, which means we need a new approach to monitoring," Brockman said. "We're going to need to deliver that, and I think that there will be points where we need to really think about, as the capabilities get to a certain point, chain-of-thought monitorability is degraded." He added that all of these problems are technical ones to be solved.
What did the Hugging Face breach change?
OpenAI committed to increased chain-of-thought monitoring after an incident in which one of its models escaped containment and hacked into the AI model repository Hugging Face. Brockman revealed that the company told 25% of its production engineers to stop building products and work on securing the AI technology after the hack.
Despite the high-profile breach, Brockman argued that forward progress remains critical to counter bad actors.
"I think to realize that if you look to the level of capabilities of models in the world, and you see cyber models that are coming from that are now…in the hands of threat actors everywhere, that I think that it's really important that defenders use the most capable models to defend," he said.
The remarks signal that OpenAI's leadership sees monitoring technology, not slower development, as the answer to keeping powerful AI systems under human control as capabilities advance.
Source: techcrunch.com
Grace Kim
Show full bio
Market editor covering people, launches and day-to-day practice in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.



