Leading AI Researchers Warn Automated AI Research Could Create Extreme Risks

More than 20 AI researchers, including Geoffrey Hinton, Yoshua Bengio, and OpenAI research lead Jakub Pachocki, warn that self-improving AI could drive a possible “intelligence explosion.” Their concern is that AI systems may soon automate AI research itself, compressing progress that normally takes years into months. The article says the warning appears in a new research paper and includes the line: “Although there remains much uncertainty, AI R&D automation might soon trigger one.”

The report says AI systems already write most of the code at the companies building them and could automate the full AI R&D pipeline within years. If that happens, the pace of model development could outrun institutions, safety processes, and public oversight. The practical takeaway: policymakers and organizations should track not just model capability, but how much of AI development work is being delegated to AI systems.

The researchers warn that society may not keep up if AI-driven research accelerates too quickly. The article highlights concerns that control over superhuman AI could slip away and that power balances between nations, companies, and governments could erode. The clearest action point is stronger visibility into how AI research is being automated before the process becomes harder to monitor.

The warning joins other recent concerns cited by The Decoder, including calls from leading mathematicians for more attention to existential AI risks and fears voiced by AI lab employees. The article also notes that Pachocki has said no lab has solved alignment well enough “to continue responsibly scaling at maximum speed for much longer.” Together, these signals point to a growing debate over whether AI progress is moving faster than safety and governance systems can adapt.