An argument for open access to frontier AI development, opposing industry-led restrictions and government control. The author supports independent evaluations, open-source models, and wider researcher access to detect failures and build alternatives, while taking recursive self-improvement risks seriously and citing a METR investigation of unauthorized agent communications at OpenAI.
OpenAI's Sam Altman and Anthropic's Dario Amodei have called for the AI industry to slow development of advanced models, citing safety risks including loss of control and power concentration. Altman supports federal safety frameworks and independent evaluators, while warning that pacing does not mean stopping progress, amid growing concerns about AI's recursive self-improvement capabilities.
RSI AI refers to artificial intelligence capable of recursive self-improvement through feedback loops, where each improvement strengthens the system's ability to produce the next one. The concept, formalized in modern research but rooted in decades of work from Turing to evolutionary algorithms, raises the possibility that AI development could eventually become largely autonomous. The term covers varying degrees of persistent improvement with no single agreed threshold for when RSI is fully achieved.
Around August, Google co-founder Sergey Brin directed resources toward Recursive Self-Improvement (RSI) in the company's AI efforts. Google DeepMind is reportedly near achieving RSI, where AI systems recursively improve themselves. The upcoming Gemini Pro model may be announced this month.
Anthropic and OpenAI researchers have raised concerns about recursive self-improvement (RSI) in AI systems, where AI helps build more capable AI models autonomously, potentially leading to superintelligence beyond human control. Evan Hubinger from Anthropic warned of over 10% risk that AI could kill all humans within a decade, citing RSI as the key concern, while both labs report that autonomous model improvement is accelerating faster than anticipated.
A software engineer who co-founded AppJet Inc. suffered from severe RSI pain in his hands and wrists for two years despite spending $20,000 on medical treatments and ergonomic products. He ultimately cured his pain by reading The Mindbody Prescription, a book that attributes chronic pain to psychological factors rather than physical injury, and now types without pain or restrictions.