Welcome to the Era of Experience

Welcome to the Era of Experience

David Silver, Rich Sutton. 2025. (View Paper → )

We stand on the threshold of a new era in artificial intelligence that promises to achieve an unprecedented level of ability. A new generation of agents will acquire superhuman capabilities by learning predominantly from experience. This note explores the key characteristics that will define this upcoming era.

AI is transitioning from an era dominated by human data to one driven by experiential learning. While current AI systems like large language models have achieved impressive capabilities by training on human-generated data, they're approaching the limits of what can be learned from existing human knowledge. The authors propose that the next breakthrough will come from agents that learn continuously through direct interaction with their environments, generating their own data that improves as they become stronger. This new paradigm will feature agents that operate in continuous streams rather than discrete episodes, with actions and observations grounded in real-world environments, rewards derived from measurable outcomes rather than human judgments, and reasoning processes that aren't limited to human modes of thinking. The authors cite AlphaProof's medal-winning performance in the International Mathematical Olympiad as evidence this transition has already begun, and suggest experiential learning will ultimately enable superhuman capabilities across many domains by allowing AI to discover knowledge beyond current human understanding.