David Silver and Richard Sutton, two renowned AI scientists, argue in A New paper This artificial intelligence will enter a brand new phase, the “era of experience”. Here AI systems are less and fewer based on data provided by humans and improve by collecting data from the world and interacting with the world.
While the paper is conceptual and forward -looking, it has a direct impact on corporations that want to accumulate with and on future AI agents and systems.
Both Silver and Sutton are experienced scientists with successful story that has made precise predictions concerning the way forward for AI. The validity predictions could be observed directly in today's most advanced AI systems. In 2019 Sutton, a pioneer in reinforcement learning, wrote the famous essay “The bitter lesson“, In which he argues that the best long-term progress within the AI is consistently created from the usage of large calculation with general search and learning methods as an alternative of mainly depend on the inclusion of complex domestic knowledge derived from humans.
David Silver, senior scientist at Deepmind, was a vital contribution to Alphago, Alphazero and Alphastar. He was also the co-author of a paper in 2021, who claimed that there can be amplification learning and a well-designed reward signal to create very advanced AI systems.
The most advanced large -speaking models (LLMS) use these two concepts. The wave of latest LLMS, which have conquered the AI scene since GPT-3, has mainly been based on the scaling of calculation and data on the internationalization of huge amounts of data. The latest wave of argumentation models akin to Deepseek-R1 has shown that learning and a straightforward reward signal are sufficient for learning Complex arguments.
What is the era of experience?
The “era of experience” builds on the identical concepts that Sutton and Silver have discussed lately and adapts it to recent progress in AI. The authors argue that the “pace of progress, which is driven exclusively by monitored learning from human data, has been shown to decelerate and signal the necessity for a brand new approach.”
And this approach requires a brand new data source that have to be generated in such a way that the agent repeatedly improves. “This could be achieved by enabling agents to continuously learn from their very own experiences, ie data that’s generated by the agent that interact with its surroundings,” write Sutton and Silver. They argue that “experience finally becomes the dominant medium of improvement and ultimately puts the extent of human data utilized in today's systems”.
According to the authors, future AI systems will break through along with learning from their very own experience data “The limits of human AI systems in 4 dimensions:
- Streams: Instead of working over separate episodes, AI agents will “have their very own stream of experience through an extended time scale, which progresses like humans”. This enables agents to plan long -term goals and to adapt to latest behavior patterns over time. In AI systems which have very long context windows and memory architectures, we see shimmer that constantly updated based on user interactions.
- Actions and observations: Instead of concentrating on human-privileged actions and observations, agents within the era of experience in the actual world will act autonomously. Examples of this are agent systems that may interact with external applications and resources via tools akin to computer use and model context protocol (MCP).
- Rewards: Current reinforcement learning systems are mainly based on people's reward functions. In the long run, AI agents should find a way to design their very own dynamic reward functions that adapt over time and match the user preferences with real signals that come from the agent's actions and observations on the earth. We see early versions of self -designing rewards with systems akin to Nvidia's Dreureka.
- Planning and argument: Current argumentation models have been developed to mimic the human pondering process. The authors argue that “more efficient monetary mechanisms definitely exist and use non -human languages that may use symbolic, distributed, continuous or differentiable calculations”. AI agents should cope with the world, observe and use data to validate and update their argumentation process and to develop a world model.
The idea of AI agents who adapt to their surroundings by learning reinforcement shouldn’t be latest. Before that, nonetheless, these agents were limited to very limited environments akin to board games. Nowadays, agents who can interact with complex environments (e.g. AI computer use) and progress when learning reinforcement will overcome these restrictions and produce concerning the transition to the era of experience.
What does it mean for the corporate?
In Sutton and Silvers paper, an statement that has a very important impact on real applications is: “The agent can use” human-friendly “actions and observations akin to user interfaces that naturally facilitate communication and cooperation and work with the user. The agent also can require” machine-frifted “actions, perform the code and enable API to react to the agent autonomy freight.
The era of experience implies that developers should construct their applications not just for humans, but additionally with AI agents. Machine -friendly actions require the creation of secure and accessible APIs, which could be easily accessed directly or via interfaces akin to MCP. It also means creating agents that could be invented by protocols akin to Google Agent2Agent. You also should design your APIs and agents interfaces to make sure each access to actions and observations. In this fashion, agents could make it possible to steadily prevent and study their interactions with their applications.
If the vision that Sutton and Silver presented in point of fact will turn out to be reality, it’s going to soon make billions of agents on the Internet (and shortly within the physical world) to do tasks. Your behaviors and wishes differ greatly from human users and developers, and an agent-friendly approach to interact with their application improves their ability to make use of future AI systems (and likewise prevent the damage they could cause).
“By constructing on the fundamentals of RL and adapting your core principles to the challenges of this latest era, we will exhaust the complete potential of autonomous learning and pave the approach to really superhuman intelligence,” write Sutton and Silver.
Deepmind refused to make additional comments for the story.
