Scaling Agent Learning via Experience Synthesis

The academic paper proposes **DreamGym**, a novel, unified framework for scaling agent learning using reinforcement learning (RL) by synthesizing diverse experiences instead of relying on costly real-environment rollouts. The core of this system is a **reasoning-based experience model** that abstracts environment dynamics into a textual space, enabling the generation of consistent state transitions and reward signals through explicit reasoning. DreamGym integrates an **experience replay buffer** to enrich synthetic data and a **curriculum task generator** that creates progressively challenging problems based on reward entropy, thereby addressing common RL challenges like sparse rewards and task scarcity. Experimental results across diverse environments, including those not traditionally "RL-ready" like WebArena, demonstrate that DreamGym substantially **improves RL training efficiency** and yields significant performance gains in both purely synthetic settings and sim-to-real transfer scenarios.