Training Agents Inside of Scalable World Models

This paper introduces Dreamer 4, a new world model designed to solve complex control tasks, particularly the Minecraft diamond challenge, purely through offline imagination training without direct environment interaction. The core innovation lies in its architecture, which uses an efficient block-causal transformer and a shortcut forcing objective to achieve high prediction accuracy of game mechanics and real-time interactive inference speed. Experiments demonstrate that Dreamer 4 significantly outperforms previous state-of-the-art offline agents in Minecraft, achieving success rates of obtaining diamonds, while also showcasing superior performance in simulating complex object interactions compared to earlier world models like Oasis and Lucid. The research highlights the potential of highly capable world models for offline reinforcement learning in challenging, embodied environments.