Cross-Environment Cooperation for Zero-Shot Multi-Agent Coordination

We discuss Cross-Environment Cooperation (CEC), a novel reinforcement learning paradigm for training agents capable of zero-shot multi-agent coordination (ZSC). Unlike prior work focusing on single-task training or enhancing partner diversity, CEC trains a single agent through self-play across a vast distribution of procedurally generated environments. The authors demonstrate that this approach fosters the development of general cooperative skills and norms, enabling effective collaboration with new partners in unseen scenarios, outperforming methods relying on partner diversity. Through simulations and human studies in the Overcooked environment, CEC agents exhibited strong generalization to novel partners and tasks, even surpassing state-of-the-art methods in subjective human evaluations of cooperative ability, suggesting a promising path toward more adaptable and human-compatible AI.