GOAT: Generative Adversarial Training for Human-AI Coordination

This paper explores improving how AI agents coordinate with humans in cooperative tasks by addressing the challenge of training agents on the vast diversity of human behaviors. The authors introduce a new method called GOAT (Generative Online Adversarial Training), which combines a pre-trained generative model of cooperative strategies with adversarial training. This framework uses an Adversary agent to find challenging but realistic human-like partners (simulated by the generative model) that expose the learning Cooperator agent's weaknesses. By optimizing a regret-based objective, GOAT encourages the Cooperator to learn robust coordination skills and achieve state-of-the-art performance when collaborating with novel human partners in the Overcooked benchmark.