π0.5: Generalization in Robotic Manipulation via Diverse Data

This paper introduces π0.5, a novel vision-language-action model designed for open-world generalization in robotic tasks. This model leverages knowledge from diverse sources, including other robots, web data, and language instructions, to enable a mobile manipulator to perform complex cleaning tasks in unseen home environments. π0.5 employs a unified architecture for both high-level task planning and low-level action execution, using a combination of discrete and continuous action representations for efficient training and inference. Experimental results demonstrate robust generalization to new homes and objects, highlighting the importance of cross-embodiment learning and the model's high-level reasoning capabilities.