Dual Goal Representations

This paper discusses dual goal representations for goal-conditioned reinforcement learning (GCRL), a novel method for encoding a state based on its temporal distance relation to all other states within an environment. The authors theoretically establish that this representation is sufficient for recovering an optimal goal-reaching policy and is invariant to extraneous noise within the state observations. Building on this theory, they propose a practical implementation using an inner product parameterization and offline value learning, demonstrating that this approach consistently improves goal-reaching performance across a suite of robotic navigation and manipulation tasks, outperforming existing representation learning methods. The overall aim is to enhance the efficiency and generalization capability of GCRL agents by providing a robust and structured goal representation.