Learning to Explore: An In-Context Learning Approach for Pure Exploration

The academic paper introduces In-Context Pure Exploration (ICPE), a novel deep learning framework that utilizes Transformers and combines supervised learning with reinforcement learning to autonomously discover efficient data exploration strategies. Unlike traditional methods requiring explicit model assumptions, ICPE learns adaptive sampling policies directly from experience, enabling it to identify correct hypotheses in sequential decision-making problems like Best Arm Identification (BAI). The authors demonstrate ICPE's robust performance across various bandit problems, including those with hidden information, and in semi-synthetic pixel sampling for image classification, showcasing its potential to match or outperform optimal instance-dependent algorithms without complex manual design. The research also explores the theoretical underpinnings of exploration and highlights how ICPE can meta-learn complex search strategies, such as a probabilistic version of binary search.