In-Context Learning for Pure Exploration

This paper introduces In-Context Pure Exploration (ICPE), a Transformer-based architecture designed to efficiently solve active sequential hypothesis testing problems, also known as pure exploration. ICPE meta-trains a model to map observation histories to actions and predicted hypotheses, enabling in-context learning to actively gather data and infer the correct hypothesis on new tasks without requiring parameter updates. The paper frames this as splitting the process into a supervised inference network and an RL-trained policy network that maximizes information gain. The system is evaluated across various benchmarks, including Best-Arm Identification (BAI) in multi-armed bandits and generalized search problems like pixel sampling, showing performance competitive with adaptive baselines while effectively discovering structured exploration strategies.