Bayesian Optimization in Language space: An Eval-Efficient AI Self-Improvement Framework

This paper discusses how to design an evaluation-efficient self-improving AI systems that beats GEPA for societal and business problems like ad optimization where the cost of generating new content is low but evaluation is expensive. It argues that traditional human-driven optimization is slow and bottlenecked by content generation, but generative AI has shifted the bottleneck to efficient evaluation and prompt refinement. T-BoN BO addresses key challenges—lack of numerical gradients in language space, and the need to balance exploration and exploitation—by adapting classic **Bayesian Optimization (BO)** principles. Theoretically, the paper proves that T-BoN BO, which uses textual gradients and a Best-of-N selection rule, **emulates gradient-based Upper Confidence Bound (UCB) BO** and inherits its theoretical guarantees for evaluation efficiency. Empirical results in digital marketing scenarios demonstrate that T-BoN BO significantly **outperforms state-of-the-art baselines** in achieving performance gains under a fixed evaluation budget.