Sample-Efficient Parametric Learning from Natural Language

This research paper provides a novel approach for sample-efficient parametric learning in large language models (LLMs) using natural language feedback, addressing the transience of traditional in-context learning (ICL) and the data inefficiency of standard fine-tuning. The authors propose a simple three-step method: obtaining natural language feedback, sampling a generation conditioned on that feedback, and then performing supervised fine-tuning (SFT) on the new generation with the feedback removed from the prompt, which forces the model to internalize the instruction into its weights. This technique is evaluated against ICL and SFT baselines across both factual rule-learning (DFAs) and stylistic adaptation tasks, demonstrating superior performance with limited data budgets. However, preliminary results on iterative learning show that while small sequential updates are possible, the compounding of feedback quickly leads to catastrophic forgetting and interference.