This paper studies mechanistic explanation for the paradox that **Reinforcement Learning with Verifiable Rewards (RLVR)** reliably improves large language model reasoning while making only minimal, sparse changes to parameters. The authors introduce the **Three-Gate Theory**, arguing that sparse updates are a surface artifact of a **model-conditioned optimization bias**. **Gate I (KL Anchor)** constrains each update, while **Gate II (Model Geometry)** steers the updates off the principal, high-c...
Nov 23, 2025•12 min
This academic paper, introduces "Just image Transformers" (JiT), a novel approach to denoising diffusion models that advocates for directly predicting clean data (**x-prediction**) rather than predicting noise or a noised quantity. The authors argue this shift is critical based on the **manifold assumption**, which posits that clean data lies on a low-dimensional manifold while noise is inherently off-manifold. Experiments, including a toy model and high-resolution ImageNet generation using plai...
Nov 23, 2025•15 min
This paper introduces the **Prompt Duel Optimizer (PDO)**, a novel, sample-efficient framework for **label-free prompt optimization** in large language models (LLMs). Recognizing that LLM performance is highly sensitive to input prompts and that collecting ground-truth labels is costly, PDO frames the optimization challenge as a **dueling bandit problem** where an LLM acts as a judge, providing noisy but usable **pairwise preference feedback**. PDO's effectiveness stems from two core components:...
Nov 22, 2025•13 min
This paper introduces a novel technique called **Generative Adversarial Distillation (GAD)** for knowledge transfer from a large, proprietary teacher language model (LLM), such as GPT-5-Chat, to a smaller student LLM in a **black-box setting**. Black-box distillation is necessary when the student only has access to the teacher’s final text outputs, not its internal parameters or probabilities. GAD frames the distillation process as a **minimax game** similar to a Generative Adversarial Network (...
Nov 20, 2025•14 min
This research paper introduces how we can reliably complete complex, multi-step tasks with zero errors. The core concept is **extreme decomposition** of a task into minimal subtasks handled by focused "microagents," which overcomes the inherent, escalating error rate of monolithic LLMs over long horizons. This modular approach integrates an **efficient error correction** mechanism—specifically, a first-to-ahead-by-$k$ voting scheme—and a process of **red-flagging** unreliable outputs, drasticall...
Nov 20, 2025•15 min
This paper studies an inference-time optimization technique designed to reduce the high computational cost of reasoning-optimized large language models (LLMs), which generate long chains of thought. LLMs' self-attention mechanism typically scales quadratically with sequence length, making long reasoning chains prohibitively expensive. RWR addresses this by exploiting the redundancy in intermediate reasoning steps, maintaining only two strategically chosen parts of the key-value (KV) cache: the f...
Nov 19, 2025•13 min
This research paper provides a novel approach for sample-efficient parametric learning in large language models (LLMs) using natural language feedback, addressing the transience of traditional in-context learning (ICL) and the data inefficiency of standard fine-tuning. The authors propose a simple three-step method: obtaining natural language feedback, sampling a generation conditioned on that feedback, and then performing supervised fine-tuning (SFT) on the new generation with the feedback remo...
Nov 19, 2025•11 min
This paper discusses how to design an evaluation-efficient self-improving AI systems that beats GEPA for societal and business problems like ad optimization where the cost of generating new content is low but evaluation is expensive. It argues that traditional human-driven optimization is slow and bottlenecked by content generation, but generative AI has shifted the bottleneck to efficient evaluation and prompt refinement. T-BoN BO addresses key challenges—lack of numerical gradients in language...
Nov 18, 2025•34 min
This whitepaper by Google titled **"Context Engineering: Sessions & Memory,"** authored by Kimberly Milam and Antonio Gulli in November 2025, which provides a detailed guide to building stateful, intelligent Large Language Model (LLM) agents. The document defines **Context Engineering** as the process of dynamically managing information within an LLM's context window, emphasizing two core, interconnected components: **Sessions** and **Memory**. **Sessions** manage the immediate, chronologica...
Nov 16, 2025•14 min
This paper introduces **Asynchronous Thinking (AsyncThink)**, a novel paradigm for large language model (LLM) reasoning designed to enable **agentic organization** and collaborative problem-solving. AsyncThink employs an **organizer-worker thinking protocol** where an LLM acts as an organizer that dynamically structures concurrent processes using **Fork and Join actions**, while workers execute sub-queries. The authors compare AsyncThink favorably to traditional sequential and parallel thinking ...
Nov 15, 2025•11 min
This paper by OpenAI discusses a new approach to **neural network interpretability** through the use of **sparse circuits**. The authors explain that understanding the behavior of complex, hard-to-decipher neural networks is critical for safety and oversight as AI systems become more capable. They distinguish their work on **mechanistic interpretability**, which seeks to fully reverse-engineer computations, from other methods like chain-of-thought interpretability. The core of their research inv...
Nov 14, 2025•13 min
The academic paper introduces Supervised Reinforcement Learning (SRL), a novel training framework for Large Language Models (LLMs) developed by researchers from Google Cloud AI Research and UCLA to address the difficulty of multi-step reasoning. SRL reformulates problem-solving as a sequence of logical actions, providing dense, step-wise rewards based on the similarity between the model's generated actions and expert trajectories, which contrasts with the sparser, final-outcome rewards used in R...
Nov 14, 2025•11 min
This research paper introduces Multi-Agent Evolve (MAE), a novel reinforcement learning framework designed to enable large language models (LLMs) to self-improve their general reasoning abilities without relying on human-curated datasets or verifiable external rewards. MAE accomplishes this through a system where a single LLM is instantiated into three interacting roles—a Proposer that creates challenging questions, a Solver that attempts to answer them, and a Judge that evaluates both the quest...
Nov 14, 2025•10 min
This paper introduces a novel self-supervised learning framework designed to resolve the pervasive issue of representation collapse in existing Joint-Embedding Predictive Architectures (JEPAs). It establishes a theoretical foundation by proving that an isotropic Gaussian distribution is the optimal embedding distribution for minimizing the worst-case risk across various downstream tasks. To enforce this optimal distribution, the paper proposes SIGReg (Sketched Isotropic Gaussian Regularization),...
Nov 14, 2025•13 min
This paper introduce a new meta-benchmark designed to evaluate large language models' (LLMs) ability to perform **interactive preference discovery** and response personalization through conversation. The framework converts existing benchmarks into interactive tasks by assigning **psychologically-grounded personas** with hidden preferences to be discovered by the AI. Evaluation of numerous frontier models showed that simply attempting personalization often **degraded performance** compared to gen...
Nov 12, 2025•15 min
The academic paper investigates the efficiency of Large Language Model (LLM) pre-training by quantifying the amount of knowledge left unextracted from training datasets. The authors demonstrate that employing retrieval-augmented generation (RAG) at test time, which involves reusing the pre-training data, leads to significant accuracy improvements across benchmarks like MMLU, Math-500, and SimpleQA, even after decontamination efforts. The study establishes that retrieval acts as a compute multipl...
Nov 10, 2025•16 min
The academic paper proposes **DreamGym**, a novel, unified framework for scaling agent learning using reinforcement learning (RL) by synthesizing diverse experiences instead of relying on costly real-environment rollouts. The core of this system is a **reasoning-based experience model** that abstracts environment dynamics into a textual space, enabling the generation of consistent state transitions and reward signals through explicit reasoning. DreamGym integrates an **experience replay buffer**...
Nov 09, 2025•17 min
This paper introduces **Continuous Autoregressive Language Models (CALM)**, a new paradigm designed to overcome the efficiency limitations of conventional, token-by-token generation in Large Language Models (LLMs). CALM achieves significant computational savings by employing a robust **autoencoder** to compress a chunk of $K$ discrete tokens into a single, high-fidelity continuous vector, thereby reducing the number of sequential generation steps by a factor of $K$. This shift necessitates a com...
Nov 08, 2025•16 min
This position paper argues for a new epistemic theory of agents that views internal reasoning and external actions as equivalent epistemic tools for acquiring knowledge. The core argument is that for an agent to achieve optimal and efficient behavior, its tool use decision boundary must be aligned with its knowledge boundary, meaning it should only resort to external tools when necessary knowledge is unavailable internally. The paper formalizes this concept by defining tools, agents, and optimal...
Nov 07, 2025•20 min
This paper introduces Nested Learning (NL), a new paradigm that addresses fundamental challenges in AI self-improvement, continual learning and memory for models like Large Language Models (LLMs). NL suggests that existing deep learning methods compress their "context flow" and explains how in-context learning emerges in large models. The authors propose the HOPE architecture, a self-referential learning module with a Continuum Memory System (CMS), which is built on the NL insights that traditio...
Nov 05, 2025•13 min
This paper introduce the GST-UNet (G-computation Spatio-Temporal UNet), a novel neural framework designed for causal inference using spatiotemporal observational data, particularly when analyzing a single observed trajectory. This framework integrates a U-Net encoder with ConvLSTM and attention mechanisms to learn spatiotemporal dependencies and explicitly address challenges like interference, spatial confounding, and time-varying confounding. The core contribution is coupling this architecture ...
Nov 05, 2025•18 min
This paper introduces a research paper focused on improving **Large Language Model (LLM) performance on tasks requiring long-term conversational memory**. The authors address limitations in existing evaluation methods by presenting a new framework that automatically generates **long, coherent conversations up to 10 million tokens** and **BEAM**, a benchmark dataset with 100 dialogues and 2,000 probing questions designed to test ten distinct memory abilities, including contradiction resolution an...
Nov 04, 2025•15 min
This paper introduces Agentic Economic Modeling (AEM), a rigorous framework proposed by superstar social scientists that leverages Large Language Models (LLMs) to reliably simulate economic decisions and generate counterfactual data for econometric inference. The core innovation is a three-stage pipeline—Generation, Correction, and Inference—designed to overcome the systematic biases found in raw LLM outputs by anchoring them to small samples of real-world human data. Specifically, AEM employs a...
Nov 03, 2025•14 min
This research by anthropic investigates the existence of **functional introspective awareness** in large language models (LLMs), specifically focusing on Anthropic's Claude models. The core methodology involves using **concept injection**, where researchers manipulate a model's internal activations with representations of specific concepts to see if the model can accurately **report on these altered internal states**. Experiments demonstrate that models can, at times, notice injected "thoughts,"...
Nov 03, 2025•16 min
This paper investigates whether large reasoning models can sustain self-training using Reinforcement Learning (RL), specifically employing majority voting as a self-feedback mechanism, termed Self-Rewarded Training (SRT). The research demonstrates that this basic approach initially improves the model's reasoning performance and enhances the quality of its self-generated feedback, achieving performance comparable to RL with ground-truth supervision. However, a critical limitation is identified: p...
Nov 01, 2025•12 min
This paper proposes a method for transforming a general-purpose large language model agent into a domain-specific expert. This system achieves specialization by systematically generating, abstracting, and curating reusable Model Context Protocol (MCP) tools from successful task executions, which are then stored in an MCP Box. At inference time, a Retrieval-Augmented Generation (RAG) mechanism selects the most contextually relevant tools from the box, thereby enhancing the agent's problem-solving...
Nov 01, 2025•16 min
The academic paper proposes a novel framework called Test-Time Self-Improvement (TT-SI) for training Large Language Model (LLM) agents more efficiently by adapting them on-the-fly during inference. This new paradigm is motivated by the high cost and inefficiency of traditional large-scale fine-tuning, which often involves redundant data. TT-SI operates in three steps: Self-Awareness identifies uncertain test instances, Self-Augmentation generates tailored training samples for those instances, an...
Oct 30, 2025•19 min
This paper recasts the complex offline RL problem as standard supervised fine-tuning (SFT) techniques that directly optimizes for rewards. Authors show that their method empirically outperforms state-of-the-art baselines such as SFT and Direct Preference Optimization (DPO) across various QA benchmarks. The experiments focus on fixed-horizon conversational policies where the agent either reasons about answers or asks clarifying questions, demonstrating that directly optimizing the reward signal l...
Oct 30, 2025•15 min
The academic paper argues that decoder-only Transformer language models, such as GPTs, are almost surely injective, meaning that distinct input prompts map to distinct internal hidden states, preserving input information without loss. This contrasts with the common assumption that non-linear components make models lossy. The authors mathematically prove that this injectivity is a structural property established at initialization and preserved during standard training procedures like gradient des...
Oct 30, 2025•12 min
This paper introduces **ReasoningBank**, a novel memory framework designed to enhance Large Language Model (LLM) agents by distilling and structuring reasoning patterns from both successful and failed task trajectories. Traditional memory systems typically overlook failure experiences and lack the ability to abstract high-level reasoning, a limitation ReasoningBank addresses by creating **structured memory items** (title, description, content) that capture transferable insights. Furthermore, the...
Oct 29, 2025•15 min
