The paper "MSL: Not All Tokens Are What You Need for Tuning LLM as a Recommender" identifies limitations of using the standard language modeling loss for fine-tuning large language models as recommendation systems. Specifically, it points out the divergence from recommendation goals and the misleading negative signals arising from treating all non-positive item descriptions as negative. To overcome these issues, the authors introduce Masked Softmax Loss (MSL) , which selectively masks invalid to...
Apr 11, 2025•16 min
Tkachenko, Jedidi, and Ansari's paper addresses the challenge of lengthy consumer questionnaires, which can increase costs and decrease response quality. They propose a novel solution using self-supervised deep reinforcement learning to rank questions by their information value. Their method outperforms traditional question ranking and competes with unordered subset selection techniques. The findings reveal that consumer data often contains redundancy, allowing for accurate reconstruction from s...
Apr 11, 2025•21 min
2504.04204 : Adaptive Elicitation of Latent Information Using Natural Language This research paper introduces a novel framework for adaptive information elicitation using natural language, addressing the challenge of understanding latent entities that cannot be directly observed. This framework employs meta-learned language models to predict future observations and quantify uncertainty, enabling the strategic selection of the most informative questions to reduce this uncertainty. The authors pro...
Apr 10, 2025•17 min
They discuss using large language models (LLMs) to generate synthetic human personas for simulations across various fields. The authors highlight that while LLM-generated personas offer a scalable and cost-effective alternative to traditional data collection, current methods lack rigor and introduce significant biases. Through experiments like predicting election outcomes and general opinion surveys, the study reveals that these biases can lead to considerable deviations from real-world data. Th...
Apr 10, 2025•18 min
This paper introduces AutoTools , a novel framework designed to empower large language models (LLMs) to function as automated tool agents. This system enables LLMs to automatically transform tool documentation into callable functions and subsequently integrate these functions into executable programs to solve practical tasks. The authors identify limitations in previous manual approaches for tool utilization by LLMs and propose AutoTools as a more scalable and flexible solution. Furthermore, the...
Apr 10, 2025•20 min
This survey examines the burgeoning field of tool learning with large language models (LLMs) , a paradigm where LLMs enhance their capabilities by using external tools to solve complex problems. The authors systematically explore why tool learning is beneficial , detailing advantages like improved knowledge acquisition and robustness, and how it is implemented , outlining a four-stage workflow of task planning, tool selection, tool calling, and response generation. The paper also provides an ove...
Apr 10, 2025•23 min
This research paper investigates why reinforcement learning (RL) often improves the fine-tuning of large language models compared to direct maximum likelihood estimation (MLE). The authors explore the theoretical equivalence of these methods under certain conditions, demonstrating that they should ideally yield similar results. However, empirical evidence shows RL-based fine-tuning, particularly with a reward model, frequently outperforms offline MLE approaches. To resolve this discrepancy, the ...
Apr 08, 2025•24 min
This paper introduces ATLAS, a novel method for enhancing large language model agents by selectively fine-tuning them on critical steps identified within expert action sequences. This approach, which uses another LLM to pinpoint crucial moments like planning, key observations, significant actions, and self-correction , aims to overcome limitations of traditional full-trajectory imitation learning, such as expert bias and poor generalization . By concentrating training on roughly 30% of the exper...
Apr 08, 2025•20 min
This research paper introduces Chain of Draft (CoD) , a novel prompting strategy for Large Language Models (LLMs) designed to mimic efficient human reasoning by generating concise intermediate thoughts. Unlike the verbose Chain-of-Thought (CoT) prompting, CoD encourages LLMs to produce minimal yet informative outputs at each step, leading to comparable or superior accuracy with significantly reduced token usage and latency across various reasoning tasks. The authors provide empirical evidence us...
Apr 08, 2025•19 min
This research paper introduces Meta Plan Optimization (MPO), a new framework to improve how large language model agents plan for tasks. MPO uses high-level, general instructions called meta plans to guide the agents, helping them avoid planning errors and the need for retraining on each new task. The framework includes a meta planner that generates these guiding plans and is refined based on feedback from the agent's task performance. Experiments on household and science tasks demonstrate that M...
Apr 08, 2025•19 min
This research paper introduces Length Controlled Policy Optimization (LCPO) , a reinforcement learning technique that enables reasoning language models to control the length of their generated thought processes based on user-specified constraints. By training a model called L1 with LCPO, the authors demonstrate precise management of reasoning length, allowing for a trade-off between computational cost and accuracy on various tasks. Notably, L1 outperforms prior length control methods and exhibit...
Apr 08, 2025•17 min
This introduces WikiBigEdit , a new large-scale benchmark for evaluating how well large language models can continuously update their factual knowledge over time, using real-world edits from Wikidata. The authors find that existing knowledge editing techniques struggle with the scale and sequential nature of these real-world updates . In contrast, simpler methods like retrieval augmentation and continual finetuning with model merging prove more effective for incorporating and retaining a large v...
Apr 08, 2025•20 min
This research paper introduces PLAN-AND-ACT , a new framework designed to enhance the ability of language agents to handle complex, long-horizon tasks. This system separates the process into two modules: a PLANNER , which generates high-level, structured plans, and an EXECUTOR , which translates these plans into specific actions within an environment. To effectively train the PLANNER , the authors present a novel method for synthetic data generation , leveraging large language models to annotate...
Apr 08, 2025•15 min
This research paper introduces SEARCH-R1 , a novel framework that enhances large language models by enabling them to learn to effectively use search engines through reinforcement learning. This approach allows LLMs to autonomously generate search queries and leverage retrieved information during their reasoning process, improving performance on question-answering tasks . Unlike traditional methods, SEARCH-R1 optimizes the interaction with search in an end-to-end manner , using techniques like re...
Apr 08, 2025•24 min
This Handbook of Industrial Organization provides a comprehensive overview of the theory of the firm , moving beyond traditional market analysis to explore internal firm behavior and organization . It examines the boundaries of firms , issues of capital structure , the impact of separated ownership and control , and the complexities of internal hierarchies . The authors synthesize various theoretical perspectives, including incomplete contracts, information economics, agency theory, and reputati...
Apr 08, 2025•25 min
This paper by Robert Gibbons explores four fundamental theories of the firm , aiming to clarify their core tenets, distinctions, and potential for integration. It examines the rent-seeking, property-rights, incentive-system, and adaptation theories , tracing their intellectual origins and highlighting key contributions from prominent scholars. The essay formally models three of these theories and proposes an integrative framework to better understand their relationships, arguing for a unified pe...
Apr 08, 2025•32 min
arXiv:2401.17464 Efficient Tool Use with Chain-of-Abstraction Reasoning Silin Gao , Jane Dwivedi-Yu , Ping Yu , Xiaoqing Ellen Tan , Ramakanth Pasunuru , Olga Golovneva , Koustuv Sinha , Asli Celikyilmaz , Antoine Bosselut , Tianlu Wang This research paper introduces Chain-of-Abstraction (CoA) , a novel method designed to enhance the ability of large language models (LLMs) to effectively utilize external tools for complex, multi-step reasoning. CoA trains LLMs to first generate abstract reasonin...
Apr 06, 2025•21 min
We discuss CodeTool, a novel framework that enhances how large language models utilize external tools by generating and supervising code execution step-by-step. This approach uses process rewards: an "On-the-spot Reward" for immediate code correctness and a "Latent Reward" to guide towards effective problem-solving paths, with the latter estimated by a trained model. CodeTool leverages the verifiable nature of code for reliable feedback at each stage, overcoming limitations of text or JSON-based...
Apr 06, 2025•17 min
This survey systematically investigates how to evaluate large language model-based agents designed for multi-turn conversations. The authors reviewed nearly 250 academic papers to understand current evaluation practices, establishing a structured framework with two key taxonomies. One taxonomy defines what to evaluate, encompassing aspects like task completion, response quality, user experience, memory, and planning. The second taxonomy details how to evaluate, categorizing methodologies into an...
Apr 06, 2025•29 min
This paper explores the epistemic alignment problem in user interactions with Large Language Models (LLMs), highlighting the mismatch between user knowledge preferences and the limited ways to express them. The authors propose the Epistemic Alignment Framework , consisting of ten challenges derived from epistemology, to bridge this gap and create a shared vocabulary. Through an analysis of user-shared prompts and platform policies of OpenAI and Anthropic, the paper demonstrates that while users ...
Apr 06, 2025•17 min
We discuss Model Context Protocol (MCP) , positioning it as a standardized, open-source protocol championed by Anthropic to unify how large language models (LLMs) interact with external APIs, akin to a USB-C for AI. It explains that while previous methods for LLM integration existed, MCP offers a consistent interface using JSON-RPC , demonstrated through examples like tools/list and tools/call, facilitating easier development of both servers and clients. The article further clarifies that MCP it...
Apr 06, 2025•29 min
We investigate how artificial intelligence (AI) impacts competitive advantage , employing a resource-based view . Through a study of chess tournaments with human, AI, and hybrid players, the authors find that AI adoption leads to both the obsolescence of traditional human skills and the emergence of new advantages stemming from human-machine collaboration. This substitution and complementation dynamic redefines competitive landscapes , requiring managers to cultivate new capabilities in an AI-dr...
Apr 05, 2025•23 min
This paper explores how to improve the effectiveness of reward modeling (RM) for large language models (LLMs) by utilizing more computational resources during inference. The authors focus on generalist RM, aiming for accurate reward signals across diverse queries, not just verifiable ones. To achieve this, they introduce Self-Principled Critique Tuning (SPCT), a novel learning method that enables reward models to generate their own guiding principles and critiques. This approach results in DeepS...
Apr 04, 2025•22 min
This research addresses the challenge of efficient exploration in linear bandit problems, aiming to identify the optimal action with minimal measurements. Existing optimal methods often involve computationally intensive steps like projections or maintaining subsets of actions. The paper introduces a novel algorithm, PEPS, which achieves asymptotic optimality using only sampling and argmax oracles, similar to the simpler Thompson Sampling. Unlike Thompson Sampling, which is suboptimal for pure ex...
Apr 04, 2025•26 min
Susan Athey's presidential address examines the expanding role of economists as designers in the data-driven innovation process for digital products, particularly those aimed at social impact. The paper outlines six key design roles for economists, such as product and market design, and six cross-cutting challenges they face, including navigating trade-offs and addressing long-term equilibrium effects. Through numerous case studies across education, agriculture, labor markets, and online platfor...
Apr 04, 2025•45 min
This paper investigates the emergent reasoning capabilities of large language models (LLMs) . Through a detailed study of the open-source LLM Llama3-70B, the authors uncover evidence for an emergent three-stage symbolic architecture that supports abstract rule induction. This architecture involves symbol abstraction , symbolic induction , and retrieval mechanisms implemented by specific attention heads within the model. The findings suggest that LLMs may achieve abstract reasoning not merely thr...
Apr 03, 2025•17 min
This research paper introduces a statistical framework for understanding and improving inference-time alignment of language models. The paper examines the limitations of the widely used "Best-of-N" sampling method, identifying its potential for reward overoptimization. To address these shortcomings, the authors propose a novel algorithm, \mainalg, that incorporates \chis-regularization at inference time using a rejection sampling scheme. Theoretical analysis demonstrates that \mainalg achieves o...
Apr 03, 2025•15 min
This research paper introduces a novel active learning method called SHARP (SHarpe Ratio-based Active Requested Preferences) and its weighted variant W-SHARP for efficiently collecting human feedback to train large language models using Direct Preference Optimization (DPO). This method uses the Sharpe ratio to assess the potential impact and risk associated with labeling different prompt-response pairs, aiming to select the most informative data points for annotation. The paper derives a computa...
Apr 03, 2025•19 min
This research paper introduces a novel approach called Active In-context Prompt Design (AICL) for improving the performance of large language models (LLMs) through adaptive prompt tuning. The paper addresses the challenge of selecting the most informative examples to include in an LLM's prompt at inference time to optimize its predictions on a set of test queries. To achieve this, the authors propose two active learning algorithms: G-Optimal design (\go) , inspired by optimal experimental design...
Apr 03, 2025•16 min
This paper introduces CoT-VLA , a novel method for vision-language-action models (VLAs) that incorporates visual chain-of-thought (CoT) reasoning . Unlike traditional VLAs that directly map inputs to actions, CoT-VLA first predicts future image frames as visual goals before generating action sequences to achieve them. This approach aims to enhance reasoning capabilities for complex manipulation tasks by leveraging both robot demonstrations and unlabeled video data. The paper details the model's ...
Apr 03, 2025•20 min
