Policy Learning with a Natural Language Action Space: A Causal Approach

This academic paper proposes a new causal framework for learning optimal strategies in natural language tasks that involve multiple steps, where the final result is only known at the end. Unlike methods requiring extensive data and multiple models, their approach utilizes Q-learning with a single model to estimate multi-stage decision processes. By performing gradient ascent on language embeddings, they optimize the process, coupled with a decoding strategy to convert optimized embeddings back into understandable language. Tested on scenarios like improving mental health interventions and countering hate speech, their method outperforms existing techniques, showing notable gains in achieving desired outcomes while maintaining fluency and content, which human evaluations also support.