Shrinivas Ramasubramanian(@stablegradients):Is RL optimizing the right objective? 🤔 Should we maximize mean reward? Best-of-k? Which k? Standard RL pulls on the mean and often the distribution collapses to a spike. The tail dies 🥲 We introduce Tail-Likelihood Reinforcement Learning (TailRL). It maximizes the mean reward while simultaneously maximizing coverage over high reward outputs. 🧵 1/n

Shrinivas Ramasubramanian

@stablegradients

MS @CarnegieMellon Undergrad @iitbombay

加入 September 2021

1.8K 正在關注 404 粉絲

Shrinivas Ramasubramanian@stablegradients

2026.09.02 18:03

Is RL optimizing the right objective? 🤔 Should we maximize mean reward? Best-of-k? Which k? Standard RL pulls on the mean and often the distribution collapses to a spike. The tail dies 🥲 We introduce Tail-Likelihood Reinforcement Learning (TailRL). It maximizes the mean reward while simultaneously maximizing coverage over high reward outputs. 🧵 1/n