In RL for streaming video, where a model keeps deciding whether to answer now or keep waiting, the way credit is handed out turn by turn has been too coarse 🎥
Turns Matter: Advancing Streaming Video RL via Turn-Level Credit
📌 Overview
The paper proposes TurnWise, a GRPO extension that assigns credit per turn for streaming video understanding. It needs no critic and no extra rollouts.
🚧 The problem
・Prior work gives every turn the same trajectory-level advantage
・A redundant reply inherits credit from an earlier success, and a useful follow-up is penalized for an earlier failure
・Token-mean loss weights long replies more heavily than short "wait" decisions
🔧 Method
・Keep the actions up to a turn, replace later ones with silence (NO REPLY), and score the result to separate each turn's contribution
・Average the loss within each turn, then across turns, so replies and waits get equal weight
🧪 Results
・On ProactiveVQA, Qwen2.5-VL-3B reaches 43.56 average PAUC, 3.96 points above the strongest alternative (39.60)
・Qwen3-VL-4B reaches 42.44, 6.26 points above ThinkStream (36.18)
・Repetitive replies drop 61.5% compared with credit-only training
Evaluation covers two models and three benchmarks, and variance across seeds is not reported.
#
VideoUnderstanding# #
ReinforcementLearning#