TL;DR Can an AI watch a video stream and help before you ask? This paper tackles implicit and changing user intent with a benchmark, a model and an agent harness.
Title: StreamVibe: Proactive Video Agents for Evolving User Intent
URL:
Key points
ใป๐ฏ Intent split into three levels: explicit, implicit, dynamic
ใป๐ StreamVibe-Bench: 4,100 samples, 20,546 help points, 183.4 hours of video
ใป๐ง StreamVibe-VL: adaptive SFT, then GRPO with trajectory-level rewards
ใป๐ RL cuts false triggers: explicit FPR 2.02 vs. 12.75 for ViSpeak
ใป๐ The Harness tracks active intents and progress, helping most on dynamic intents
ใป๐ Dynamic Pointwise F1 rises from 58.70 (SFT) to 66.34 (RL)
ใปโ ๏ธ Long interactions remain hard: dynamic Trajectory C-F1 is only 8.91
The hard part of proactivity is knowing when to stay quiet, not just when to speak.
#
VideoAgents# #
VLM#