Judging a game-playing AI on just a handful of noisy playthroughs? A new dataset-and-benchmark combo says that's not good enough anymore. Title: GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay URL: GameHorizon Suite densely annotates 5,000 hours of gameplay from 21 AAA titles with short-, mid-, and long-horizon instructions, letting a single benchmark measure both execution skill and planning ability. Three highlights stand out. 🎬 Dense, multi-horizon dataset 5,000 hours across 21 titles, 411M keyboard-mouse events, and one instruction roughly every 2.63 seconds on average. Every frame is simultaneously aligned to short, medium, and long-horizon instructions. 🧩 Bottom-up annotation meets top-down evaluation Annotation builds actions bottom-up from raw inputs, while evaluation forces goal decomposition top-down. The 28.9-point accuracy gap between the two directions shows recognition and planning are genuinely different skills. 📊 Offline plus online, side by side 5,000 reproducible offline multiple-choice questions pair with an online track that resets after every failure to pinpoint where models break down. Across 44 models the mean accuracy is 64.7%, with GPT-6-Astra leading at 80.2%. What strikes me most is how clearly the numbers expose the gap between recognizing an action and actually planning ahead. #GameAI# #Benchmark#