Camera pose matters for video understanding! Today's MLLMs excel at recognizing activities, but still struggle with the underlying space and ego/object dynamics in video. We trace this gap to a missing piece: camera pose. Introducing Cambrian-P: a multimodal LLM natively grounded in camera pose. (1/n)

🧠 Can your model act on information that's no longer on screen? Even frontier multimodal LLMs fall apart at a memory card game and a 3D maze. 📰 Title: Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games 🔗 URL: 💡 Overview This paper introduces RNG-Bench, a benchmark that measures whether MLLMs can hold past, no-longer-visible observations in mind and act on them. It isolates in-context belief-state tracking through two games: Matching Pairs (memory cards) and a first-person 3D Maze. 🔍 Challenges Solved Existing benchmarks expose full state or only test "remember-to-answer" after an episode ends. This work moves into closed-loop "remember-to-act" settings, where a recall error reshapes the observations that follow, capturing the real difficulty agents face. 🛠 Methodology & Proposed Approach ・Games framed as POMDPs, requiring belief states maintained from interaction history ・Difficulty controlled by grid size, visual pattern, and text vs image modality ・A Duel protocol pits two models on the same board, plus an Oracle that injects true hidden state ・A Memory Gap metric compares normal vs Oracle play to separate forgetting from poor decisions 📊 Use Cases / Results On 10×10 Matching Pairs, GPT-5.4 hit 62.3%, Gemini-3.1-Pro 50.0%, Qwen3.5-397B 25.3%. On a 13×13 maze, Gemini-3.1-Pro led with 50% success. Qwen3.5-397B collapsed from 90.6% (4×4) to 0.7% (12×12). Removing action-text traces dropped GPT-5.4 by ~75%, showing visual recognition, not history length, is the real bottleneck. #MultimodalLLM# #AIAgents#

🖼 Test-time scaling for image editing tends to hand every edit the same compute budget, wasting a lot of it. By allocating budget by difficulty and pruning with edit-specific verification, this work hits up to 2.2x speedup while preserving quality. Title: From Scale to Speed: Adaptive Test-Time Scaling for Image Editing URL: 📝 Overview ADE-CoT is a test-time scaling method tailored to goal-directed image editing. Instead of reusing Image-CoT methods built for text-to-image generation, it combines three strategies, difficulty-aware allocation, edit-specific early verification, and opportunistic stopping, to cut compute substantially while preserving quality. ❓ Challenges Solved Prior methods had three mismatches. ・Fixed sampling budgets waste compute on easy edits that barely improve ・General MLLM scores wrongly prune about 40% of samples that start low but ultimately score high ・Large-scale sampling produces redundant identical correct outputs, adding needless compute 💡 Methodology & Proposed Approach ・It reads edit difficulty, giving easy edits a minimal budget and expanding the search for hard ones ・A one-step preview estimates clean latents from noisy intermediates without extra denoising, making early verification reliable ・Grounded SAM2 checks that only the intended region changed, and DINOv2 embeddings remove redundant candidates ・It generates candidates sequentially and stops, via depth-first opportunistic stopping, once enough intent-aligned results are found 🎯 Use Cases It fits complex pose changes, multi-object removal or replacement, fine-grained regional edits, multi-turn editing, and high-quality editing under compute constraints, and is especially valuable where inference cost matters, like a production image-editing API. 📊 Experimental Results ・On GEdit-Bench, FLUX.1 Kontext is 2.2x, BAGEL 1.8x, and Step1X-Edit 2.0x faster than Best-of-N ・Reasoning efficiency more than doubles on a fixed 32-sample budget, and outcome efficiency rises 4.9x, 2.7x, and 2.9x across three benchmarks ・On hard multi-object edits like "remove the person standing next to the lady in white," it fixes the baseline's misidentification #ImageEditing# #DiffusionModels#

For agent memory, the real question isn't "how to store" — it's "what to remember" 🧠 A fresh take that learns what to memorize via reinforcement learning. Title: Task-Focused Memorization for Multimodal Agents URL: 🧠 Overview This work proposes TaskMem, which treats long-term memory for multimodal agents as a learnable policy optimized with reinforcement learning, focused on deciding what to memorize. From an unbounded stream of observations, it selectively retains only the content relevant to the agent's role and task. ❓ Challenges Solved A multimodal agent operating in the real world continuously receives an unbounded stream of observations. ・Most prior work focused on how to store memories (designing memory modules) ・But the essential problem is what to memorize — without a principled way to select role-relevant content from an endless stream, memory simply fails This work starts from that shift in perspective. 💡 Methodology & Proposed Approach TaskMem treats memorization as a learnable policy, optimized in two phases. ・Phase 1: learn high-quality memorization under fidelity requirements ・Phase 2: post-deployment fine-tuning that uses task rewards to align memorization with the environment's demands ・It builds on the MLLM Qwen3-VL-30B-A3B and optimizes the policy lightly via adapter tuning ・Reward models derived from real tasks steer the policy toward selecting relevant content 🌍 Use Cases / Experimental Results On reformulated streaming benchmarks, it delivered clear accuracy gains. ・VideoMME: 67.9% VQA accuracy (+6.3%) ・EgoLife: 45.4% VQA accuracy (+7.0%) ・EgoTempo: 27.6% VQA accuracy (+5.3%) ・Strong precision across all benchmarks (80.5-85.6%) It charts a practical path for long-running, always-on agents to selectively remember the right things while keeping context bloat in check. #AIAgents# #Memory#