Camera pose matters for video understanding! Today's MLLMs excel at recognizing activities, but still struggle with the underlying space and ego/object dynamics in video. We trace this gap to a missing piece: camera pose. Introducing Cambrian-P: a multimodal LLM natively grounded in camera pose. (1/n)

An AI that can turn the right corner doesn't mean it can navigate an entire city. UrbanGround measures exactly that gap, at real scale in Hong Kong. UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City 🏙️ Overview MLLMs have shown impressive local spatial skills — visual recognition, short-range movement, VQA — but whether those skills translate into sustained city-scale action was untested. UrbanGround is a physically-simulated replica of Hong Kong, built from geospatial data (OSM, satellite maps) in Unity, that evaluates MLLM spatial agency through 810 manually-verified task instances arranged across a five-level evaluation ladder. 🔍 The Problem Existing spatial reasoning benchmarks are mostly small-scale, synthetic, and short-range. None test the compounding of local decisions into kilometers-long routes, nor robustness to dynamic changes like weather, road closures, or pedestrian crowds. ⚙️ Methodology A three-layer framework (geospatial / simulation / agent) gives models first-person vision and an interactive map interface. Tasks escalate in five levels: ・Level 1: Visual recognition, orientation, active exploration ・Level 2: Short/long-range and instruction-constrained navigation ・Level 3: Implicit destination inference from description ・Level 4: Multi-task scheduling and route optimization ・Level 5: Dynamic adaptation to closures and pedestrians 📊 Results Visual recognition scores 77–93% — relatively strong. Orientation judgment drops to 23–58%. Short-range navigation success (~70%) collapses to near-zero for long-range tasks. Weather and lighting cut QA accuracy by 5–20 points. Pedestrian collision rates exceed 75% across all models. GPT-5.5 and Kimi-K3 lead overall, but all models share the same failure modes at scale. The core finding: "Local abilities do not compose into sustained exploration." Agents move locally compliant routes but cannot maintain spatial estimates beyond visible scenes or revise plans when routes become invalid. #MLLMs# #EmbodiedAI#

When you add annotations as rollouts in RL training, something counterintuitive happens — good policy actions start receiving negative advantages. OraRL names the problem and fixes it. Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs ❓ What goes wrong when you add oracle rollouts to GRPO training? 💡 "Advantage Inversion" occurs. Standard GRPO normalizes advantages across the rollout group using group mean and variance. Injecting a high-reward oracle rollout raises the group mean, which in turn causes policy rollouts that outperform the current average to receive negative advantages — a sign flip. Measuring 92,024 rollouts, naive oracle mixing inverted advantages in 42.5% of groups and 22.4% of rollouts. ❓ How does OraRL solve this? 💡 Five decoupled components. First, an oracle-free on-policy baseline is constructed without variance normalization (inversion is impossible by construction). A directional gain from the oracle-policy gap amplifies above-mean rollouts. The oracle advantage itself is calibrated against the strongest on-policy signal so it can't dominate the update. Sign-balanced advantage pruning retains equal numbers of positive and negative rollouts, compressing the residual flip rate to 0.3% and delivering a 1.48× speedup. ❓ How does it perform? 💡 Video-ORA-9B scores 73.1 on VSI-Bench, surpassing GPT-5 (55.0) and Gemini-3-Pro (55.1) by 18+ points. ReasonVOS segmentation improves +42.2 J&F and VideoHolmes gains +15.2 points over backbone. Training cost is just 2.2× SFT — under half the 4.9× overhead of GRPO with chain-of-thought. ❓ What about inference efficiency? 💡 No chain-of-thought is required, so P90 inference latency is 25.15 seconds vs 62.67 seconds for the CoT backbone. On data scaling, OraRL gains +5.2 points at 100k prompts versus +2.8 for GRPO, showing better sample efficiency throughout. #VideoMLLM# #ReinforcementLearning#

Tec-Do has completed a new financing round💰 Led by Huatai-General Atlantic, the round includes participation from Forebright Capital, GAC Capital, GSR United Capital, and other investors. The financing reflects leading investors' growing confidence in Tec-Do’s AI-powered approach to business growth, supported by our Navos Marketing Multi-Agent Platform and Tec-Chi Specialized MLLMs. The trend of Agentic Commerce is on its way!

Speech-to-speech no longer needs speech-to-text! Until now, our stack was VAD -> STT -> LLM -> TTS. Now it can send audio directly to multimodal LLMs: VAD → MLLM → TTS No STT. The model understands your voice. Now go build better voice agents!

🧠 Can your model act on information that's no longer on screen? Even frontier multimodal LLMs fall apart at a memory card game and a 3D maze. 📰 Title: Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games 🔗 URL: 💡 Overview This paper introduces RNG-Bench, a benchmark that measures whether MLLMs can hold past, no-longer-visible observations in mind and act on them. It isolates in-context belief-state tracking through two games: Matching Pairs (memory cards) and a first-person 3D Maze. 🔍 Challenges Solved Existing benchmarks expose full state or only test "remember-to-answer" after an episode ends. This work moves into closed-loop "remember-to-act" settings, where a recall error reshapes the observations that follow, capturing the real difficulty agents face. 🛠 Methodology & Proposed Approach ・Games framed as POMDPs, requiring belief states maintained from interaction history ・Difficulty controlled by grid size, visual pattern, and text vs image modality ・A Duel protocol pits two models on the same board, plus an Oracle that injects true hidden state ・A Memory Gap metric compares normal vs Oracle play to separate forgetting from poor decisions 📊 Use Cases / Results On 10×10 Matching Pairs, GPT-5.4 hit 62.3%, Gemini-3.1-Pro 50.0%, Qwen3.5-397B 25.3%. On a 13×13 maze, Gemini-3.1-Pro led with 50% success. Qwen3.5-397B collapsed from 90.6% (4×4) to 0.7% (12×12). Removing action-text traces dropped GPT-5.4 by ~75%, showing visual recognition, not history length, is the real bottleneck. #MultimodalLLM# #AIAgents#

🖼 Test-time scaling for image editing tends to hand every edit the same compute budget, wasting a lot of it. By allocating budget by difficulty and pruning with edit-specific verification, this work hits up to 2.2x speedup while preserving quality. Title: From Scale to Speed: Adaptive Test-Time Scaling for Image Editing URL: 📝 Overview ADE-CoT is a test-time scaling method tailored to goal-directed image editing. Instead of reusing Image-CoT methods built for text-to-image generation, it combines three strategies, difficulty-aware allocation, edit-specific early verification, and opportunistic stopping, to cut compute substantially while preserving quality. ❓ Challenges Solved Prior methods had three mismatches. ・Fixed sampling budgets waste compute on easy edits that barely improve ・General MLLM scores wrongly prune about 40% of samples that start low but ultimately score high ・Large-scale sampling produces redundant identical correct outputs, adding needless compute 💡 Methodology & Proposed Approach ・It reads edit difficulty, giving easy edits a minimal budget and expanding the search for hard ones ・A one-step preview estimates clean latents from noisy intermediates without extra denoising, making early verification reliable ・Grounded SAM2 checks that only the intended region changed, and DINOv2 embeddings remove redundant candidates ・It generates candidates sequentially and stops, via depth-first opportunistic stopping, once enough intent-aligned results are found 🎯 Use Cases It fits complex pose changes, multi-object removal or replacement, fine-grained regional edits, multi-turn editing, and high-quality editing under compute constraints, and is especially valuable where inference cost matters, like a production image-editing API. 📊 Experimental Results ・On GEdit-Bench, FLUX.1 Kontext is 2.2x, BAGEL 1.8x, and Step1X-Edit 2.0x faster than Best-of-N ・Reasoning efficiency more than doubles on a fixed 32-sample budget, and outcome efficiency rises 4.9x, 2.7x, and 2.9x across three benchmarks ・On hard multi-object edits like "remove the person standing next to the lady in white," it fixes the baseline's misidentification #ImageEditing# #DiffusionModels#

For agent memory, the real question isn't "how to store" — it's "what to remember" 🧠 A fresh take that learns what to memorize via reinforcement learning. Title: Task-Focused Memorization for Multimodal Agents URL: 🧠 Overview This work proposes TaskMem, which treats long-term memory for multimodal agents as a learnable policy optimized with reinforcement learning, focused on deciding what to memorize. From an unbounded stream of observations, it selectively retains only the content relevant to the agent's role and task. ❓ Challenges Solved A multimodal agent operating in the real world continuously receives an unbounded stream of observations. ・Most prior work focused on how to store memories (designing memory modules) ・But the essential problem is what to memorize — without a principled way to select role-relevant content from an endless stream, memory simply fails This work starts from that shift in perspective. 💡 Methodology & Proposed Approach TaskMem treats memorization as a learnable policy, optimized in two phases. ・Phase 1: learn high-quality memorization under fidelity requirements ・Phase 2: post-deployment fine-tuning that uses task rewards to align memorization with the environment's demands ・It builds on the MLLM Qwen3-VL-30B-A3B and optimizes the policy lightly via adapter tuning ・Reward models derived from real tasks steer the policy toward selecting relevant content 🌍 Use Cases / Experimental Results On reformulated streaming benchmarks, it delivered clear accuracy gains. ・VideoMME: 67.9% VQA accuracy (+6.3%) ・EgoLife: 45.4% VQA accuracy (+7.0%) ・EgoTempo: 27.6% VQA accuracy (+5.3%) ・Strong precision across all benchmarks (80.5-85.6%) It charts a practical path for long-running, always-on agents to selectively remember the right things while keeping context bloat in check. #AIAgents# #Memory#