Camera pose matters for video understanding! Today's MLLMs excel at recognizing activities, but still struggle with the underlying space and ego/object dynamics in video. We trace this gap to a missing piece: camera pose. Introducing Cambrian-P: a multimodal LLM natively grounded in camera pose. (1/n)

An AI that can turn the right corner doesn't mean it can navigate an entire city. UrbanGround measures exactly that gap, at real scale in Hong Kong. UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City 🏙️ Overview MLLMs have shown impressive local spatial skills — visual recognition, short-range movement, VQA — but whether those skills translate into sustained city-scale action was untested. UrbanGround is a physically-simulated replica of Hong Kong, built from geospatial data (OSM, satellite maps) in Unity, that evaluates MLLM spatial agency through 810 manually-verified task instances arranged across a five-level evaluation ladder. 🔍 The Problem Existing spatial reasoning benchmarks are mostly small-scale, synthetic, and short-range. None test the compounding of local decisions into kilometers-long routes, nor robustness to dynamic changes like weather, road closures, or pedestrian crowds. ⚙️ Methodology A three-layer framework (geospatial / simulation / agent) gives models first-person vision and an interactive map interface. Tasks escalate in five levels: ・Level 1: Visual recognition, orientation, active exploration ・Level 2: Short/long-range and instruction-constrained navigation ・Level 3: Implicit destination inference from description ・Level 4: Multi-task scheduling and route optimization ・Level 5: Dynamic adaptation to closures and pedestrians 📊 Results Visual recognition scores 77–93% — relatively strong. Orientation judgment drops to 23–58%. Short-range navigation success (~70%) collapses to near-zero for long-range tasks. Weather and lighting cut QA accuracy by 5–20 points. Pedestrian collision rates exceed 75% across all models. GPT-5.5 and Kimi-K3 lead overall, but all models share the same failure modes at scale. The core finding: "Local abilities do not compose into sustained exploration." Agents move locally compliant routes but cannot maintain spatial estimates beyond visible scenes or revise plans when routes become invalid. #MLLMs# #EmbodiedAI#

When you add annotations as rollouts in RL training, something counterintuitive happens — good policy actions start receiving negative advantages. OraRL names the problem and fixes it. Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs ❓ What goes wrong when you add oracle rollouts to GRPO training? 💡 "Advantage Inversion" occurs. Standard GRPO normalizes advantages across the rollout group using group mean and variance. Injecting a high-reward oracle rollout raises the group mean, which in turn causes policy rollouts that outperform the current average to receive negative advantages — a sign flip. Measuring 92,024 rollouts, naive oracle mixing inverted advantages in 42.5% of groups and 22.4% of rollouts. ❓ How does OraRL solve this? 💡 Five decoupled components. First, an oracle-free on-policy baseline is constructed without variance normalization (inversion is impossible by construction). A directional gain from the oracle-policy gap amplifies above-mean rollouts. The oracle advantage itself is calibrated against the strongest on-policy signal so it can't dominate the update. Sign-balanced advantage pruning retains equal numbers of positive and negative rollouts, compressing the residual flip rate to 0.3% and delivering a 1.48× speedup. ❓ How does it perform? 💡 Video-ORA-9B scores 73.1 on VSI-Bench, surpassing GPT-5 (55.0) and Gemini-3-Pro (55.1) by 18+ points. ReasonVOS segmentation improves +42.2 J&F and VideoHolmes gains +15.2 points over backbone. Training cost is just 2.2× SFT — under half the 4.9× overhead of GRPO with chain-of-thought. ❓ What about inference efficiency? 💡 No chain-of-thought is required, so P90 inference latency is 25.15 seconds vs 62.67 seconds for the CoT backbone. On data scaling, OraRL gains +5.2 points at 100k prompts versus +2.8 for GRPO, showing better sample efficiency throughout. #VideoMLLM# #ReinforcementLearning#

Tec-Do has completed a new financing round💰 Led by Huatai-General Atlantic, the round includes participation from Forebright Capital, GAC Capital, GSR United Capital, and other investors. The financing reflects leading investors' growing confidence in Tec-Do’s AI-powered approach to business growth, supported by our Navos Marketing Multi-Agent Platform and Tec-Chi Specialized MLLMs. The trend of Agentic Commerce is on its way!

🧠 Can your model act on information that's no longer on screen? Even frontier multimodal LLMs fall apart at a memory card game and a 3D maze. 📰 Title: Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games 🔗 URL: 💡 Overview This paper introduces RNG-Bench, a benchmark that measures whether MLLMs can hold past, no-longer-visible observations in mind and act on them. It isolates in-context belief-state tracking through two games: Matching Pairs (memory cards) and a first-person 3D Maze. 🔍 Challenges Solved Existing benchmarks expose full state or only test "remember-to-answer" after an episode ends. This work moves into closed-loop "remember-to-act" settings, where a recall error reshapes the observations that follow, capturing the real difficulty agents face. 🛠 Methodology & Proposed Approach ・Games framed as POMDPs, requiring belief states maintained from interaction history ・Difficulty controlled by grid size, visual pattern, and text vs image modality ・A Duel protocol pits two models on the same board, plus an Oracle that injects true hidden state ・A Memory Gap metric compares normal vs Oracle play to separate forgetting from poor decisions 📊 Use Cases / Results On 10×10 Matching Pairs, GPT-5.4 hit 62.3%, Gemini-3.1-Pro 50.0%, Qwen3.5-397B 25.3%. On a 13×13 maze, Gemini-3.1-Pro led with 50% success. Qwen3.5-397B collapsed from 90.6% (4×4) to 0.7% (12×12). Removing action-text traces dropped GPT-5.4 by ~75%, showing visual recognition, not history length, is the real bottleneck. #MultimodalLLM# #AIAgents#

Speech-to-speech no longer needs speech-to-text! Until now, our stack was VAD -> STT -> LLM -> TTS. Now it can send audio directly to multimodal LLMs: VAD → MLLM → TTS No STT. The model understands your voice. Now go build better voice agents!

Can test-time scaling work for diffusion language models? In our #ICML2026# paper "UnMaskFork," we show that having multiple masked diffusion language models collaborate on a single answer improves performance on coding and math tasks. Blog: Test-time scaling is an actively researched technique that boosts LLM performance by using inference-time compute, for example, by having a model think longer or repeatedly refine its answers. This allows us to enhance performance simply by increasing computation during inference without relying on additional training, giving us the flexibility to balance compute costs and performance based on the specific use case. Unlike standard LLMs that generate text left-to-right, masked diffusion language models (MDLMs) generate text by gradually filling in a fully masked sequence. MDLMs can generate multiple parts of a sequence in parallel, offering potential speed-ups, and they can generate flexibly while seeing the entire sequence at once. This makes them an actively studied new paradigm in language modeling. We found that the standard LLM approach of "raising the temperature to increase randomness and generate diverse answers" does not work well for MDLMs like Dream-Coder. Instead of relying on this randomness, our proposed method, UnMaskFork (UMF), creates diversity through "model switching." Multiple MDLMs share the task of unmasking a single answer, and we use Monte Carlo Tree Search to search for a promising sequence in which different models handle different stages. Each model picks up where the others left off, filling in the parts it is most confident about. This collaborative approach allows us to explore diverse answers while maintaining generation quality, consistently outperforming existing test-time scaling methods on coding benchmarks and scaling effectively on math as well. Test-time scaling is also crucial for advancing MDLMs, and our work shows that UMF can sidestep the difficulties specific to them. UMF requires no additional training or changes to the models; it works simply by combining pre-trained models at inference time. This allows us to leverage the diversity of diffusion language models trained on different data and with different methods to improve performance. We believe the value of UMF will only grow as more diverse MDLMs emerge. This work is part of our broader research into "collective intelligence of AI," alongside methods like AB-MCTS and Sakana Fugu that have multiple LLMs collaborate. We'll continue pursuing research that turns model diversity into a source of strength. For details of the algorithm and illustrative examples showing how this collaboration works, please see our blog and paper. Paper: 🐟

"In the Wide World the Wood-elves lingered in the twilight of our Sun and Moon, but loved best the stars; and they wandered in the great forests that grew tall in lands that are now lost" ~ J.R.R. Tolkien 🖼 detail from 'Elf Play' by August Malmström #TolkienTuesday#

🖼 Test-time scaling for image editing tends to hand every edit the same compute budget, wasting a lot of it. By allocating budget by difficulty and pruning with edit-specific verification, this work hits up to 2.2x speedup while preserving quality. Title: From Scale to Speed: Adaptive Test-Time Scaling for Image Editing URL: 📝 Overview ADE-CoT is a test-time scaling method tailored to goal-directed image editing. Instead of reusing Image-CoT methods built for text-to-image generation, it combines three strategies, difficulty-aware allocation, edit-specific early verification, and opportunistic stopping, to cut compute substantially while preserving quality. ❓ Challenges Solved Prior methods had three mismatches. ・Fixed sampling budgets waste compute on easy edits that barely improve ・General MLLM scores wrongly prune about 40% of samples that start low but ultimately score high ・Large-scale sampling produces redundant identical correct outputs, adding needless compute 💡 Methodology & Proposed Approach ・It reads edit difficulty, giving easy edits a minimal budget and expanding the search for hard ones ・A one-step preview estimates clean latents from noisy intermediates without extra denoising, making early verification reliable ・Grounded SAM2 checks that only the intended region changed, and DINOv2 embeddings remove redundant candidates ・It generates candidates sequentially and stops, via depth-first opportunistic stopping, once enough intent-aligned results are found 🎯 Use Cases It fits complex pose changes, multi-object removal or replacement, fine-grained regional edits, multi-turn editing, and high-quality editing under compute constraints, and is especially valuable where inference cost matters, like a production image-editing API. 📊 Experimental Results ・On GEdit-Bench, FLUX.1 Kontext is 2.2x, BAGEL 1.8x, and Step1X-Edit 2.0x faster than Best-of-N ・Reasoning efficiency more than doubles on a fixed 32-sample budget, and outcome efficiency rises 4.9x, 2.7x, and 2.9x across three benchmarks ・On hard multi-object edits like "remove the person standing next to the lady in white," it fixes the baseline's misidentification #ImageEditing# #DiffusionModels#