Register and share your invite link to earn from video plays and referrals.

Search results for VLM
VLM community
One keyword maps to one global community path.
Create community
People
Not Found
Tweets including VLM
How H-Company Optimizes VLM Serving for Computer Use Agents With NVIDIA Dynamo
Transferring the Intelligence of VLMs to Robotic Control paper:
Just deciding when a diffusion VLM looks at the image can cut decoding FLOPs to about a third 👀 Look Only When Needed: Certificate-Guided Visual Routing for Efficient Diffusion VLMs 📌 Overview LOWN is a training-free, threshold-free decoding method for diffusion VLMs such as LLaDA-V. Instead of processing the image at every step, it cheaply checks whether a step needs the image at all. 🚧 The problem ・Diffusion VLMs use bidirectional attention, so KV caching does not apply and the image is recomputed at every step ・Visual tokens are 89–97% of the input, so most compute goes to re-processing an unchanged image ・Removing the image changes the unmasked token at only 22.8% of steps on average 🔧 Method ・Keep the predictions from a FULL (image-conditioned) forward as an anchor ・At each step, run a TEXT-ONLY forward with image tokens removed ・The longest prefix whose predictions match the anchor is "certified" and unmasked without a FULL forward ・If too few are certified, run FULL and unmask disagreement-first 📊 Results ・On LLaDA-V, 99.6% of baseline score at 0.353x decoding FLOPs ・With RedVTP, 0.223x FLOPs and the best score on all ten benchmarks ・Standalone, 57.4% less generation time on average ・With Fast-dLLM, 38–50% less generation time, though scores fall below Fast-dLLM alone ⚠️ Caveat Savings shrink on chart and document reading, where visual dependence persists. #DiffusionLM# #VLM#
Show more
🦾 Can a single frozen general-purpose VLM drive a robot with no learned action model and no segmentation tools at all? Title: MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation URL: A harness that gets a general VLM behaving like a human teleoperator, without learned action experts, coding agents, or grounding tools like SAM3. Three things stand out. 🎮 A mid-level action interface Instead of emitting joint commands, the VLM proposes parameterized moves, rotations, and gripper operations. It picks the actions and their magnitudes while a deterministic controller converts them into embodiment-specific motion and records what the robot actually did, so an intended movement never gets mistaken for an achieved one. 🧠 Five roles on one frozen VLM Planner, Executor, Monitor, Verifier, and Memory all share the same model but get different context and different output authority. The Monitor runs in the background and can fire a STOP alert that discards queued commands, say when the wrong object got grasped. Notably it never picks the corrective motion itself; that call goes to outcome assessment. 📈 It improves as the backbone does On LIBERO-PRO, where zero-shot VLA policies all score 0.0%, it reaches 66.7%, and its 53.8% under perturbations edges out the fine-tuned OpenVLA-OFT at 51.2%. On a real xArm6 it averages 95%. Swapping the backbone for GPT-6 Sol lifts it to 83.3% with no redesign. The reframing is the interesting part: zero-shot control as aligning general model capability with the right control abstraction, rather than learning a specialized policy. #Robotics# #VLM#
Show more
We comprehensively evaluated 16 recent frontier VLMs - including Opus 5.5 and GPT-6 Sol/Luna - on whether higher effort led to better document parsing performance. Higher effort typically leads to improvements on other benchmarks (coding, knowledge work), but up until recently it wasn't obvious that this natively improved capabilities for reading PDFs. Results: ✅ Out of the frontier models, Opus 5.5 has the best performance relative to its price. It's especially good at parsing tables. ✅ Astra is also quite good, but starts at a more expensive price than Opus. ✅ GPT-6 Luna is more compelling at the cheaper end of doc parsing If you're parsing documents at scale, you'll still want a dedicated OCR solution like LlamaParse ( that has better performance at a cheaper price. But if you're parsing docs "in the agent loop" within an app like Codex/Claude code, and you're too lazy to integrate a dedicated solution, then Opus 5.5 is currently the leader. Full results on ParseBench:
Show more
“In-Context Robot Learning with VLM Agents” Most robots need lots of task-specific training before they can do something new. This paper shows you can instead just show a general VLM what to do and let it adapt directly from context. GPT-Policy essentially turns robot learning into prompting. So they gave the frozen VLM an example of the task, even just a human video with no robot action labels, and it can translate what it sees into robot actions on the fly. This brings few-shot learning into the physical world, where teaching a robot a new behavior could look more like showing it an example than retraining a policy.
Show more
🤖 What if you could pilot a robot with a VLM that never sees a single robot training example? RoboDawn tackles exactly that question. Title: Transferring the Intelligence of VLMs to Robotic Control (RoboDawn) URL: It gives a frozen, pretrained VLM a human-intuitive interface of translation, rotation, and gripper commands, plus a handful of in-context demonstrations, and lets it directly drive a robot arm. Three things stand out. 🎮 A game-like control interface The VLM issues discrete move, rotate, and gripper commands, all defined relative to the gripper interaction point. This lets it reuse spatial manipulation knowledge it already picked up from web-scale pretraining. 📚 One demo makes a huge difference No parameter updates at all. Just a command primer plus task demonstrations as context lift success on RoboTwin 2.0 from 53.2% zero-shot to 73.6% one-shot. 🏆 It beats robot-trained policies outright With zero task-specific training, RoboDawn's zero-shot performance already surpasses policies trained on dedicated data, like π0.5 (46.0%) and LingBot-VLA (50.4%). On a real Franka robot it hits a 90% success rate. It suggests the real bottleneck may not be collecting more robot data, but designing the interface that unlocks the intelligence VLMs already have. #Robotics# #VLM#
Show more