가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
가입 May 2026
286 팔로잉 중    468 팬
Just deciding when a diffusion VLM looks at the image can cut decoding FLOPs to about a third 👀 Look Only When Needed: Certificate-Guided Visual Routing for Efficient Diffusion VLMs 📌 Overview LOWN is a training-free, threshold-free decoding method for diffusion VLMs such as LLaDA-V. Instead of processing the image at every step, it cheaply checks whether a step needs the image at all. 🚧 The problem ・Diffusion VLMs use bidirectional attention, so KV caching does not apply and the image is recomputed at every step ・Visual tokens are 89–97% of the input, so most compute goes to re-processing an unchanged image ・Removing the image changes the unmasked token at only 22.8% of steps on average 🔧 Method ・Keep the predictions from a FULL (image-conditioned) forward as an anchor ・At each step, run a TEXT-ONLY forward with image tokens removed ・The longest prefix whose predictions match the anchor is "certified" and unmasked without a FULL forward ・If too few are certified, run FULL and unmask disagreement-first 📊 Results ・On LLaDA-V, 99.6% of baseline score at 0.353x decoding FLOPs ・With RedVTP, 0.223x FLOPs and the best score on all ten benchmarks ・Standalone, 57.4% less generation time on average ・With Fast-dLLM, 38–50% less generation time, though scores fall below Fast-dLLM alone ⚠️ Caveat Savings shrink on chart and document reading, where visual dependence persists. #DiffusionLM# #VLM#
더 보기