登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
参加 May 2026
286 フォロー中    469 ファン
TL;DR: One encoder and one JSON schema turn documents, images, audio and video into structured, grounded text. It posts the top reported score on document parsing. Youtu-Parsing-Omni: One Encoder, One Schema, All Modalities Key points ・🧩 A single shared Transformer encodes images, audio and video (initialized from Qwen3-0.6B-Base) ・⏱ Rotary positions split into time/height/width, so video and sound share one timeline ・📐 OmniSchema puts 10 tasks (documents, chemistry, sheet music, charts, audio, video...) into one JSON format ・🔁 Post-training combines OSAD distillation from a self-refreshing teacher with schema-aware RLVR ・📄 OmniDocBench v1.6: 96.96 overall, the highest reported ・🎬 OmniParsingBench: 75.08, best among open-weight models; text-rich video 75.62 vs 65.36 Gemini-3-Pro still scores higher at 77.44, and it trails Qwen3-Omni on Video-MME-v2. ASR is also a weak spot. Still, a promising step toward an "any-input parser" front end for LLM agents. #OmniModal# #DocumentParsing#
もっと見る