TL;DR: One encoder and one JSON schema turn documents, images, audio and video into structured, grounded text. It posts the top reported score on document parsing.
Youtu-Parsing-Omni: One Encoder, One Schema, All Modalities
Key points
ใป๐งฉ A single shared Transformer encodes images, audio and video (initialized from Qwen3-0.6B-Base)
ใปโฑ Rotary positions split into time/height/width, so video and sound share one timeline
ใป๐ OmniSchema puts 10 tasks (documents, chemistry, sheet music, charts, audio, video...) into one JSON format
ใป๐ Post-training combines OSAD distillation from a self-refreshing teacher with schema-aware RLVR
ใป๐ OmniDocBench v1.6: 96.96 overall, the highest reported
ใป๐ฌ OmniParsingBench: 75.08, best among open-weight models; text-rich video 75.62 vs 65.36
Gemini-3-Pro still scores higher at 77.44, and it trails Qwen3-Omni on Video-MME-v2. ASR is also a weak spot. Still, a promising step toward an "any-input parser" front end for LLM agents.
#
OmniModal# #
DocumentParsing#