Agents are benefiting from frontier model intelligence, but are now blocked by search performance. So we developed NeoMME-Retriever, an efficient visual document retriever for Visual RAG.
We contribute two models:
• NeoMME, a family of 260M and 800M multimodal-native, multilingual foundation encoders trained from scratch. One bidirectional Transformer processes raw image patches and text tokens.
• NeoMME-Retriever, a visual document retriever built on the NeoMME backbone. It encodes PDF page as images without OCR and returns dense and late-interaction embeddings in one pass.
Both retriever model sizes sit on ViDoRe v3's model-size Pareto frontier. With matched 2048x2048 inputs on one NVIDIA L40S, the 260M model encodes about 51 pages per second, nearly 2x ColModernVBERT's 26 pages per second.
Built at
@hcompany_ai by
@tonywu_71 and
@Aurelien_L_. We released the weights under Apache 2.0 with day-0 Hugging Face Transformers support. Read
@tonywu_71's thread for more details. 👇
👀 Meet NeoMME: a family of 260M and 800M Multimodal-Native Multilingual efficient Encoders
One bidirectional Transformer processes text tokens and raw image patches, with no pretrained vision tower, text encoder, or decoder. (1/N 🧵)
Show more