Register and share your invite link to earn from video plays and referrals.

Search results for DocumentProcessing
DocumentProcessing community
One keyword maps to one global community path.
Create community
People
Not Found
Tweets including DocumentProcessing
TL;DR MDFlux is a local-first desktop app for Windows and Linux that converts PDFs and office documents into clean, AI-ready Markdown, with built-in OCR for scanned pages and up to 6x fewer tokens. Title: MDFlux URL: Points 📄 Supports PDF, DOCX, PPTX, XLSX, EPUB, HTML, CSV, JSON, XML, images, and audio 🔍 Built-in OCR (RapidOCR) recovers text from scanned PDFs other tools can't read 📦 Batch-converts entire folders with concurrent processing 🔒 Fully offline after first setup, no cloud upload by default 🧹 Choose cleanup mode: off, rule-based, or AI-powered (local or API) ⚡ Uses 2-6x fewer tokens than vision-model approaches, 5.7x fewer on scanned pages 🛠 Built with Tauri 2 (Rust) plus Svelte 5, on top of Microsoft's MarkItDown It's a nice fit for prepping internal documents for a RAG pipeline while keeping everything private. #DocumentProcessing# #OCR#
Show more
Ready to learn how to build document processing pipelines for insurance agents? Join LlamaIndex Solutions Architect Abrar Mahi for a webinar on turning insurance documents into structured data for underwriting, policy review, and claims workflows. Using LlamaParse and Extract, we’ll cover: ✅️ Parsing complex accord forms, policy documents and other popular document types in the insurance industry. ✅️ Extracting policy details, property information, and claims history into a defined schema. ✅️ Verifying extracted values with citations and bounding boxes. ✅️ Combining confidence scores with validation rules to decide what proceeds automatically and what needs human review. We’ll bring these steps together in a working pipeline you can adapt to your own workflows. Register:
Show more
This summer, we brought in two first-year Waterloo interns (‘rising sophomores’ in 🇺🇸) and tasked them with shipping complex production features that help push the frontier of our document processing and extraction capabilities. Yash created a dashboard that allowed customers to better understand the distribution of documents flowing through the system - and gives us anonymized metrics that we can use to improve our own capabilities. Jishnu helped to do deep research into grounded confidence scores for all of our document extraction capabilities. These scores are a lot more calibrated to real uncertainty about any value being extracted compared to our earlier implementation, allowing customers to trigger HITL in the right places within their document workflow. More on this coming soon 🙂 Outside of work, I have fond memories of watching the Knicks final game 4 with them (the one where the Knicks came back down 29 points). We’ve historically brought on full-time engineers with a bit more experience, but Yash and Jishnu ramped up extremely quickly and exemplified the high-agency, high-energy scrappiness that we look for in anyone joining the @llama_index team. They weren’t old enough to drink but old enough to ship production code 😂. We’re so happy they spent time with us so early in their careers, and wish them the best with the next 4 years of school. If you’re young but cracked, we’d love to chat. We’re hiring across *a lot* of roles across product/engineering/research (as well as growth and GTM). Careers page:
Show more
The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. 2️⃣ A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within @llama_index to help any agent do two-pass document processing with higher accuracy and lower cost. 1️⃣ We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types 2️⃣ We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses to achieve SOTA in accuracy and cost across various doc parsing and extraction tasks. It can be called from any agent harness as an MCP or skill. It takes in page numbers as input, so that the agent can choose to run LlamaParse over a subset of the doc instead of the full doc as a “zoom-in” pass. Come check it out! LiteParse: LlamaParse: All the relevant docs, including MCP, are here:
Show more
Reliability Sprint Token usage grew 3.5x in a quarter and led to reliability issues across our databases, vector search, job queues, and document processing infra. We implemented database sharding and tighter incident detection, and migrated to horizontally scalable systems like Turbopuffer and Temporal among many other efforts. Resolved all major reliability issues in two months, now Harvey is scaling as fast as before but much more reliably for our customers.
Show more
A useful but little-known Gemini API feature 📑 A 1,000-page PDF with text, tables, and charts, understood all at once. Document processing just leveled up. Gemini's "Document understanding" processes up to 1,000 pages of PDFs multimodally, understanding text, tables, charts, and images together. Far deeper than text extraction alone. 📌 Title: Document understanding 🔗 URL: 🧩 Overview Document processing traditionally meant OCR text extraction, but understanding tables, charts, and layout information was a much harder problem. Gemini's document processing "sees" pages multimodally, incorporating text, tables, figures, and layout into its understanding. Supports up to 1,000 pages. 🛠 How to use it Upload a PDF via the Files API or include it inline in the request. Gemini visually interprets each page, comprehensively understanding text, tables, figures, and charts. It can answer questions like "aggregate the data in this table" or "explain the diagram in chapter 3." 🏗 Building it into production ・Contract review: pass hundreds of pages of contracts and automate clause search and condition comparison. ・Financial report analysis: read charts and tables in financial statements, auto-extract trends and risks. ・Technical document comprehension: Q&A on design docs and spec sheets including diagrams and tables. ・Academic paper analysis: summarize and compare papers with full understanding of graphs and experimental result tables. 💡 Use cases 📋 Automated contract and legal document review 💹 Financial statement and earnings report analysis 🔧 Technical spec comprehension including diagrams 🔬 Academic paper analysis with graphs and tables ⚠️ Watch out Low-quality scanned PDFs can reduce reading accuracy. While 1,000 pages are supported, cost and latency scale with page count. Narrowing to the relevant page range is a practical optimization. For confidential documents, also verify data handling policies. ✨ From "just extract text" to "understand the whole thing, charts and all." Try it on a chart-heavy document first and see the difference. #Gemini# #LLM#
Show more
QVAC Workbench 0.7 is here. We’re making local-first intelligence faster and more intuitive. This update removes friction between your hardware and your workflow, making sovereignty feel much more seamless. Desktop - GPU Embeddings: Faster document processing by leveraging local GPU power - Smart Search: Higher quality results with transparent source links. - Efficiency: New keyboard shortcuts and bulk thread deletion. Mobile - iOS Persistence: Improved background handling so the app stays active when minimized. - Edge Compute: GPU-accelerated embeddings brought to mobile. The foundation of stable intelligence is moving forward. Run it on your own hardware and own your future. Download 0.7 today.
Show more
Major breakthrough in my mini-data center operations. Over the weekend, I reworked most of the hardware. And built my own "Open Router" in-house front-end. Thanks to Qwen-27B and a fork in llama.cpp allowing KV cache streaming from system RAM, I was able to bring online a bunch of 16GB cards to handle multiple 27B streams to achieve large-scale document processing for our AI engines. As a result, where I used to have 48 streams running at peak, I now have HUNDREDS of streams online, using the exact same hardware. This will allow us to index and clean (normalize) more science papers, more article content, more PDFs, etc., as we continue to build out our in-house knowledge base that powers and both of which are free to the public. Enjoy!
Show more
Most AI research demos show you a polished answer. This one showed me the disagreement that happened before the answer. I gave Ling-3.0-flash @AntLingAGI a deliberately difficult question: Do four-day workweeks actually increase productivity, or do they simply compress the same workload into fewer days? Instead of asking for a quick summary, I asked it to coordinate five specialist roles: a scientist, a data analyst, a cross-validator, an archivist, and a research writer. Each role had a separate responsibility. The scientist defined the competing hypotheses. The analyst extracted comparable findings. The archivist tracked the sources. The writer could only use approved claims. And the cross-validator had one job: challenge anything that sounded more confident than the evidence allowed. That last role changed the result. The team reviewed 12 sources and challenged six major claims. Three claims were narrowed. One was rejected entirely. Even a widely repeated claim about a 40% productivity increase did not survive the evidence check. That is the part I wanted to see from an AI research workflow. Not just more information, but visible resistance to weak evidence. The final output included: - a direct executive answer - a structured research paper - a source and evidence table - a disagreement log - a six-slide executive deck - a quality-control summary The conclusion was also more useful than a simple yes or no: reduced working hours may maintain productivity and improve wellbeing under certain conditions, while compressing the same workload into fewer days can increase fatigue and intensity. The evidence did not support a universal productivity claim. What impressed me was not that Ling-3.0-flash generated a long response. Plenty of models can do that. It was the way the model maintained multiple roles, evidence standards, objections, citations, and deliverables across one extended workflow, while preserving uncertainty instead of smoothing it away. That makes Ling-3.0-flash especially interesting for work where execution matters as much as reasoning: research, search, coding, document processing, tool use, repeated checks, and other multi-step agent workflows. The strongest AI systems will not use the largest model for every task. They will combine deep planning with fast, cost-efficient execution. Ling-3.0-flash is built for that execution layer. Ling-3.0-flash is now available on OpenRouter and free to use through August 3, 2026. Try it in your coding, search, research, and tool-use workflows. Then show us what you build. Try Ling-3.0-flash: Documentation:
Show more