PDF Parsing Libraries Compared for ML Pipelines

A PDF looks like a document but behaves like a picture: it stores where ink sits on a page, not what that ink means. Choosing a parsing library for a machine learning pipeline comes down to matching a tool's specific error pattern (how it handles tables, column order, scanned text) against whatever your downstream system can't survive getting wrong. Many teams pick by GitHub star count or a colleague's offhand recommendation, which is a bit like picking a surgeon by Yelp rating. This piece lays out a framework for making that match on purpose.

The three families of PDF parsers and what each trades away

Every PDF parser falls into one of three camps, and each camp gives something up to get something else.

Rule-based extractors read the PDF's internal object structure directly: the text runs, the XY coordinates, the font metrics that the file format already stores. Libraries here include pypdf, PyMuPDF, pdfplumber, pdfminer, PDFium (and its Python wrapper pypdfium2), Poppler, and Xpdf. They're fast, deterministic, and accurate on clean, text-native PDFs, meaning files that were generated by software rather than scanned from paper. They do no OCR and no layout inference. Whatever the PDF's internal object model says, that's what comes out. Feed one a scanned invoice or a document with a complicated multi-column layout, and it degrades fast because there's no visual reasoning happening, only object parsing.

Vision-pipeline extractors take a different route: render each page as an image, then recover the content through OCR, layout detection, and table-structure models stacked in sequence. Some platforms built around this layer expose the whole ingestion pipeline as a production-ready API rather than leaving teams to wire the stages together themselves. Docling, MinerU, Marker (built on the Surya model), and a handful of others fall in this family. Each is a chain of specialist models, layout detection feeding OCR, feeding table recognition, feeding reading-order reconstruction, and every link in that chain is a place a page can fall apart. The payoff is a much higher accuracy ceiling on messy, complex documents. The cost is heavier compute and slower throughput.

Commercial and managed APIs sit in a third lane: parsing-as-a-service, priced per page, with no infrastructure to run.

All three families get judged on the same axes: accuracy, cost, latency, how much structure survives extraction, whether scanned documents are supported at all, and whether you can run the thing on your own servers or have to send documents somewhere else. PDF-Bench v3.0, a December 2025 evaluation covering 17 parsers across more than 353 documents, found something that should worry anyone picking a parser on a single headline number: parsers hit 74% text accuracy on average but only 35% structure preservation, and the correlation between the two metrics was just 0.174. A parser that reads text almost perfectly can still mangle every table on the page, and the benchmark's own numbers say you can't predict one from the other.

The four accuracy dimensions that actually determine pipeline fit

Text fidelity, meaning character-level accuracy on extracted prose, is the number most people check first and the least useful number on its own. PDF-Bench's 0.174 correlation between text score and structure score means a parser can nail the prose and still butcher the table sitting three lines below it.

Table structure accuracy asks a harder question: do rows, columns, merged cells, and nested tables survive the trip from PDF to structured output? This is where most retrieval-augmented pipelines quietly rot. A dropped row in a large table extraction doesn't throw an error. It just delivers incomplete data to whatever agent or search index sits downstream, and nothing in the pipeline complains, because nothing was told to expect the missing row.

Reading order covers whether a parser correctly sequences multi-column layouts, mixed sections, and pages where text doesn't flow top-to-bottom in one lane. OmniDocBench, a benchmark built specifically to test this, covers 5 layout types and documents that multi-column and unconventional layouts present distinct challenges across the parsers it evaluates. Get reading order wrong and a chunking step downstream can stitch together sentences from two unrelated columns into one nonsensical passage, which then gets embedded and retrieved as if it made sense.

OCR fidelity on degraded input is its own axis. Printed text OCR on clean, high-resolution scans is a mostly solved problem at this point. What separates parsers is what happens with low-DPI scans, physical creasing, faded ink, or handwriting, conditions real-world document intake actually produces.

Each of these dimensions fails differently downstream: bad text fidelity corrupts the base content, bad table structure corrupts structured data extraction, bad reading order corrupts chunking, and bad OCR corrupts everything at the source. PDF-Bench's domain breakdown makes the stakes concrete: the same parser scored very highly on legal contracts but barely half that on invoices, a dramatic swing depending purely on document type. An aggregate benchmark score hides exactly the number that matters if your pipeline only ever sees invoices.

Rule-based extractors: where they hold and where they break

pypdfium2 posted 80.6% edit similarity overall in PDF-Bench v3.0, placing it among the top rule-based performers in the benchmark, with 98.8% accuracy on legal contracts specifically and 100% reliability across the full test corpus, meaning it never failed to return output. pypdf, often the first library a developer reaches for, is not built for table extraction, and its table scores reflect that plainly: it's a general-purpose reader, not a structure tool.

PyMuPDF scored 77.3% edit similarity and offers a fast, low-friction path to LLM-ready output through its PyMuPDF4LLM extension, which posted 74.7% edit similarity and 84.8% TEDS (a table structure metric) on tables. It's a strong pick for high-volume, text-native PDF processing, but it degrades on scanned documents unless paired with an OCR layer.

pdfplumber scored a middling 70.4% edit similarity overall, but 93.4% TEDS on table structure, the highest table-structure score of any open-source rule-based parser in PDF-Bench. It was built for table extraction specifically, and the numbers show that focus paid off.

pdfminer came in at 68.2% edit similarity, with its real strength in precise text positioning rather than tables or layout inference.

Every rule-based extractor shares the same blind spot: none of them can infer structure from visual layout. If a table's boundaries are implied only by spacing, or a document's reading order depends on column position rather than any tag in the file, these tools have no way to know it. Invoice processing is the standard failure case here. Most rule-based parsers in PDF-Bench topped out around 70 to 80% accuracy even on relatively clean invoice sets, because invoices vary in layout more than almost any other document type and encode meaning in visual position rather than file structure.

Vision-pipeline extractors: higher accuracy ceiling, harder operational tradeoffs

Docling, built by IBM Research and donated to the Linux Foundation in early 2026, runs on a layout analysis model trained on DocLayNet, a set of more than 80,000 manually annotated pages. It's MIT licensed, scored 71.3% edit similarity overall in PDF-Bench with 97.4% reliability (meaning roughly 1 in 40 documents failed to process, a real risk to plan around at volume), and hit 84.5% TEDS on tables. Independent evaluations on specialized document sets have found Docling competitive on complex table extraction. On standard two-column layouts Docling performs well, with occasional column merges showing up on pages with narrow gutters between columns. It plugs natively into common orchestration frameworks.

Marker, built by Datalab, wraps Surya, the underlying vision-language model that handles OCR, layout, and table recognition across more than 90 languages (reading order and text-line detection run on separate, smaller models alongside it). Marker itself is the higher-level tool, exposing a clean API that outputs Markdown, JSON, or HTML. It's built for speed relative to Docling and MinerU, and gives up some accuracy on dense tables in exchange. It scored 0.861 on the same reading-order benchmark.

MinerU, from OpenDataLab and Shanghai AI Lab, runs on the MinerU2.5-Pro accuracy model, a 1.2-billion-parameter model that scored around 95.7 overall on the latest revision of OmniDocBench. It produced the most consistent reading-order predictions across layout types in that benchmark, with a noticeable dip on handwritten documents. It produced the most consistent reading-order predictions across layout types in that benchmark.

A separate entrant, Dolphin, built by ByteDance and released open-source on Hugging Face, came out as the fastest parser tested in a financial-PDF evaluation run by F22 Labs, though it has meaningful VRAM requirements and requires extra cleanup work on headings, symbols, and Markdown formatting after extraction. It handles standard, text-heavy PDFs reasonably well but struggles on nested tables and dense multi-column layouts.

Every vision pipeline shares the same structural risk regardless of vendor: it's a chain, and chains break at their weakest link. A layout detection error feeds a wrong OCR region, which produces a broken table, which produces a wrong reading order. Tracing a bad output back to which stage caused it is a much harder debugging problem than anything a rule-based extractor presents, because rule-based tools fail in one step, not four.

Commercial and managed APIs: what you pay for beyond accuracy

Managed APIs sell convenience: no infrastructure to run, no model weights to host, a per-page price and a response. That convenience has a ceiling. In the F22 Labs financial-PDF test, a leading document-parsing API missed sections of a complex financial document entirely, and independent evaluations show commercial APIs can still struggle on complex layouts generally, not just at the margins. Direct comparison against open-source reading-order benchmark numbers remains difficult without standardized independent testing across vendors.

Managed APIs tend to fit best at modest volume, roughly under a few hundred to a thousand pages a day, on documents that aren't sensitive, and for teams already committed to a particular orchestration ecosystem.

One vendor worth naming on cost alone: pricing around 3 cents a page, with strong results on structured financial tables, though the F22 Labs test flagged inconsistencies in footnote parsing and heading hierarchy, and There's no self-hosting option, which is the tradeoff that defines the whole category.

A separate vendor claims high accuracy on enterprise documents in its own benchmark, reportedly outperforming several other commercial and open-source options tested alongside it. That's a self-reported number from the company selling the product, worth treating as a directional signal rather than an independent finding.

The hardest constraint in this category isn't accuracy at all. It's data residency. Sending a document to a managed API means that document leaves your infrastructure, and in healthcare, finance, and legal work, that's often not a preference you can override. It's a compliance wall. No accuracy number on any benchmark changes that.

How benchmarks measure parsers, and where they mislead

OmniDocBench, presented at CVPR 2025, tests 1,651 PDF pages spanning 10 document types, 5 layout types, and 5 languages, with 28 block-level and 4 span-level annotations covering text OCR, table recognition, formula recognition, layout detection, and reading order. Its April 2026 update, version 1.6, added a scoring method called Multi-Granularity Adaptive Matching built to remove matching bias in how predicted and ground-truth text get compared, and added 296 new pages covering nested tables, dense formula layouts, and unconventional document structures. For reading order and layout diversity specifically, it's the strongest public benchmark available.

PDF-Bench v3.0, released under an MIT license in December 2025, tests 17 parsers across more than 353 documents spanning 7 domains, and includes both open-source tools and commercial APIs, including frontier language models used as parsers. Its most useful contribution isn't the overall leaderboard, it's the 49-point gap between legal-contract and invoice performance on the same parser, which is the clearest public demonstration that an aggregate accuracy score can hide the one number that actually matters for a specific use case.

LongExtractBench, built by Micro1, takes a different angle: an independently audited set of 225 long documents averaging around 358 pages, scored with deterministic, key-based row matching designed specifically to catch silent row-dropping, the failure mode that aggregate accuracy metrics are worst at surfacing.

A methodology paper called ConfBench (arXiv 2608.01792, August 2026) raises a warning worth taking seriously: most public benchmarks test clean, high-quality documents, and skip the scanning artifacts and physical wear that real document intake produces. Those artifacts affect both extraction accuracy and how confident a system should be in its own output, and existing benchmarks don't have enough low- and mid-accuracy examples to measure that calibration properly.

Vendor-run benchmarks, the November 2025 accuracy claim mentioned earlier among them, and RD-TableBench results published by parsing vendors, are useful as directional signals. They are not independent, and treating a vendor's own benchmark the same way you'd treat OmniDocBench or PDF-Bench is a mistake worth avoiding on its face. The only benchmark that actually tells you how a parser performs on your documents is the one built from your documents: run the candidates on a representative sample from the specific domain your pipeline touches, invoices, contracts, financial statements, whatever it is, and measure the four dimensions above directly, rather than trusting an aggregate score to speak for a case it was never built to describe.