Document Routing Logic in Multi-Agent AI Pipelines

Document routing logic is the layer that decides which agent handles which document before any specialized model ever touches the content. It sounds like plumbing, and in a sense it is: unglamorous, easy to overlook, and the first thing that floods the floor when it fails. Most teams get the plumbing wrong in the same place, too: they build the classifier and skip the part where the system admits it might be wrong.

Document routing in an AI pipeline is the layer that classifies each incoming document, selects the appropriate agent or model to handle it, and defines what happens when that selection turns out wrong. Classification relies on structural signals (scanned image versus native PDF, number of columns, mixed versus single content), content signals (tables, signatures, handwritten fields), and a confidence score that determines whether a document moves straight to a specialist agent or gets escalated to a heavier parsing tier first. Sensitivity is factored in alongside type: a low-sensitivity intake form and a high-sensitivity medical record may look identical on the page, but sound routing logic maps each to different agents based on access controls, not just document structure. The handoff itself must carry the structured extraction output, the document type label, classification confidence, field-level confidence markers, and the routing rationale, because an agent receiving incomplete or unexplained context at the seam between pipeline steps is where most real-world routing failures actually originate.

What a routing decision actually consists of

Every agentic router does three things, no more and no fewer. It classifies what's coming in, it picks the model or sub-agent suited to handle that thing, and it decides what happens if that pick turns out wrong. Teams skip that third piece constantly, and it's the one that costs the most later, because a router with no fallback path isn't really a router. It's a one-way door.

In a document pipeline, classification is the trigger. A router can't act until it knows what kind of document just landed in front of it: an invoice, a signed contract, a claim form, a pay stub. But "knowing the type" involves more than a label. The router also needs structural signals: is the page a scanned image or a native PDF, does it have five columns or one, is it a clean single form or a packet with mixed content stapled together. Then there are content signals: tables, signatures, barcodes, handwritten fields. And there's a confidence score attached to all of it. A low-confidence classification shouldn't get routed the same way a high-confidence one does; it should get bumped to a heavier parsing tier first, full stop.

Sensitivity matters just as much, and most routing systems underweight it. A low-sensitivity intake form and a high-sensitivity medical record can look identically "simple" on the page, but routing logic that treats them the same is asking for trouble the first time the medical record gets handled by an agent with no business seeing it. Research out of Georgia Tech on agentic routing (the Topaz paper, 2026) lays out a customer-support pipeline where sensitivity tiers, low, moderate, high, map directly onto which specialist agent gets the work. That mapping should be non-negotiable, not a nice-to-have layered on after the fact.

None of this matters, though, if the handoff itself is sloppy. A routing decision isn't just "send this to Agent B." It's a commitment to an interface: what format Agent B expects, which fields have to be filled in before the document arrives, what the fallback is if the parser upstream only got through half the page. Get the classification right and botch the handoff, and the whole exercise still fails. And there's a deeper problem sitting underneath all of it: most routing systems today don't explain themselves. A document gets assigned to a model or agent, and there's no readable rationale attached, per the Topaz paper. Without that, a developer can't tell if the system made a smart efficiency call or just took the cheap option because the budget said so. That distinction should worry anyone signing off on a production pipeline.

The three main routing strategies and where each fits in document workflows

Rule-based routing is the oldest and plainest approach. Someone writes rules that match document labels, keywords, or structural features to specific agents. It's easy to explain to a compliance team, and it's auditable in the way regulators like: point to the rule, point to the document, done. The cost shows up over time, though. Rules take real effort to write and even more to maintain, and they handle new document types about as gracefully as a filing cabinet handles a new folder category nobody planned for, per the ORCH paper. This approach earns its keep in high-volume, well-defined settings, standard loan applications, fixed-format insurance claims, where auditability matters more than flexibility. It is the wrong choice the moment document variety grows past what a rule set was written to anticipate, and teams that keep patching rules past that point are just delaying the rebuild.

Performance-based adaptive routing takes a different bet: pick agents based on how well they've done on similar documents before. Multi-Armed Bandit methods have been the standard tool here, balancing exploration (try the agent that might be better) against exploitation (stick with the one that already works), but convergence tends to be slow. ORCH itself sidesteps that with a deterministic setup for reproducible routing, layered with an optional EMA-guided module that tracks performance history to keep agent selection stable. The result is accuracy gains over the best cheap single model, without leaning on the expensive frontier model for everything.

EvoRoute, from a National University of Singapore and Tongyi Lab collaboration in 2026, pushes this idea further still. It's a self-evolving router that picks Pareto-optimal model backbones at each step of a pipeline, drawing on a growing memory of past routing outcomes. On the GAIA and BrowseComp+ benchmarks, it cut execution cost by as much as 80% and latency by more than 70%, while holding performance steady or improving it. That number says something uncomfortable about how much waste sits in naive "always call the big model" pipelines: most of them are burning frontier-model budget on documents that never needed it. The tradeoff with any adaptive approach is the exploration phase. On a document type the system hasn't seen much of, accuracy risk is real while the router is still learning, and teams need to plan for that cold-start window rather than pretend it away.

Tier-based routing is the third path, and the most defensible default for anyone running document AI at real volume. Clean, simple pages go to fast, cheap parsing. Messy pages, dense tables, handwritten margins, multi-column layouts, get escalated to something stronger. Not every page of a two-hundred-page contract deserves frontier-model treatment, and tier-based routing is built around that fact. The catch is that it depends entirely on accurate difficulty estimation upstream. Mislabel a page as "simple" when it actually buries a complex table in the middle, and the pipeline fails silently rather than loudly, which is worse than failing loudly. The EvoRoute paper frames this as a trilemma: performance, cost, and latency can't all be maximized at once, and tier-based routing is the practical compromise most document pipelines settle on, given how much difficulty varies from page to page.

How upstream document parsing quality determines what the router can work with

None of the three strategies above matter if the parser feeding the router produces bad output. A router cannot make a sound classification decision on malformed or incomplete text, and misrouting downstream of bad parsing isn't really a routing failure at all. It's a parsing failure wearing a routing costume.

Reliable parser output means a handful of specific things. Reading order has to survive across columns, headers, and footers, not scramble into something a human would have to untangle. Table structure needs to come out with cell relationships intact, not flattened into a wall of linear text where "Subtotal" and "$4,200" end up three lines apart. Multi-page boundaries need to be identified and split before routing starts: a fifty-page packet holding five different document types has to be broken apart first, because the router can't classify five things as if they were one. And every extracted field needs some kind of confidence score attached, so the router knows when to escalate a shaky extraction instead of quietly passing it along.

The accuracy numbers here are worth sitting with, and they should unsettle anyone who assumes parsing is a solved problem. On RD-TableBench, an open benchmark built around a thousand complex tables, table extraction accuracy ranged from 90.2% at the top down to 64.6% at the bottom across major document AI providers. A router working off 64.6%-accurate table data is going to misroute or misfill on more than a third of table-heavy documents that come through it. That's not a rounding error. That's a structural weakness baked into the front of the pipeline, and no amount of clever routing logic downstream fixes a parser that's wrong one time in three.

The failure mode that should worry teams most is the silent one. A parser can return incomplete output without surfacing any error, so the router receives what looks like a complete, well-formed result and routes on it confidently, while the downstream agent acts on data that was never whole to begin with. Nothing in the system flags this. It just produces a wrong answer that looks exactly like a right one, and that's a far worse outcome than a parser that fails loudly and stops the pipeline cold.

Benchmarks themselves deserve some skepticism, too. One widely circulated comparison claiming a lopsided 116 out of 120 wins over competing parsers turned out, per a published critique on Medium, to use no standard OCR metrics like CER or WER, no ground-truth data, and likely internal reviewers instead of independent evaluators. Academic benchmarks like OmniDocBench (CVPR 2025, built on 981 PDF pages in its v1.0 release) are a step up, using Tree Edit Distance Similarity (TEDS) to score table structure rather than a blunt accuracy percentage. Still, no published benchmark will tell a team whether their specific invoice template, with its specific quirks, parses cleanly. That test has to happen on the actual documents a pipeline will see, not on someone else's test set.

Confidence scoring and built-in evaluation at the parsing layer aren't a nice extra feature. They're the mechanism that lets a router decide whether to move forward, escalate, or reject a document before it ever reaches a specialist agent. Skip that mechanism, and the router is just guessing with extra steps.

Pipeline architecture patterns for document routing: sequential, parallel, and hybrid

Sequential pipelines are the simplest shape: one agent's output feeds directly into the next agent's input, and routing decides which agent a document visits and in what order. A multi-step document pipeline might run data extraction, then categorization, then report generation, then compliance review, one after another. This is easy to build and easy to audit, since every dependency is explicit and visible. It's also fragile in a way that catches teams off guard: one broken step halts the entire chain, there's no branching logic to route around a problem, and a document misclassified at step one poisons every step after it.

Parallel pipelines run multiple specialist agents at once on pieces of work that are genuinely independent, then aggregate the results through a coordinating layer. This fits when a document packet, once split apart, contains sections that don't depend on each other: a loan file where the income verification agent and the property appraisal agent can work at the same time without waiting on one another. The catch is that routing has to correctly judge independence in the first place. Send two sections that actually depend on each other to separate agents that each assume they have the full picture, and the outputs will conflict, and nobody notices until reconciliation.

Most real enterprise workflows are neither purely sequential nor purely parallel, and pretending they fit one clean shape is where a lot of pipeline designs go wrong. They branch depending on document type, merge results back together, and loop when an extraction comes back low-confidence. Directed Acyclic Graph, or DAG, orchestration models this kind of branching explicitly, which makes the routing conditions auditable rather than buried in code. Tools like LangGraph handle that logic graph, while separate infrastructure layers connect agents to storage, wire up dependencies, and control which agents are allowed to call which tools.

The framework a team picks shapes what kinds of routing logic are easy to write, though no single framework wins across the board. LangChain offers a broad ecosystem with wide tool integration. CrewAI leans into multi-agent coordination specifically, with role-based hierarchies and sequential, hierarchical, and flow-based orchestration built in. AutoGen, from Microsoft, handles multi-agent collaboration through natural-language conversation between agents. Haystack is oriented toward retrieval-augmented generation and document workflows. The choice comes down to what shape of routing a given pipeline actually needs, not which name is best known.

What separates teams that get this right from teams that don't isn't the framework choice, though. It's whether they treat the whole agent pipeline as a piece of software, with version control, automated tests, real deployment pipelines, and site-reliability practices applied to the routing layer just as much as to the agents themselves, as production experience with agent pipelines consistently shows. Teams that skip this step tend to find out the hard way, usually in production, usually on a document type nobody tested.

Handoff protocols: how agents communicate routing decisions to each other

Routing a document to the correct agent solves half the problem. The receiving agent still needs the document in a form it can act on, with the context it needs attached and nothing extra cluttering the handoff.

Two protocols are shaping how this works in production right now. The Model Context Protocol, or MCP, standardizes how an agent reaches external tools and contextual data, giving the receiving agent one consistent way to pull from the document store, the extraction schema, and any relevant metadata, instead of custom wiring built for every single agent pair. The Agent-to-Agent protocol, or A2A, governs how agents coordinate with each other directly, negotiating and delegating tasks. Google announced A2A in April 2025. It lets agents hand off sub-tasks and coordinate without routing every decision through a central orchestrator.

A well-built handoff carries a few specific things, and skipping any one of them is where handoffs quietly rot. It passes the structured extraction output, not the raw document, so the receiving agent isn't re-parsing something already handled upstream. It carries the document type label and the classification confidence score alongside it. It includes field-level confidence markers, so the receiving agent knows which values it can trust and which ones need a human to look at them. And it carries the routing rationale itself, which rule fired, which score drove the decision, so both the receiving agent and any observability tooling watching the pipeline can audit why that call got made.

Topaz, the Georgia Tech project mentioned earlier, tackles the explainability problem directly. Instead of a silent model assignment, it runs a three-stage router: skill-based profiling of what each agent is good at, cost-aware routing that balances budget against quality through multi-objective optimization, and a natural-language explanation layer that tells a developer why a given routing trace happened the way it did. That last piece matters more than it sounds. It's the difference between a developer tuning a cost-quality tradeoff on purpose and a developer discovering months later that the system had been quietly routing everything to the cheapest option available. MLflow's agent engineering tools add another layer of observability here too, tracing agent reasoning, running LLM-as-a-Judge evaluations for faithfulness and completeness, and flagging drift or hallucination. The handoff point is a natural place to instrument all of it.

Skip the protocol, and the failure mode looks exactly like the fintech loan approval case described in Lyzr's research: documents misrouted to agents that lacked the context to handle them correctly. Each agent worked fine on its own. The system broke at the seam between them, and no amount of per-agent testing would have caught it, because the seam is exactly what per-agent testing doesn't cover.

Where do document routing decisions break in real pipelines, and how do you catch them?

Misclassification at the trigger point is the most common break, and it's almost always preventable. A document tagged as the wrong type gets routed to an agent whose extraction schema doesn't match what's actually on the page, and the agent either returns blank fields or starts hallucinating values to fill slots that don't apply. Most of the time, this traces back to a low-confidence classification that should have triggered an escalation and didn't. The fix isn't complicated in principle: set a confidence threshold, and route anything below it to a human review queue or a stronger parsing tier, not straight to a specialist agent that's going to act on a bad guess with full confidence.

Split boundary errors are the second recurring problem, and they tend to hide better than misclassification does. A multi-document packet that never got separated before routing sends mixed content, an invoice's line items tangled up with a contract's terms-and-conditions text, to a single specialist agent that has no way to tell where one document ends and the next begins. The extraction that comes back is corrupted, and it often looks plausible enough that nobody catches it right away. Splitting has to happen before classification, not after, because a classifier fed compound content will produce a compound, meaningless label.

Both failure types share a root cause worth naming plainly: routing systems that treat confidence scores as optional metadata instead of as the primary signal for whether a decision should proceed at all. A pipeline that logs confidence but doesn't act on it has, in effect, no confidence threshold. The document goes through regardless of what the score says, and the score becomes a number nobody looks at until something has already gone wrong downstream. Catching these breaks means building the escalation path into the routing logic itself, not bolting it on as a monitoring dashboard someone checks after the fact. A dashboard tells you what already broke. An escalation path stops it from breaking in the first place, and that difference is the whole argument for treating routing as infrastructure rather than an afterthought.