Document intelligence: what it is, how it works, and API vs platform

Document intelligence is AI that turns unstructured files, such as PDFs, scans and photos, into structured data, meaning the text, the key-value pairs, the tables and the type of document. It builds on OCR, which only returns characters, by working out how those characters relate, such as which label a value belongs to or which row a price sits in. It's a category of software rather than one product. Microsoft sells a cloud service called Azure Document Intelligence, and Google, Amazon and platforms such as Docsumo, our product, sell the same capability.

We start with the definition and its two sister terms, then show what it adds to OCR, where it breaks, and how to choose between a cloud API and a platform.

What is document intelligence?#

Picture an invoice with "Total Due: $4,120.50" at the bottom. OCR sees a line of text. Document intelligence knows the figure is the amount owed and returns it as the invoice total, ready for your accounting system, with no one typing it in.

Vendors describe the core the same way. Microsoft says its service extracts key-value pairs, text and tables and returns them as structured JSON; Google says Document AI turns unstructured document data into structured data. That's the extraction layer. A business process needs more around it. Files get sorted and split, values get checked against other documents, and unsure fields go to a person before the data moves on. That's where an API and a platform differ, covered below.

What is document understanding?#

Document understanding is what researchers call the same problem. The goal is software that reads a page's text, layout and visual cues together, the way a person does, rather than only the characters OCR returns, so it can classify the document, pull out its fields or answer questions about it. Microsoft researchers define document AI, or document intelligence, as techniques for "reading, understanding, and analyzing business documents", and models such as LayoutLM are tested on form understanding, receipt understanding and document classification. Some vendors use it as a product name. UiPath's Document Understanding extracts data from forms and invoices, and Google calls Document AI a document processing and understanding platform.

For a buyer, the names point to one category.

For tools, see AI document understanding tools compared.

How document intelligence differs from basic OCR#

Feed OCR a receipt and you get a string of characters. Document intelligence adds four layers on top.

• Layout analysis

  Finds headers, paragraphs, tables and reading order. More in layout detection.

• Key-value pairs

  Links a label to its value, such as "Invoice Number" to "INV-2024-0892". More in key-value pair extraction.

• Table reconstruction

  Keeps each price on the row of its item and quantity. More in table extraction.

• Classification

  Works out whether a file is a W-2, a bank statement or a purchase order before extraction starts. More in document classification.

OCR output usually needs custom parsing or a person before it's usable. Document intelligence output can go straight to your systems once it passes validation.

Core capabilities, and where each one struggles#

Most products share the same capabilities. They differ in how well each one holds up on hard documents.

Most vendors offer prebuilt models for common documents, such as invoices, receipts, IDs and tax forms, and custom models trained on your own layouts. Handwriting goes through intelligent character recognition, and accuracy depends on legibility.

The document intelligence workflow#

Extraction is one stage. The stages around it decide whether the output is safe to use without a person checking every page.

Each extracted value carries a confidence score. A 0.95 on an invoice total means the model is fairly sure, not certain. Validation then checks the values against rules and other documents. Does the total equal the line items? Does the PO number match an open purchase order? Values below their threshold and failed checks go to a person, whose corrections can improve the model. The full workflow is in automated document processing.

Where document intelligence fails#

Four problems come up again and again. Plan for them before go-live.

• Layout driftA vendor redesigns its invoice and a model trained on the old layout pulls the wrong fields. Track accuracy by sender so the change shows up in days.
• Miscalibrated confidenceA model can report 95% confidence on a wrong value, because confidence measures certainty, not correctness. Set thresholds per field on your own documents.
• Cross-document mismatchesThe invoice says $10,000, the PO $9,500, and the receiving record shows 95 units at $100. Single-document extraction can't catch it; you need checks across the set.
• Handwriting edge casesDoctors' notes, rushed signatures and forms filled in outside the boxes often need a person, whatever the model.

Aim for predictable automation rather than total automation. You want to know in advance which documents will flow through untouched and which will need a person.

API versus platform: when each makes sense#

Microsoft, Google and Amazon sell document intelligence as cloud APIs. You send a document and get structured data back. Document intelligence platforms add the layers above extraction. The choice usually comes down to who owns the problem.

A platform pays off once exceptions outgrow a shared inbox, or when several document types feed one decision, like a loan file or a claim. Docsumo runs in the cloud only; if documents can't leave your own data center, a cloud API that also ships containers, as Azure's does, is the closer fit.

Our take. Most teams start by comparing extraction accuracy, and most vendors now do well on clean pages. The better question is who deals with the pages that aren't clean. If your engineers own those exceptions, an API is a fine start. If an operations team answers for the numbers, buy the review queue and the checks along with the model.

• 99%99%field-level accuracy across 250+ document types
• 95%+95%+of documents processed straight through, without manual review

Before you pick a tool#

Whatever a vendor calls it, the job is to turn documents into data you can trust. Run a week of your own messiest files through each option and count how many come through untouched. That number, and who handles the rest, should decide it.

Book a demo with a few of your own documents, or start a free trial. Plans are on the pricing page.

Frequently asked questions#