Correct Words, Wrong Meaning: Audit Document Reading Order

A document parser returns every word on a page. The spelling is right, the numbers are intact and the output looks tidy. Yet two columns have been interleaved, a table value has inherited the neighbouring header, and a qualification beneath the table has become an unrelated paragraph. The resulting AI answer can quote the source accurately while explaining it incorrectly.

This is a structural extraction problem, not necessarily a hallucination by the answering model. The reader decision is whether a parser's output preserves enough relationships to become searchable evidence. Recognising characters is one condition; preserving what those characters belong to is another.

The documentation and research below establish specific mechanisms and limitations. The proposed audit, acceptance rules and worked document are AIEngine's engineering analysis, not findings from an AIEngine deployment. The aim is a reproducible test of document meaning before retrieval and generation obscure where it was lost.

Reading order is part of the evidence

A page has more than a bag of words. Paragraphs continue across columns, headings govern sections, captions describe figures and exceptions constrain nearby statements. Flattening these relationships can create a different argument without changing any individual word.

W3C's PDF reading-order technique explains that logical reading order in a tagged PDF is primarily determined by its tag order. Its two-column example illustrates the failure: incorrectly structured content can be read across both columns instead of down one and then the other. Complex layouts may require correction after conversion. This accessibility technique is useful evidence about document structure, not a guarantee that any particular parser follows it.

Do not adopt a universal rule that sorts all boxes by vertical position and then horizontal position. First establish regions and their intended relationships. A sidebar can sit beside a paragraph without being its continuation; a wide heading can introduce both columns below it.

For the audit, distinguish transcription, sequence and attachment. Transcription asks whether text was recognised. Sequence asks which passage comes next. Attachment asks which heading, cell, caption or qualification governs a passage. Test these independently before combining them into an application-level decision.

Keep a structural record before producing plain text

Plain text is a useful delivery format, but a weak sole record of extraction. Once geometry and relationships are discarded, debugging a wrong answer may require parsing the original again with a configuration that has already changed.

Amazon Textract's layout response documentation describes typed layout elements, geometry and relationships to underlying content. Lists can contain layout-text objects that themselves refer to lines. A consumer therefore needs to traverse the documented hierarchy, not assume every child is a line or that one flat list expresses the whole page.

Use a structure-preserving intermediate record. The following is a proposed application contract, not a universal vendor schema:

• Exact source revision and byte hash, with the original retained separately.
• Parser identity, version, options and extraction-run identifier.
• Page reference, orientation and coordinate units for each region.
• Node type, recognised text and bounding region where available.
• Explicit parent, continuation and semantic attachment relationships.
• Table row, column, span and header associations where required.
• Transformations applied after extraction, with their reasons.
• Unresolved relationships and the action needed to resolve them.

Do not treat a provider's node identifier as a permanent document address. The Textract Block API says block IDs are unique only within one operation. Scope such IDs to the source revision and extraction run; cross-run correspondence needs a deliberate mapping.

Keep recognised text separate from cleaned display text. Removing repeated headers or joining a hyphenated line may be reasonable, but the transformation should remain inspectable. An apparently harmless normalisation can otherwise erase the evidence needed to explain a disagreement.

Keep table meaning outside the table rectangle

A correct cell value is not a complete fact. Its row label, column header, units, period and qualifying notes may all be necessary to interpret it. A rectangle tightly enclosing the grid can exclude the most important condition.

Microsoft's Document Intelligence layout documentation makes this boundary explicit for version 4.0: table and figure bounding regions cover the core content and exclude associated captions and footnotes. Its Markdown output uses HTML tables to represent structures such as merged cells and multi-row headers. These details matter when a downstream converter assumes a table crop or pipe-delimited text contains everything.

Define a table evidence unit as the cells plus their required context. Preserve the original grid and an explicit association to its caption, units and applicable notes. If producing row-oriented records for retrieval, carry the relevant headers with each row and identify inherited context as inherited, not repeated source text.

Treat page continuations carefully. Google documents that its layout parser may split a PDF table spanning multiple pages. Repeated headers can help a continuation detector, but two adjacent tables with similar columns are not necessarily one table. Require supporting structure and review ambiguous joins rather than automatically concatenating every matching grid.

The broader document-AI workflow guide covers field validation and review. Structural extraction supplies the evidence those validators depend on; it cannot replace checks on the resulting business record.

Choose a parsing path by document family

Treat parser selection as a routing decision rather than a single product ranking. A controlled, digitally authored policy, a scanned maintenance manual and a dense slide deck present different failure opportunities.

Start with document families the application genuinely receives. Record whether each family has usable embedded text, trustworthy tags, scans, rotation, multiple columns, tables, handwriting or mixed languages. Test the intended route for each family, including the fallback. A fallback that handles one failed page can also change paragraph boundaries or identifiers elsewhere in the document.

A native-text path can preserve useful source structure where it exists. OCR plus layout analysis can recover regions from page images. A vision-language parser can produce a convenient structured rendition. None of these descriptions establishes adequate accuracy for the corpus; each route still owes the same evidence contract.

If a parser adds an explanation of a diagram, label that text as generated interpretation. Google describes optional generative annotations for figures and tables. Such a description may help discovery, but it is not a verbatim passage from the source and should not silently become one.

Include operational constraints in selection: supported formats, processor version and release status, page limits, processing location, latency and review effort. A route that preserves complex tables but makes most documents wait for manual repair may be unsuitable for the service. Measure that tradeoff without hiding rejected documents from the denominator.

Audit a document whose words are all correct

Construct a small test document in which structural mistakes have unambiguous consequences. This example is invented for testing, not an account of a real customer or safety procedure.

Imagine a two-column equipment catalogue. The left column describes standard service; the right describes emergency service. A table lists response targets for two product families, with separate standard and emergency columns. A note beneath the grid restricts the emergency target to covered locations. A second page continues the table with the same column headings.

Write expected relationships against the source before running either parser. Then create test mutations that leave the recognised words intact but disturb one relationship. These mutations test the audit itself: if the checker accepts them, a high OCR score cannot rescue it.

For each failure, a reviewer should be able to move from the disputed output to the exact source page and region. A file-level citation is too coarse to show whether the wrong header was attached during parsing or selected later during retrieval.

Keep a clean version and a genuinely ambiguous version as controls. The former tests unnecessary rejection; the latter tests whether the pipeline admits uncertainty instead of inventing a neat relationship. Do not force an annotation where the source itself fails to establish one.

Score relationships separately from transcription

The OmniDocBench paper, first submitted in December 2024 and revised in March 2025, evaluates document parsing across text, tables, formulas and reading order. Its annotations include layout and affiliation relationships. Importantly, its reading-order metric covers text components and excludes tables, images and ignored components; a good reading-order score is not a complete table or footnote audit.

That research supports separating dimensions, not importing a historical leaderboard as a procurement answer. Build the local test set around the layouts and consequential relationships the application must preserve.

Use distinct measurements:

• Character or word errors within correctly matched text regions.
• Missing, duplicated or incorrectly classified regions.
• Violations of required before-and-after relationships.
• Incorrect cell-to-header or row-label associations.
• Missing or incorrect caption and qualifier attachments.
• Source-location links that fail to identify the supporting region.

For ordering, label only relationships that matter. Two independent figures need not have one mandatory sequence if either presentation preserves their meaning. Conversely, an exception that must accompany a statement is an attachment requirement even if both appear somewhere in the correct overall order.

Report counts and errors by document family, not just one weighted average. Define critical failures before evaluating candidates. For example, a demonstrated header swap in a table used for factual answers should block that representation even when thousands of ordinary paragraphs are correct. Passing a finite test set does not prove that the same failure can never recur.

Avoid a universal confidence threshold. Textract describes block confidence as confidence in recognised text and geometry, not in the business meaning of every association. A confident word can still sit inside a wrongly interpreted structure.

Chunk only after structure is accepted

Chunking should consume the accepted structural record, not repair an anonymous text stream. Section headings, table headers and attached notes need a defined route into the retrieval unit. Google's layout-aware chunking documentation describes carrying ancestral headings and table headers into chunks; test what survives in the exact processor and configuration you use.

Keep inherited context distinguishable from source spans so citations remain honest. When a table cannot fit in one chunk, choose a representation that retains enough context for the supported questions. A row without its units or exceptions is not a smaller equivalent of the full table.

Then run downstream tests with the retriever, embedding model, answer model and question set held fixed while comparing extraction candidates. Inspect both retrieved evidence and final answers. This separates parser regressions from retrieval failures; the enterprise retrieval-quality audit provides the wider diagnostic sequence.

Structural approval is not security approval. Source trust, malicious instructions and admission policy remain separate checks, as described in the RAG ingestion-security guide. A perfectly reconstructed document can still contain false or hostile material.

Revisit the parser when its evidence changes

Retain the source revision, raw parser response, structural record, transformations, test labels and acceptance result together. A parser upgrade should produce a new rendition, not silently overwrite the one supporting existing citations. Record which index release consumed which rendition.

Watch relationship failures, unresolved documents, fallback usage, manual repair time and citation-location accuracy by document family. Re-test when the parser, layout model, normalisation rules, table serializer, chunker or incoming templates change. A new supplier template may invalidate a previously adequate path without any software deployment.

The useful acceptance statement is narrow: this extraction configuration preserves the required relationships for these document families, with these unresolved cases routed for review. That is stronger evidence than a tidy Markdown file or an impressive character-accuracy score, because it tests what the answering system actually needs to know: not only which words exist, but which words belong together.