What Gets Lost Converting Documents to Text for an LLM

A PDF is not text. Neither is a Word file or a spreadsheet. Something has to walk the container and decide what the words were. That step is where a document can stop meaning what it meant.

Structure, mostly. The words usually survive, and what they belonged to often does not.

The losses that hurt are not exotic. A running header spliced into the middle of your sentences, a footnote detached from its claim, a link reduced to its anchor text, a table value shifted one column over. None of them look like errors. A row missing a cell still reads as a row, and a date arriving as 46037 still reads as a number somebody meant. A model has no way to know.

So if a structure matters to your question, check that it survived. None of these checks needs a tool, and each one is faster than reading the document again.

It can be either, and only a second reader on the same file tells you which.

Some structure is not in the file at all. A PDF holds positioned text with no notion of a cell, so every reader has to guess where a table's cells begin and end.

Other losses are a reader's choice. We ran every PDF through an independent reader that shares no code with ours. Both read an ordinary two-column page correctly. On one whose internal order runs across the gutter, the other reader alternated one line from each column, where ours kept the columns apart. On pages with a running header and footer, it poured every copy into the prose. Ours keeps the header once and drops the page numbers, because the page breaks are already marked.

That is why swapping libraries on the strength of a published ranking is a gamble. The rankings score whole pages and average them, and the largest one leaves headers and footers out of the score. Test the page you care about.

Drop a document and read what comes out before you paste it. Every loss we found in August is closed in the browser now, and the PDF reader that closed them brought two of its own. One is fixed: a running header printed on every page arrives once instead of not at all. The other is open: curly quotes and the en dash arrive as their plain forms. A scanned page is no longer dropped in silence: FileConcat names it and reads it with OCR on your machine.

The question this page answers has two halves. When a document becomes LLM context, which structures survive? And where one does not, is the loss forced by the format or chosen by the reader?

The second half is what makes the first half actionable. A structure that every reader loses is a property of the file format, and no amount of library shopping will get it back. A structure that another reader recovers on the same bytes is a decision, and decisions can be changed.

Document extraction is well benchmarked and the benchmarks answer a different question.

Two things are missing from all of it.

Nothing isolates a structure. These studies score whole real pages, so a merged cell, a running header and a two-column layout all land in one edit-distance number. When that number moves you cannot say what moved it. Our corpus goes the other way: one file, one structure, one check.

The largest benchmark excludes headers and footers. OmniDocBench states its ignore rule plainly: headers, footers, page numbers and page footnotes "participate in matching but are excluded from metric calculations", because tools handle them inconsistently. Captions are excluded for the same reason. That is a fair choice for ranking parsers against each other. It also means the benchmark is silent on a structure we measured doing real damage. A 40-page report that interleaves its running header into every page seam is not a small scoring artifact to the model reading it.

And nobody separates the format from the library. This is the finding that changes what you do next, and it needs two readers on identical bytes to say anything at all.

Twenty-seven documents, twelve formats, each file written to isolate one structure:

Every fixture is generated, written with the docx, exceljs, pptxgenjs and pdfkit libraries plus PIL for the scanned pages. No real corpus was available, and a real one could not be published anyway. The directory is gitignored precisely because a document dropped there is somebody's. What is published is the generator and this measurement, not the files.

That is the sample's main weakness and it is not a small one. A merged cell written by the docx library and a merged cell written by Word are not guaranteed to be the same XML. Findings that could plausibly differ on a file Word itself produced are marked in the results.

The corpus is deliberately not weighted toward what people extract most. It is weighted toward structures that are hard, which is a different thing and the right thing for a floor check.

Every file goes through the path the product uses. Nothing here reimplements extraction.

1. 1

   Route from the bytes, never the name

   routeBytes reads the leading bytes and decides what the file is. A renamed .docx is still extracted, an extensionless PDF is still read, and a plain zip is told apart from the OOXML container that shares its signature. The file extension is never consulted.

2. 2

   Extract through both reader chains

   The browser starts a PDF with liteparse, a workbook with SheetJS and an RTF with anydoc, and moves to a second reader when the first returns nothing usable. The CLI reads all of them with officeparser. Both chains run from the code the product ships, so the text measured here is the text a bundle would carry. Up to FileConcat 2.0 the two were one chain.

3. 3

   Score each structure as a check

   Every defect from the earlier reading is written as a test over the extracted text. A rerun reports FIXED or STILL BROKEN. Nobody re-reads a paragraph and decides.

4. 4

   Read the PDFs again with an unrelated library

   The same bytes go through pypdf 6.14.2, which shares no code with our stack. Running the same check against both outputs is what turns a defect into an attribution.

Two details in that fourth step matter more than they look.

A check that hunts for a defect can pass for the wrong reason. Suppose a check looks for the broken output and the corpus is regenerated with different wording. The check then finds nothing and reports FIXED while nothing was fixed. Every such check therefore carries an anchor: text the fixture must produce for the check to mean anything. A missing anchor reports SIGNAL ABSENT and is never counted as a pass.

A check written for our output shape does not transfer. Our reader renders a recovered table with pipes. No other library does, so running that check against pypdf would score output formatting rather than whether data survived. That check is marked not compared, and the honest consequence is that the table finding below has no attribution.

Routing

Twenty-seven of 27 files routed correctly from their leading bytes. No file was misidentified.

Two produced no text, and both are correct behaviour rather than failure. The encrypted PDF is named as password protected, which is something a person can act on, instead of disappearing under a generic parse error. The fully scanned PDF has no text layer at all, so it yields nothing, and the browser chain names its page as a scan.

Re-testing our own extraction

The earlier reading found 16 defects and checked one correctness property that has never broken. This run adds two losses found on the browser chain itself. Re-run against FileConcat 2.2:

docx-3.2 carries the generated-file caveat: Word's merge XML differs slightly from what the docx library emits. That one is worth re-checking against a file Word wrote.

The CLI reads the same files with one library and scores 13 of 16 on the August list, as FileConcat 2.0 did on both. It keeps the curly quotes, so the one open loss is the browser's alone.

docx-3.1 is listed apart from the rest on purpose. It is a standing guard, not a defect that got fixed, and folding it into the headline would inflate the count by one. A tracked deletion has never reached a bundle, and the check exists so that stays true.

Where the loss belongs

Five PDF findings have checks that read only the text, so they can be run against both readers on identical bytes. This is the part no benchmark reports.

The third row is the practical one. The test file is three pages of prose with a running header and a Page n of 3 footer. pypdf pours the header into the text three times and the footer three times. FileConcat emits the header once and no footers, and marks the page boundaries instead. On a 40-page report that is the difference between 80 lines of headers and footers spliced through your sentences and one line.

This row read differently on the first run of this measurement. The browser's PDF reader, liteparse, strips every copy of a repeated line by default, title or confidentiality marking included. We now keep its copies and remove the repeats ourselves. None of the five readers we checked keeps exactly one copy: docling, marker and liteparse drop all of them, unstructured and pymupdf4llm keep all. A single page keeps liteparse's stripping, because there it can only remove page numbers, URLs and copyright lines.

The experiment that separates a column model from luck

Reading order on a two-column page is the classic extraction failure, and it is usually reported as a single pass or fail. That is not enough to tell a reader that understands columns from one that got lucky. A PDF's content stream often happens to be written column by column already.

So the corpus contains the same visual layout twice. One file writes the content stream column by column. The other writes it line by line across the gutter, which is what a reader following the stream would produce if it had no column model.

Ours finishes the left column through line 18 before the right column starts, and writes each column as one paragraph. pypdf interleaves every line. Both readers handle the naturally ordered file, so a test using only that file would have scored them equal and told you nothing.

This is the whole argument for structure-isolating fixtures in one place. The manipulation is free, it takes one extra file, and it converts a pass into evidence.

When the comparison refuses to answer

The footnote-marker row reports SIGNAL ABSENT rather than a verdict, and the reason is itself a result. On the typography fixture, pypdf's character mapping breaks against the embedded font: its own output reads Claim needin' support, with letters substituted for punctuation throughout. The browser's reader recovers that page's ligatures and puts the footnote marker after its claim, but writes the curly quotes and the en dash as plain ASCII. An o with an umlaut comes out as o" from all three readers, which points at how the generator embedded it rather than at any of them.

A comparison against a reference whose own text is corrupted proves nothing, so the measurement says so instead of scoring a false pass. Most extraction comparisons do not make that distinction, and it is the difference between "we fixed it" and "we cannot see it from here".

What is still broken, and where

Curly quotes and the en dash arrive as their plain forms, in the browser. The reader writes " for both curly quotes and - for the en dash in every output it has. The words are intact, so this one matters only where the exact characters do.

Three August defects remain in the CLI. It reads every PDF and workbook with the one library that had them: spreadsheet dates arrive as serial numbers like 46037, a superscript footnote marker sorts above its line, and a page with no text layer disappears without a note. The browser falls back to that library for a PDF its own reader cannot open, so those two PDF defects can still reach a browser bundle that way.

• Every fixture is generated. Real documents are written by software with its own habits, and Word's merge XML is not the docx library's. docx-3.2 is flagged for exactly this reason. Treat every result here as a floor check on hard structures, not as a survey of documents in the wild.
• One file per structure. A structure that passes here passes on one instance of it. This design buys attribution and pays for it in breadth.
• The independent reader covers PDF only. pypdf reads PDFs. The docx, xlsx, pptx and rtf findings therefore have no attribution column at all, and could be format limits or our own choices. We do not know which, and say so rather than guessing.
• One comparison is unattributable even within PDF. The table finding's check reads our own pipe rendering, which no other library emits.
• Both chains run under Node. The browser chain is the same code, but OCR does not run here, so a scanned page counts as named, not as read. Scan results here are floors.
• The header fix is checked on one file here. On olmOCR-bench's 753 header and footer tests, whose pages are all single pages, the score is 57.4% before and after it. A long document whose header changes wording from page to page keeps every copy.
• The fallback path is not measured. A PDF the browser's own reader cannot open goes to the CLI's library, where the superscript marker and the silent scanned page still hold. How often that happens has not been counted.
• The severity labels are ours. BROKEN means the bundle says something false or loses data invisibly, DEGRADED means it is poorer than it should be but not misleading. That is a judgment, it is published so it can be argued with, and a different reasonable line would move rows between them.
• This measures what survives, not what helps. Whether a model answers better from a document whose footnotes are attached is a separate experiment that needs a model in the loop. It is not this one.

# The corpus, written by real writer libraries. Gitignored output, tracked generator.
cd packages/core/tests/fixtures/real && node generate.mjs

# The independent reader.
pip install pypdf
python3 packages/cli/scripts/extract-pdf-with-pypdf.py

# Every structure, every predicate, both reader chains and pypdf.
pnpm --filter @fileconcat/cli measure-extraction

The last command prints both tables on this page: the per-file extraction for each chain and the findings with a column per chain and the attribution column. Run it without the pypdf step and the comparison column reports "not run" rather than assuming anything. How the same extraction behaves in day-to-day use, including what a parse failure does to a run, is in the CLI usage docs.

One number here cannot be reproduced from the current build: FileConcat 2.0's 13 of 16, measured on 2026-09-07 on a browser chain that no longer exists. The CLI column still reads 13 of 16 because its chain has not changed.

What gets lost when you convert a PDF to text?

Structure, mostly. Text in a PDF is positioned glyphs with no notion of a column, a cell or a heading. A reader has to infer all of it. In our measurement the recoverable-but-commonly-lost structures were column reading order, table cell positions, running headers and footers, and footnote attachment. The words themselves usually survive; what they belonged to often does not.

Is a document extraction failure the library's fault or the format's?

Both happen, and you cannot tell without a second reader. On identical bytes, our reader and pypdf both read a naturally ordered two-column page correctly. Only ours read a page whose content stream was written across the gutter. That is a library difference. A table's cell boundaries, by contrast, do not exist anywhere in a PDF, so no reader can recover them reliably.

Should I upload the PDF to the model instead?

Sometimes. Anthropic's documentation says each page is converted to an image and its text extracted, and both are sent. That is why charts and diagrams can be asked about. It costs more: their Bedrock documentation puts text-only extraction at roughly 1,000 tokens for a three-page PDF against roughly 7,000 for the full visual mode. Encrypted PDFs are rejected outright. Extracting to text first is cheaper and lets you see exactly what the model will see, which is the point of checking the headers and footers.

Why do my spreadsheet dates come out as numbers?

Because a date in a spreadsheet is stored as a number of days and its date-ness lives in the cell's format, not its value. A reader that ignores the format passes the raw serial through. FileConcat's browser reader writes each cell as the spreadsheet shows it, so a date arrives as a date. FileConcat 2.0 and the CLI pass the serial through.

Does a scanned PDF work?

In the browser, yes: a page with no text layer is named as a scan and read with OCR that runs on your machine, on its own for a small drop and on request past that. A partly scanned document says which page was the scan. In the CLI, no, and there an image-only page still disappears without a note.

How do I check whether my document survived extraction?

Open the extracted text and search for the specific thing you plan to ask about: the figure, the footnote, the link target. Every defect ranked BROKEN here produces output that looks plausible, so plausibility is not evidence.

• Adhikari, N. S. and Agarwal, S. A Comparative Study of PDF Parsing Tools Across Diverse Document Categories. arXiv:2410.09871, 2024. Ten parsers over the DocLayNet dataset across six document categories, scored with F1, BLEU-4 and local alignment.
• Ouyang, L. et al. OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations. CVPR 2025, arXiv:2412.07626. 981 pages over nine page types; normalized edit distance for text and reading order, TEDS for tables; headers, footers, page numbers and footnotes excluded from scoring.
• Anthropic. PDF support. Claude Platform documentation, read 2026-09-07. Page-to-image plus text extraction, 32 MB request ceiling, 600 page maximum, encrypted files unsupported.
• pypdf, version 6.14.2, used as the independent reader.
• FileConcat, How Many Tokens Is a Codebase? The companion measurement, on what reaches the model rather than what survives conversion.