How a PDF Stores Text And Why Copying Goes Wrong

How a PDF stores text explains almost every problem people have with one. PDF files do not contain text. It contains instructions for putting marks on paper. PDF files hold no paragraphs in it, no sentences and no reading order, so getting text out is reconstruction rather than reading. Every complaint about copying from a PDF follows from that one fact.

The format was built to make a page look identical everywhere. It succeeded completely. Everything it is bad at is the price of that.

What a PDF File Actually Says

Strip a page down and the instructions read more like a printer than a document.

set the font to F1 at 11 points
move to position 72, 700
draw the glyphs 44 65 61 72
move to position 72, 686
draw the glyphs 53 61 6D

Nothing there says paragraph. Nothing says heading. Nothing says sentence. Nothing says this line continues the one above. There are marks and there are positions. The resemblance to a document exists only in your eye.

This is why a PDF looks the same everywhere. Nothing is being laid out when you open it. The layout already happened and the file records the result, which is exactly the property an HTML page does not have and exactly why the format won.

The Split That Explains Garbled PDF Copies

Here is the part almost nothing explains. It accounts for the strangest failure of the lot.

What gets drawn comes from the font. Which character that mark represents comes from a mapping stored separately in the file. Two different pieces of information. Nothing forces them to agree.

glyph 0044
  draw this shape

the page shows
  a capital D

glyph 0044
  means U+0044

missing or wrong
  the copy is
  whatever it says

This is the same shape of problem as character encoding. Bytes that need a table to mean anything. A table that may be absent or wrong. The difference is that a PDF renders correctly regardless, so the fault is invisible until somebody copies something.

Four PDF Text Failures With One Cause

The fix it articles list these as separate problems. They are one design decision seen four ways.

The spaces row surprises people most. A PDF need not store a space character between words. It can simply draw the next word further along, so the gap you see is a position rather than a character. Extraction has to decide how wide a gap counts as a word break. That is a guess made per file.

Why Scanned PDFs Are a Different Problem

A scanned PDF is not a PDF with difficult text. It holds none at all.

The file holds an image of a page. To a reader it is a photograph. The letters in it are no more text than the letters on a photograph of a street sign.

A searchable scan is two layers. The picture people read. A text layer nobody sees, holding a machine's opinion of the first. Where a search across scanned archives comes back empty, the document may well be there and the guess may simply have been wrong.

The Kind of PDF That Does Extract Cleanly

There is a version of PDF built to answer all of this. Most files are not it.

A tagged PDF carries a structure layer alongside the drawing instructions. It records what is a heading, what is a paragraph, which cells belong to which table and what order any of it should be read in. Exactly the information the base format leaves out.

Tagging was built for screen readers and it fixes extraction by accident. Where a document has to be readable by a screen reader it also extracts cleanly, which is a rare case of an accessibility requirement paying for itself in an unrelated place.

Getting Text Out of a PDF With Fewer Surprises

1. Establish which kind you have first. Magnify the page hard. Sharp means text objects. Soft means an image. The two need entirely different approaches.
2. Try selecting a single word before doing anything else. If it highlights and pastes correctly, the mapping is intact and the rest is a formatting problem rather than a data one.
3. Expect to rejoin the lines. No extraction preserves paragraphs a PDF never held. Plan for a pass that joins lines and keeps the real breaks.
4. Check a column layout deliberately. Copy a paragraph from a two column page. Where it interleaves the columns, the drawing order and the reading order disagree and no setting fixes that.
5. Treat a copy restriction as a preference. It asks the reader not to allow copying and changes nothing in the file. A tool that ignores the request reads the text objects exactly as before.

Where the extraction has to be right rather than roughly right, read the result rather than the original. A person comparing the extracted text against the page finds every one of these faults in a couple of minutes. No automatic check finds them at all.

The first of the five checks above needs no software beyond what you already have. Zoom to four hundred percent and the answer is on screen. Where the file has to be examined more closely than that, our PDF viewer shows whether a page carries text objects at all, which is the one question that decides everything after it.