The copy-paste test is not enough

Why sensible people get this wrong

A commenter on a legal blog put it better than any security guide has:

Users are trained WYSIWYG. What you see is what you get. Quoted by Eugene Volokh, writing about redaction failures in court filings

Every piece of software we use has taught us that the page on the screen is the document. It is not. A PDF is a container, and the picture of the page is one of the things inside it. The others are not displayed, and most people have never been given a reason to know they exist.

This is not a failure of care. The people who leak documents this way are usually being careful. They are checking the only thing they were told to check.

Eight things the copy-paste test will never catch

1. A hidden text layer behind a scan. When a scanned page is put through text recognition, the software leaves an invisible copy of every word behind the picture, so the document can be searched. Black out the picture and the invisible copy stays exactly where it was. You cannot select it by dragging over it, because it is not where your cursor thinks it is — but it is still in the file, and software reads it without difficulty.
2. An image pasted over the words. A black rectangle drawn in a picture editor, or a screenshot with a box on it, covers the text visually. The text underneath is untouched. Whether copy-paste catches this depends entirely on how the page is put together, and you have no way to tell by looking. We found exactly this in a real bank statement during testing — and it had been produced by professional redaction software.
3. An earlier version of the document, still inside it. Many programs save changes by adding to the end of the file rather than rewriting it. The old version stays in place underneath. If the first save contained the unredacted text, it can still be in the file you are about to send — and nothing on the screen will ever show it to you.
4. Document properties. Author, title, subject, keywords, the program that made it, and sometimes the full path of the folder it was saved in. A file path alone can name a client, a case, or a person.
5. Bookmarks and page labels. The navigation panel down the side of a PDF carries its own text. A redacted figure was once recovered from a published contract because the bookmark titles still named it. Nobody copy-pastes the sidebar.
6. Page thumbnails. Some PDFs store a small picture of each page for quick previewing. If that picture was made before the redaction, it shows the unredacted page. It is a photograph of the thing you removed, travelling inside the file that removed it.
7. Attachments. A PDF can carry other files inside it. A carefully redacted report with the original spreadsheet attached is not a redacted report.
8. Form field values. Text typed into a form lives in the field, not in the page. The page can look blank while the value sits underneath it, and it will not always come out with a copy-paste.

And one more, which no manual check can find

In 2023, researchers published work showing that even a correctly redacted document can give up its secrets. When the hidden words are genuinely deleted, the letters on either side of the gap stay exactly where they were. Letters are not all the same width, so the size of the gap and the precise positions of the surviving letters carry information about what used to be there. With a shortlist of likely names, that is often enough.

They assessed eleven widely used redaction tools, including the best-known commercial one, and found that all of them leaked this way. Two of the findings were serious enough to be issued security identifiers. They recovered redacted names from real published documents, and notified the US courts, the Department of Justice, the Census Bureau and Adobe.

No amount of copying and pasting will show you this. The information is in the spacing.

So what should you actually do?

Keep doing the copy-paste test

It is quick, it needs no software, and it catches the most common mistake. Select the whole document, copy it, paste it into a plain text editor, and read what comes out. Just stop treating a clean result as an all-clear.

Check the rest of the file too

The eight items above are all inspectable. Most people have no practical way to inspect them, which is why we built a free tool that does it in one pass: Is your redaction real? Drop in a PDF and it reports on every one of them. It only reads the file — it never changes it, and it never produces a redacted version. Your document is not transmitted anywhere; the whole thing runs in your browser, and it works with your wifi switched off.

If something was blacked out rather than removed, redo it properly

Covering is not removing. The words have to come out of the file, and the safest way to guarantee that is to rebuild the page as an image — which also removes the spacing that the 2023 research exploits, because a picture has no letter positions to measure.