The summary said the pilot saved 18%. The report said 8%, in a table, on page 46, and the 18 was a target from the introduction that the pilot missed. The model had averaged the hope and the result into a sentence a director would have forwarded. I caught it because I had the PDF open. That is the work. The button is the easy part.
What I fed it, on purpose
The full file was 90 pages. I did not upload 90 pages of context for a question about the pilot. I split pages 40–55, the pilot chapter plus the table, and ran AI PDF Summarizer on that. Twelve pages. The output was a short executive note and a handful of bullets. One bullet repeated the 18% claim from a sentence the chapter had quoted out of the intro. The table two pages later had 8%. The model does not know which number you consider load-bearing. You do.
Scanned chapters produce a vague summary or an empty one, because the summarizer sees extracted text, not the picture. I OCR those pages first, read the OCR, then summarize the corrected text. Summarizing raw OCR without that read is how a misread name becomes a finding.
| Input I tried | What came back | Usable? |
|---|---|---|
| All 90 pages | Smooth, generic, missed the table | No. It sounded like the introduction |
| Pages 40–55 only | Specific, one inflated figure | Yes, after I fixed the figure |
| OCR of a scanned appendix, uncorrected | Confident bullets with a wrong site name | No |
| The same OCR after I fixed names | Shorter and duller | Yes. Dull is fine |
The sentences I delete on sight
“This report provides a comprehensive overview.” It might. The sentence does not tell me anything I can check. I delete framing and I keep numbers, names, and dates. If a bullet has no noun I can find with Ctrl+F in the PDF, the bullet is atmosphere. Atmosphere is how the 18% survived a first reading.
I also delete advice the report did not give. Models like to end with recommendations. If the chapter did not recommend a rollout, the summary does not get to recommend one. I will write a recommendation myself, in a different paragraph, labeled as mine, so a reader can see the seam.
Length: I ask for something I can read in three minutes. On this tool the prompt is the summarizer’s own instruction — executive summary, key points, important details — not a novel you steer with a paragraph of style notes. If I need a different shape, I summarize the chapter and then paste that text into the rewriter with a blunt request: “cut this to 150 words, do not add recommendations.” Two steps. One step tries to be everything and smuggles the rollout.
A check I can do in five minutes
Pick three concrete claims. Find each one in the source. If one is missing or flipped, do not “lightly edit” the summary. Re-read the whole output with worse faith. Errors cluster. The 18% was not the only soft number. A date in the third bullet was the start of the pilot, written as if it were the end. I had skimmed past it because the percent had used up my attention.
Page references: the summary does not cite them unless the text you supplied contained the words “page 46.” I add the citations myself after I have verified them. A fake citation is worse than none. I do not ask the model to guess page numbers it cannot see.
Very large extracts hit practical limits and the request fails or truncates. The failure is useful. Split smaller. A summary of a summary of a whole volume is how the introduction eats the results. I would rather have four short summaries, each tied to a range in the filename: summary-p40-55.txt. The filename is the citation when the prose forgets.
Confidential reports do not belong in the box just because the summary would be convenient. The text goes to the Worker. I said that in the privacy note and I mean it on Tuesday nights when the shortcut is tempting. If the chapter cannot leave the machine, the summary is a human job or it waits.
I keep the model output in a scratch file and the edited summary in the email. I do not paste the scratch file by mistake. The scratch file still says 18% with a line through it. The email says 8% and names the table. That is the artifact. The button’s first draft is not the artifact, even when it reads cleanly. Clean is the danger. The wrong percent was grammatical.
Two summaries I refused to merge into one
Finance wanted “one page on the whole report.” The report was a pilot, a budget annex, and a risk register that contradicted the pilot’s cheerfulness. One page would have sanded the contradiction off, which was the only reason to read the pack. I sent three blocks with filenames in the headings: pilot pages 40–55, budget 70–74, risks 81–88. Each block was a checked summary. The contradiction stayed visible because I did not ask the model to reconcile them. Reconciliation is a human decision about which table wins. A model will reconcile by tone, and tone picks the introduction.
The budget block was almost useless until I deleted every adjective and left the four line items. Summarizers decorate line items. “Significant investment in infrastructure” was a $40,000 line for cables. I replaced the adjective with the figure. The director’s reply asked about the cables, which means the figure survived contact with a reader. The adjective would have survived too, and nobody would have asked anything.
Timing: the first run on pages 40–55 took a few seconds once the text was extracted. The checking took longer than the run, which is the correct ratio. When the run is the long part, the extract is too big or the service is retrying fallbacks. I do not sit and re-roll hoping for a luckier percent. I shrink the range. A second roll on the same bloated input reproduces the same bias toward the front of the file. I have compared them. The 18% moved paragraphs. It did not become 8% without me.
I archive the scratch file next to the extract, not next to the sent email, so a later “can you resend the summary” does not attach the uncorrected one. The sent email is in the mail. The scratch is evidence of what I threw away. Different folders. I learned the difference by attaching the wrong one, once, to the person who cared about the percent. They caught it. The next reader might not.
If finance later asks for a single paragraph anyway, I write it myself from the three checked blocks, and I include the contradiction in one clause instead of averaging it. “Pilot came in at 8% against an 18% target; the risk register still lists rollout as unfunded.” That sentence is mine. It is short because the checking already happened. Asking the model to glue the three summaries together at the end is how the target wanders back into the subject position of the sentence. I have watched the glue step undo the hour. Skip the glue step.