Skip to content

Keep content in place when a rewritten page is cut into streams - #125

Merged
bobsingor merged 1 commit into
embedpdf/mainfrom
embedpdf/fix/content-cut-into-streams
Nov 18, 2026
Merged

bobsingor merged 1 commit into
embedpdf/mainfrom
embedpdf/fix/content-cut-into-streams

Conversation

@bobsingor

Copy link
Copy Markdown
Contributor

A redaction on a page written by "Microsoft: Print To PDF" moved most of the page: text, table lines and images ended up displaced or off the page. v2 does not do this. Any change to existing page content triggers it, not only redaction.

Cause

That producer sets one cm for the whole page (scale and flip, never undone) and then cuts the content into streams wherever a size limit falls, in the middle of q ... cm ... Q. So each stream after the first ends at another CTM than it began with.

A change to existing content writes every stream again (rule faithful-content-rewrite). After each stream the generator wrote a cm to give the next stream the CTM the old stream ended at. That line is upstream's and has the matrix product the wrong way round:

CFX_Matrix ctm_difference = prev_ctm.GetInverse() * ctm;  // needed: ctm * prev_ctm.GetInverse()

It is right only where the two matrices commute (the first stream, or translations only), so everything from the third stream on moved. Upstream rewrites only the streams that changed and its one fixture for this (rectangles_with_leaky_ctm) has a single stream, so it rarely runs there. Each stream also began by undoing the CTM the old stream began with, which cannot work where that CTM has no inverse.

History: v2 wrote one canonical stream for this (936df8f, May 19), which had no test of its own. 13d3251 (Aug 5) took it out again in favour of rewriting every stream in place, and no fixture carried a transform across a stream boundary, so nothing failed.

Change

Every page object is written with its whole matrix, so nothing needs to pass from one stream to the next.

  • New CPDF_PageObjectHolder::ForgetStreamCTMs() drops the CTMs the parser recorded per stream.
  • CPDF_PageContentGenerator::GenerateModifiedStreams() calls it where a pass adds every stream to the dirty set. Each stream then begins and ends at the CTM the content begins with, stands alone, and upstream's cm is never reached: a pass either appends a stream or writes them all.
  • Upstream's line is left as it is, to keep the diff against upstream small. An append still reads the recorded CTMs, as upstream does.
  • The ledger's faithful-content-rewrite rule now says that where content is cut into streams makes no difference.

Tests

In epdf_content_rewrite_embeddertest.cpp, each on a document and on a layer in a transaction. All fail without the change.

  • KeepsContentCutIntoStreams and KeepsContentCutInsideATransformWithoutAnInverse: the new fixture redact_split_content_streams.pdf, hand-written. Page 0 is the Print To PDF shape, page 1 a cut after a cm with no inverse.
  • RewritingAgainKeepsContentCutIntoStreams: a second redaction on the loaded page.
  • AnObjectAddedAroundARewriteIsDrawnInPlace: an object added before and after a rewrite.
  • WhereContentIsCutIntoStreamsChangesNothing: every page of redact_rewrite_cases.pdf and three other fixtures, written again with its content as it is, cut into three streams, and with every operator in a stream of its own. Each cut copy must draw what the file draws, before the rewrite and after it, and every stream written must stand alone (q ... Q balanced, no cm outside it). About 6 s per mode in a debug build.
  • TextLeftAfterARedactionKeepsItsPlace: the characters left after a redaction keep their origins to 0.001 pt. Renders can't show a drift under half a pixel, because each glyph is drawn on a whole pixel.

On the unchanged code, the cut copies of the 51 case pages fail only the stand-alone check (52 streams with a cm outside q ... Q). Their transforms nest one level deep, where the wrong product still gives the right matrix. The pixels differ only on the new fixture.

Checked

macOS arm64, against the same tree without the change:

  • Embedder: 1820 pass, 4 skipped. The 12 failures are the ones that fail without the change; FPDFEditEmbedderTest.SetTextRenderMode, the only one that writes content, fails with the same output either way.
  • Unit: 1402 pass.
  • Pixel: 51 failures and 2 surprises, as on record for this Mac. pdfium_test never regenerates content.
  • check-fork-tags.py: no untagged block.
  • Layer-mode writers: the new tests leave nothing in a layer through the embedder hooks, so embedpdf_layer_mode_writers.txt is unchanged.
  • The reported document, through the C API with this build: a redaction changes pixels only inside its rectangle, and writing all nine pages again without a change alters no pixel.

Not run: the Linux suite. It should run before this merges, since the release runs it.

Left

  • The order of the product is an upstream bug; reporting it to PDFium with a test is a separate step.
  • A rewrite round-trip over all test PDFs (write each page again without a change, uncut and cut, and compare) as a check before a PR and in the sync gate. A probe of it found 8 of 207 cut pages changed on the old code and none with this change.

A redaction on a page from "Microsoft: Print To PDF" moved most of the page.
That producer sets one cm for the whole page, never undone, and cuts the
content into streams in the middle of q ... cm ... Q, so each stream ends at
another CTM than it began with.

A change to existing content writes every stream again. After each one the
generator wrote a cm to give the next stream the CTM the old stream ended at.
That line is upstream's and has the matrix product the wrong way round
(prev^-1 * ctm where ctm * prev^-1 is needed): right only where the two
commute, so everything from the third stream on moved. Each stream also began
by undoing the CTM the old stream began with, which cannot work where that CTM
has no inverse.

Every object is written with its whole matrix, so nothing needs to pass from
one stream to the next. A rewrite of every stream now forgets the CTMs the
parser recorded per stream (CPDF_PageObjectHolder::ForgetStreamCTMs): each
stream begins and ends at the CTM the content begins with, stands alone, and
upstream's cm is never reached. Upstream's line is left as it is.

v2 wrote one stream for this (936df8f), which 13d3251 took out again
without a test for the case; no fixture carried a transform across a stream
boundary.

Tests, each on a document and on a layer in a transaction:
- redact_split_content_streams.pdf: the Print To PDF shape, and a cut after a
  cm with no inverse.
- A second rewrite of the loaded page, and an object added before and after a
  rewrite.
- WhereContentIsCutIntoStreamsChangesNothing runs every page of the rewrite
  cases with its content as it is, cut into three streams, and with every
  operator in a stream of its own, and expects the same page each time. Every
  stream written must stand alone (q ... Q balanced, no cm outside it).
- TextLeftAfterARedactionKeepsItsPlace: the characters left after a redaction
  keep their origins. Renders can't show a drift under half a pixel.
@bobsingor
bobsingor merged commit 4a52c97 into embedpdf/main Nov 18, 2026
1 check passed
@bobsingor
bobsingor deleted the embedpdf/fix/content-cut-into-streams branch November 18, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant