A PDF endpoint returning 200 does not prove that its document is usable. A stylesheet change can add a blank page, clip a table, or move the total into an unexpected position while the generation request still succeeds.
The example in this guide combines three checks: required text, page count, and rendered-page differences. It also includes an intentionally broken document. The test must reject that document, otherwise the test harness has not demonstrated that it can detect the failure it claims to prevent. Download the example projects, fixtures, and results.
Define the document's acceptance rules
Our fixture is a 72-row service report. It must contain Service 001, Service 072, and Grand total USD 900.00. The approved local Chromium output has three A4 pages. A table-font change from the normal size to 22px intentionally expands the document to four pages.
The broken PDF still contains the expected text. This is a useful negative test: it shows why text assertions alone cannot protect layout. Conversely, a small missing account number might alter too few pixels to exceed a visual threshold, so visual comparison alone is also insufficient.
Prepare a deterministic renderer
Use the same browser, fonts, locale, page size, and rendering options when producing a baseline and a candidate. Freeze timestamps and sample data. Avoid external images or fonts that can change independently of your code.
The downloadable corpus embeds its content directly and records the local Chromium and Node versions in results/rendering.json. The npm lockfile pins the JavaScript dependencies. The browser executable remains an explicit environment dependency: the lockfile does not pin it for you.
Install Chromium and Poppler, then run:
npm ci
npm run fixtures
npm run compare
npm testThis runs the demonstration harness. It creates a fresh reference and a separate clean render, then proves that the deliberately broken fixture is rejected. In your own CI, keep an approved baseline from a reviewed commit; do not regenerate the baseline from the candidate on every run.
Compare PDF meaning and appearance
The supplied regression.mjs reads page counts with pdf-lib, extracts text with pdftotext, and rasterizes pages at 96 DPI using pdftoppm. Pixelmatch compares the resulting PNGs.
A candidate fails if its page count changes, a required text anchor disappears, page image dimensions change, or more than 0.2% of a page's pixels differ after Pixelmatch's color-distance threshold. That percentage is an example threshold, not a universally safe setting.
Compare two explicit files with:
node regression.mjs fixtures/report.pdf work/candidate.pdfThe command exits unsuccessfully when a comparison fails and writes diff images under work/custom. The included required-text anchors are specific to the service report; replace them with the invariants of your own document.
Recorded regression results
| Candidate | Expected pages | Actual pages | Maximum changed pixel fraction | Decision |
|---|---|---|---|---|
| Separately rendered clean report | 3 | 3 | 0% | Pass |
| Report with enlarged table text | 3 | 4 | 5.89% | Reject |
The results JSON lists the failed checks. Download the approved report and broken report to inspect the pagination difference.
These checks do not establish PDF/UA accessibility, PDF/A conformance, digital-signature validity, or OCR accuracy. Add specialized validators when those are document requirements. A PDF can look identical while its semantic structure has changed.
Use reviewed baselines in CI
A practical CI job installs pinned tooling, generates a candidate from the application under test, runs the comparison, and uploads both the candidate and diff images as build artifacts. Make baseline replacement a reviewed source change.
- name: Install example dependencies
working-directory: examples/blog-document-engineering
run: npm ci
- name: Compare candidate PDF
working-directory: examples/blog-document-engineering
run: node regression.mjs fixtures/report.pdf work/candidate.pdfThis is a job fragment, not a complete workflow: your preceding step must generate work/candidate.pdf, and the runner must have the recorded browser, fonts, and Poppler installed. Avoid hiding those prerequisites in a generic “run tests” command.
When the browser version changes, inspect and approve new baselines deliberately. Otherwise an automatic renderer update can either cause unexplained failures or normalize a real regression.
Triage failures instead of raising the threshold
If a page count changes, inspect margins, font loading, row heights, and page-break rules. If the text changes but the images look similar, check the source data and glyph mappings. If only a date differs, stabilize the test input instead of tolerating a larger global pixel difference.
Some differences are intentional. A redesigned header should produce a visible diff. Review that diff and replace the baseline only after confirming the rest of the document still meets its requirements.
The rendering test suite adds mixed page sizes, font fallback, SVG, and oversized rows. For common CSS causes of pagination failures, use the page-break guide. The same verification approach applies whether you render locally or through RelayPDF's HTML-to-PDF API.