Preserve-Design (in-place flyer tagging)

Tags designed PDFs (flyers, posters, short brochures) in place — marked-content injection + a PDF/UA-1 structure tree via pikepdf/PyMuPDF — instead of re-authoring. Output is verified pixel-identical (≥99.95%) and veraPDF UA-1 compliant, or it is not delivered. Shipped 2026-08-21/22 across PRs #2117, #2131, #2143, #2145, #2146, #2147.

Authoritative contract: services/flyer-tagger/docs/flyer-tagger-integration.md.

Architecture

upload (apps/web) ──► POST /api/v1/flyer/preflight ── sync Lambda invoke {preflight_only}
                          │  route: in-place | reflow  (+ eligible, designDense,
                          │  signed, formCentric — 'forms' route retired by #2144)
                          ▼
                  choice card (shown whenever eligible; designDense picks emphasis;
                  preserve on a non-in-place route sends force:true)
                          │ user picks preserve
                          ▼
                  POST /api/v1/flyer/tag ── entitlement gate (reserveUsage) ─► job row
                          ─► SQS flyer-tagger-jobs ─► settle (deductCredits, charge_units)
                          ▼
                  Lambda flyer-tagger (us-east-1, arm64 container, 3 GB / 600 s)
                    R2 download → preflight recheck → Bedrock vision analysis (1 call/page)
                    → inject tags → verify gate (pixel, qpdf, veraPDF, unclaimed ops)
                    → R2 upload (quarantine/<key> on gate failure) → job row + cost_ledger
                          ▼
                  GET /api/v1/flyer/jobs/:id  (poll: refund reconciliation, file completion)
                  GET  …/jobs/:id/download    GET …/jobs/:id/report (WCAG HTML report)
                  POST …/jobs/:id/accept      POST …/jobs/:id/reflow (comped conversion)

Routes live in workers/api/src/routes/flyer.ts, registered on BOTH prod compositions (index-aws.ts Lambda + server.ts Node) — never the legacy CF worker.
Queue: flyer-tagger-jobs + DLQ (maxReceiveCount 3, visibility 660 s), ESM batch size 1 with ReportBatchItemFailures.
Model: Bedrock Sonnet via the Lambda execution role (ZDR posture — no Gemini). Compact index-based response schema (PR #2146) ≈ $0.07/page. FLYER_TAGGER_MODEL env overrides for canary (Haiku ≈ $0.015/page but 70% role agreement + figure undercounting — human-reviewed canary only, see PR #2146 eval).

Billing (the part you’ll get support tickets about)

Charged at enqueue (consume_entitlement, product pdf_convert, 1 unit/page); the Lambda never charges.

Pre-flight quote (Polish, apps/pro). Before the user starts, the confirm bar counts each PDF’s pages in the browser (pdf-lib, same counter as /tag) and calls POST /api/v1/flyer/quote { pages } — a read-only checkUsageAffordable (no hold, no writes) returning { pages, mode, affordable, cost, available, unitPriceCents, shortfall, reason, purchaseUrl } (paygo amounts in cents, quota in pages). When short, Start is disabled and the bar shows the shortfall with an Add funds link to https://pdf.gevell.com/settings?tab=billing (TOP_UP_URL, also the /tag 402 purchaseUrl). Every drop — including a single PDF — now waits for an explicit start. /tag’s reserveUsage gate stays authoritative; if it still returns INSUFFICIENT_CREDITS (race, or the quote failed) Polish shows an informational toast with Add funds and does not POST /api/errors, so no admin ticket is filed for an expected balance condition (admin ticket #8). Real failures are still recorded. Resolution of a non-delivered job is exactly-once and mutually exclusive, enforced end-to-end:

Outcome	Trigger	Column stamped
Auto-refund	job `failed`, preflight-skipped, swept, or fidelity gate failure (pixel/qpdf)	`refunded_at`
Accept as-is	user keeps a compliance-only failure (veraPDF/untagged ops; pixel-perfect)	`accepted_at`
Comped reflow	user takes the free standard conversion instead	`comp_conversion_id` (stores the files.id)

DB backstop: flyer_resolution_exclusive CHECK (num_nonnulls(...) <= 1), migration 259.
All claims are guarded PostgREST PATCHes (…=is.null on all three columns); rollbacks are guarded on the request’s own stamp and check res.ok (PR #2145 review fixes — a silent rollback failure is a kept charge; grep logs for CLAIM ROLLBACK THREW).
The comped conversion’s billing bypass is _compedFlyerJobId in the server-built SQS message only; the client-facing zod schema strips it. Settlement skip lives in chunk-scheduler.ts.
Refund reconciliation is lazy (on poll) plus the sweep’s terminal pass for users who stopped polling.

Stale-job sweep

workers/api/src/services/flyer-stale-sweep.ts, Node (.4) scheduler only, every 10 min:

Rows stuck processing >45 min or queued >60 min → marked failed (status+cutoff-guarded PATCH; a concurrently-completing Lambda wins) → refunded immediately.
Terminal pass: failed/fidelity-failed rows with charge_units > 0 and no resolution, older than 30 min → refund retry. Compliance-only failures are excluded (they await the user’s accept/reflow choice).

Deploy runbook

Piece	How	Gotchas
Lambda `flyer-tagger`	`cd services/flyer-tagger && bash deploy/deploy.sh`	Env: script merges, never replaces (#2147 — a replace once wiped the R2/Supabase secrets, 2026-08-22). Secrets are set out-of-band from `.4`’s `.env.node-server`; never print values. Buildx: `BUILDX_NO_DEFAULT_ATTESTATIONS=1` if Lambda rejects the manifest.
workers/api (Lambda)	CI on merge to main (CDK carries `FLYER_TAGGER_QUEUE_URL` + SQS/invoke grants)	—
workers/api (Node .4)	`npm run rebuild` from repo root	Check `large_conversion_jobs`, `form_conversions`, AND `flyer_tagging_jobs` for in-flight rows first. Sweep + comp settlement only run here.
apps/web	CI on merge	—
Migrations	257 (table + cost op type), 258 (refund cols), 259 (accept/comp/file_id + CHECK + `files.flyer_job_id`), 260 (`human_verifications` jsonb — per-criterion user attestations)	All applied to prod 2026-08-21/22. Apply before code that writes the columns.

Smoke test after a Lambda deploy (must return route, no errorMessage):

aws lambda invoke --function-name flyer-tagger --region us-east-1 \
  --cli-binary-format raw-in-base64-out \
  --payload '{"preflight_only": true, "inputKey": "<any real users/... key>"}' /tmp/out.json \
  && python3 -c "import json; d=json.load(open('/tmp/out.json')); print(d.get('route'), d.get('errorMessage'))"

Alt-text review/edit (#2164)

GET /api/v1/flyer/jobs/:id/figures and PUT .../figures/:index (owner-only, delivered jobs) invoke the tagger Lambda synchronously with {"op": "list_figures"|"set_alt", "key": <output_key>} (flyer_tagger/altedit.py). Figure identity is positional (depth-first structure-tree order — stable across pikepdf save cycles). set_alt rejects empty alt (PDF/UA-1 7.3), edits only the structure tree (pixels untouched by construction), qpdf-checks, and only then overwrites the stored object. Thumbnails are PyMuPDF crops of the figure’s /A /BBox, returned as data URIs.

OCR-mode figures get their /A /BBox from the vision analysis directly (#2166): the compact wire shape carries b = the image region on Figure elements (text lines can’t supply one — OCR figures have none). A missing or invalid region is a validation problem in the retry loop, but the final-attempt fallback degrades it to no-thumbnail (region_dropped) rather than failing the page — regions are cosmetic, never gated. Pictorial pages use the whole page as the region. Jobs tagged before #2166 have no /BBox on OCR figures and show the “No preview” placeholder.

Verification gate

evaluate() in services/flyer-tagger/flyer_tagger/cli.py fails a run on: untagged text ops, unclaimed text ops (model omitted a matched line — added PR #2146), orphaned MCIDs (a mark in the content stream no structure element references — added with the #2155 multi-page fix; independent of veraPDF), pixel identity <99.95%, qpdf errors, veraPDF unavailable/non-compliant (absence of the validator is a failure, never a pass). Failed output goes to quarantine/<key>, never the delivery key.

Multi-page documents: ONE StructureContext accumulates the tree, ParentTree keys, and heading-clamp state across pages (#2155 — a fresh tree per page previously left every earlier page’s marks orphaned). Heading levels are clamped document-wide (first heading → H1, no skipped levels; PDF/UA 7.4.2).

Troubleshooting

Choice card never appears: user must be authenticated; preflight fails open to normal conversion on any error (check [flyer/preflight] logs in the API composition serving the request).
Job stuck: sweep resolves + refunds within ~10 min of the staleness cutoff. If the sweep itself errors, [flyer-sweep] in .4 api-node logs.
User says they were charged for nothing: check the row’s resolution columns; a NULL-all-three terminal non-delivered row older than the grace period means the sweep/poll refund is failing — look for CLAIM ROLLBACK THREW / refund_entitlement failed.
Cost anomalies: cost_ledger operation_type='flyer-tagging'; metadata.unpriced_model=true flags a model id the rate table didn’t recognize.
Debug a specific PDF locally: services/flyer-tagger/.venv + python -m flyer_tagger.cli <in> <out>; preflight only: run_preflight() (see README). Model A/B: tools/eval_models.py.

Known limits / open work

#2144 — SHIPPED: AcroForm fields are tagged in place (<Form> + OBJR + /TU from adjacent labels; /Tabs /S; fields stay fillable). forms retired as a route: XFA is the only form-related hard bar; digital signatures surface as signed (consent warning — tagging invalidates validation); high field density surfaces as formCentric (choice card adds a Gevell Forms offer). Fixture generator: services/flyer-tagger/tools/make_form_fixture.py.
#2148 — shipped: OCR + ActualText mode for flattened/scanned flyers. Preflight classifies flattened design-dense rasters as mode: 'ocr' (route stays in-place); the vision pass recognizes text with per-line confidence; structure attaches as /ActualText over the untouched raster. Two new gate failures — OCR coverage and low-confidence OCR — are compliance-only (accept / comped-reflow choice, never auto-refund). Confidence threshold: FLYER_TAGGER_OCR_MIN_CONF (default 0.80) on the Lambda.
#2118 — opt-in minimal visual fixes (small text / contrast are report-only advisories today).
Haiku canary — parked pending quality; procedure in services/flyer-tagger/README.md Cost section.