Textract OCR lane

Recognizes flattened/scanned pages (preflight mode ocr) with Amazon Textract instead of a whole-page vision call. Module: services/flyer-tagger/flyer_tagger/textract_ocr.py (issue #3415).

Why

Measured 2026-09-21 on the usd.edu census corpus (169 image-only documents, 639 pages):

	vision OCR (OCR_PROMPT)	Textract LAYOUT
cost/page	~$0.033	$0.004
639 pages	~$21	~$2.56
determinism	self-correction retry loop	one call
page furniture	not detected	LAYOUT_FOOTER / LAYOUT_PAGE_NUMBER
figure localization	IoU 0.21–0.37	tight box, 97.1%
transcription	may “correct” the source	verbatim, 99.1% mean word confidence
column order	model-dependent	left column, then right

How it fits

Textract output is mapped to the compact OCR wire shape the pipeline already consumes, so nothing downstream changed:

render page -> Textract AnalyzeDocument(LAYOUT)
            -> blocks_to_compact()        # roles + geometry + confidence
            -> rejoin_hyphenated()        # line AND column breaks
            -> describe_figures()         # one cropped vision call per LAYOUT_FIGURE
            -> expand_ocr_analysis() -> validate -> adapt_ocr_analysis -> inject

• LAYOUT_HEADER / LAYOUT_FOOTER / LAYOUT_PAGE_NUMBER emit no element, which is how the region stays /Artifact (OCR mode artifact-wraps the raster and attaches structure as empty marked-content sequences).
• Figures: Textract locates, vision describes — the vision call is cropped to Textract’s box, which is cheaper and more accurate than a full page. A failed description ships the honest placeholder alt, flagged for review (#3352 policy), never a blank alt.
• Lists: Textract returns a LAYOUT_LIST’s members both under the container and as direct PAGE children, so the top-level walk suppresses list members and the LI branch re-emits them. Without that every item ships twice — as LI and again as P — which in OCR mode is duplicated /ActualText over the same raster (#2766).
• Tables: the page runs LAYOUT first; only a page where a LAYOUT_TABLE was actually detected is re-run with LAYOUT+TABLES. If that second call fails, the already-billed LAYOUT result is kept and the unresolved table is flagged for review rather than the page being treated as a Textract outage.

Enabling it — the order matters

Flag	FLYER_TAGGER_TEXTRACT_ENABLED=true (code default OFF; ON in production, see below)
Where	the flyer-tagger Lambda’s own env (no IaC; set out of band on the function) and the EC2 anchor’s host lane via infra/anchor/anchor.config.env (since #3865).
IAM	textract:AnalyzeDocument on the flyer-tagger role, added by deploy/deploy.sh
Region	AWS_REGION (default us-east-1)

This lane does NOT reuse PDF_TEXTRACT_ENABLED. That variable is already true in two other runtimes — the API Lambda (infra/cdk/lib/stacks/api-stack.ts) and the Node server .4 (.env.node-server), both since 2026-09-07 — and the flyer-tagger function’s environment is seeded out of band from that same .env.node-server file, while deploy/deploy.sh merges rather than replaces the env map. One wholesale env copy would therefore have enabled this lane, routing customer documents to a distinct third-party processor, with nobody deciding to. A separate flag name makes that accident impossible.

Order of operations

Run these in this order. The reverse order is an incident:

1. deploy/deploy.sh — puts textract:AnalyzeDocument on the role.
2. Then set FLYER_TAGGER_TEXTRACT_ENABLED=true on the flyer-tagger function.

Setting the flag first gives AccessDeniedException on every scanned job. That is deliberately classified as a misconfiguration (see below), so it fails loudly rather than degrading — a silent degrade would hide the broken deploy behind an 8x cost regression on every scanned page.

Observed state (checked 2026-10-03, account 696081601609)

The lane is ON. The first ledgered Textract page was 2026-09-25; the enable gate #3502 was closed 2026-10-03.

runtime	variable	observed
flyer-tagger Lambda	FLYER_TAGGER_TEXTRACT_ENABLED	true
EC2 anchor host lane	FLYER_TAGGER_TEXTRACT_ENABLED	true in infra/anchor/anchor.config.env since 2026-09-28 (#3865)
flyer-tagger role	textract:AnalyzeDocument	present since 2026-09-23 (deploy of 46feeac19)
API Lambda, EC2 anchor	PDF_TEXTRACT_ENABLED	true (Convert’s own Textract use; does not affect this lane)

Cost: each page’s Textract fee is booked to cost_ledger with its band on the host lane and with its chunk on the chunked lane (#4106), so the per-job cost guard sees it.

Check the observed value rather than assuming it:

aws lambda get-function-configuration --function-name flyer-tagger \
  --query 'Environment.Variables.FLYER_TAGGER_TEXTRACT_ENABLED'

Failure classes

With the flag off, the vision lane runs exactly as before. With it on, cli._recognize_ocr_page distinguishes three outcomes:

class	examples	behaviour
TextractUnavailable	throttling (after bounded backoff), 5xx, connection failure, no boto3, empty response, UnsupportedDocumentException / BadDocumentException	degrade to the vision lane with a WARNING — the old path can still do this page
TextractMisconfigured	AccessDeniedException, UnrecognizedClientException, no/expired credentials, InvalidParameterException	fail the job. A broken deploy must not be papered over
validation failure	a response that came back and does not validate	fail the job — a content problem must surface

An error code in neither set propagates unchanged rather than being guessed at.

Note in particular that “no credentials” is a misconfiguration, not a transient outage. Earlier drafts of this document and of the PR claimed it degrades gracefully; it does not, on purpose.

Cost

Per-page spend rides on analysis["_textract"] and is summed onto report["textract"]. deploy/handler.py:_record_cost adds it to the same cost_ledger row (metadata textract_pages / textract_cost_usd); a page with no figure-alt call has no tokens and is ledgered as model aws-textract, backend textract.

Rates are per call, not per page:

page	calls	billed
text only	LAYOUT	$0.004
a table detected	LAYOUT, then LAYOUT+TABLES	$0.004 + $0.015 = $0.019

Max-of-features applies within one call — it does not net out a call already made. The TS side has the matching per-call rates in workers/api/src/utils/llm-cost.ts (textract = LAYOUT+TABLES $0.015, textract-layout = LAYOUT $0.004).

The spend record is built as soon as the first billable call returns, so a page that fails afterwards still logs what was spent (and carries it on the raised exception as textract_spend) rather than losing the record.

Chunked jobs (>= FLYER_CHUNK_MIN_PAGES pages) ledger each chunk’s spend as that chunk finishes. The record stored to S3 is billed out by textract_ocr.strip_billed_textract — its cost is zeroed but its review flags are kept, so assemble cannot double-charge and still sets needs_review.

Compliance — prerequisite met, two obligations open

The AWS Organizations AI-services opt-out policy is attached (policy p-90qu1cuzca, all services optOut, child overrides disallowed, target r-sml2 of org o-jehmi38iqm, verified 2026-09-21).

What that control does: it tells AWS not to store or use this account’s Textract content for service improvement. It is an AWS Organizations control and is not the same mechanism as Bedrock’s zero-data-retention agreement. The two currently point the same way; they are not one posture and should not be described as one.

Two obligations remain open:

1. Textract is not in the subprocessor listing. docs/admin/security/data-flow-subprocessors.md lists AWS inference as Bedrock only and contains no occurrence of “Textract”. That document is a customer-facing HECVAT/procurement artifact, so the gap is real — tracked in #3176. Add the row before this lane processes customer documents in production.

2. The opt-out does not travel. It is attached at the org root and reaches member accounts by inheritance. If this deploy moves to an account outside o-jehmi38iqm, re-verify with:

   aws organizations describe-effective-policy \
     --policy-type AISERVICES_OPT_OUT_POLICY --target-id <account-id>

   Account-level list-policies-for-target returns [] here because the policy is attached at the root, not directly to the account. That empty list is not evidence of absence; describe-effective-policy is the check that matters. See docs/admin/products/pdf/textract-figure-detection.md.

CRM task 628225a8-a139-4396-993c-33471c20a9d6 must NOT be closed. The prerequisite it blocked on is satisfied, but the task is also the only durable record that one document (job 8bab5965) was sent to Textract on 2026-09-05 during evaluation — before the policy was attached on 2026-09-06. A policy attached afterwards does not cover a call already made.