Bedrock Zero-Data-Retention (ZDR) Rollout Roadmap

Goal

Today, Amazon Bedrock is wired into exactly one path — VPAT/ACR parsing (workers/vpat-parse, issue #1192). Every other LLM call goes directly to the Anthropic API, the Google Gemini AI-Studio API, or the OpenAI API, none of which give us a contractual Zero-Data-Retention guarantee by default.

The objective of this roadmap is ZDR on every LLM path, with the PDF conversion pipeline as the first and highest-value target. ZDR means: the provider does not persist prompt/response content after serving the request and does not use it for training.

What “ZDR via Bedrock” actually requires

ZDR is not a code change — it is an AWS account posture. The code change is only “route the request through Bedrock instead of the direct provider.” The guarantee comes from two AWS settings (already done for the VPAT account, see bedrock-vpat-setup.md):

1. bedrock:PutAccountDataRetention → mode: none (account-wide).
2. Model invocation logging OFF — otherwise prompts land in CloudWatch/S3 and defeat ZDR.

Critical model constraint already discovered in #1192: models that require provider_data_share (Fable 5 / Mythos 5 tier) return status: unavailable under mode: none. Pre-Fable-5 Claude models (Sonnet 4.x, Opus 4.x, Haiku 4.5) and Amazon Nova support none. Any path we migrate must use a ZDR-eligible model on Bedrock.

The hard problem: Gemini and OpenAI have no Bedrock equivalent

This is the single most important decision in the whole roadmap. Bedrock hosts Anthropic Claude, Amazon Nova, Meta Llama, Mistral, etc. — it does not host Google Gemini or OpenAI GPT. So “ZDR on all paths” forces a choice on every Gemini/OpenAI call site:

Option	ZDR?	Effort	Notes
A. Re-point to a Claude/Nova model on Bedrock	✅ (Bedrock account ZDR)	Medium — re-prompt + re-tune per task	Cleanest. Consolidates everything onto one ZDR provider. May change output quality/cost per task.
B. Move Gemini to Vertex AI with data-governance ZDR	✅ (Google enterprise ZDR)	Low code (the getGeminiClient factory already supports Vertex)	Conflicts with a standing rule: this repo is “Gemini AI-Studio API key only; never re-enable USE_VERTEX_AI.” Needs an explicit reversal of that decision to use.
C. OpenAI ZDR addendum / OpenAI on Azure	✅ (contractual)	Low code	Keeps GPT-4o-mini; requires a zero-retention agreement with OpenAI. Adds a third ZDR vendor to manage.
D. Leave as-is	❌	None	Not acceptable for the stated goal.

Recommendation: default to Option A (consolidate to Bedrock) for the conversion pipeline, because it gives one auditable ZDR story (“everything runs on our ZDR Bedrock account”) instead of three separate vendor agreements. Hold Option B/C in reserve for the handful of tasks where a Gemini/GPT model is materially better and re-prompting to Claude/Nova regresses quality.

Decision needed before Phase 2 — confirm A vs B for the Gemini-heavy course-map and image-enhancement paths. This is a business/quality call, not a purely technical one.

Current-state inventory (what we are migrating)

Already on Bedrock

• workers/vpat-parse/src/worker.ts — Converse API via aws4fetch + SigV4, static AWS keys as Wrangler secrets. Fails closed (HTTP 409) if ZDR required but Bedrock unavailable.

Anthropic direct → Bedrock (easy: same models exist on Bedrock)

~25 call sites in workers/api/src, each constructing new Anthropic({ apiKey }) inline. Highlights: claude-converter, agentic-vision-converter, opus-page-converter, unpdf-claude-converter, alt-text-suggester, opus-jury, verification-prograder, css-designer, ux-optimizer, ai-fix, quality-gate, region-refiner, image-region-recovery, vision-table-extractor, all org-chart/*, plus routes/support-chat.ts, routes/campaigns.ts, the segment-cli tool, and the exemption-review package.

Gemini direct → needs A/B decision

workers/api: gemini-converter, image-ocr, image-enhancer, qr-alt-text-enhancer, vision-equation-audit, visual-polisher, wcag-visual-checker, wcag-ai-quality-checker, long-description-generator, prompt-optimizer, ai-content-suggester, plus the Gemini branch of smart-cascade-converter and mistral-ocr-converter. workers/course-map-api: entirely Gemini (gemini.ts, plan-extract.ts, llm-course-extract.ts, gemini-onboarding-discovery.ts, catalog-lookup.ts, map-quality.ts, catalog-recon.ts). workers/remediate: Gemini via the Vercel ai SDK.

OpenAI direct → needs A/C decision

smart-cascade-converter (gpt-4o-mini vision tier), image-enhancer.

Architectural facts that make this easier

• Gemini already has a factory (workers/api/src/utils/gemini-client.ts) — one chokepoint for all workers/api Gemini calls. Course-map and remediate have their own.
• Anthropic has NO factory — 25 inline new Anthropic() calls. This is the main refactor cost, but it’s mechanical.
• workers/api runs on AWS Lambda (prod) + EC2 Node (10.1.1.4). Both can carry an IAM role, so Bedrock from here needs no static keys — unlike the CF worker, which had to use aws4fetch with secret keys. Use the official @anthropic-ai/bedrock-sdk, which mirrors messages.create() 1:1.
• Cost table (workers/api/src/utils/llm-cost.ts) is centralized — add Bedrock model IDs/pricing here once.

---

Phased plan

Phase 0 — Foundations (1–2 days, no behavior change)

1. Confirm the Bedrock ZDR account posture covers the AWS account that Lambda/EC2 use (the VPAT worker may use a different account/keys — verify, don’t assume). Re-run the PutAccountDataRetention + logging-off checks from bedrock-vpat-setup.md against the conversion account.
2. Add IAM bedrock:InvokeModel (+ inference-profile ARNs) to the Lambda execution role and the EC2 instance role. No static keys.
3. Add Bedrock model IDs + pricing to llm-cost.ts (Sonnet 4.x, Opus 4.x, Haiku 4.5, Nova-lite/micro inference-profile IDs).
4. Build a provider abstraction. Create utils/anthropic-client.ts mirroring the existing gemini-client.ts:
   • getAnthropicClient(config) returns either a new Anthropic() (direct API) or new AnthropicBedrock() (@anthropic-ai/bedrock-sdk) — both expose the same .messages.create(), so call sites are untouched below this line.
   • anthropicConfigFromEnv() reads a new LLM_BACKEND flag (anthropic | bedrock, default anthropic) plus AWS_REGION.
   • Add an optional disableDataRetention concept that fails closed (mirror the vpat-parse #1192 pattern) — never silently fall back to a retaining provider when ZDR was requested.

Phase 1 — PDF conversion pipeline, Anthropic call sites (the priority) (3–5 days)

Migrate the Claude-based converters and quality gates to route through the new factory. These are the highest-value paths and the lowest risk (same models on Bedrock):

1. Replace every inline new Anthropic({ apiKey }) in workers/api/src/services and routes with getAnthropicClient(...). Mechanical; one PR per cluster (converters / quality-gates / org-chart / routes).
2. Flip LLM_BACKEND=bedrock in a staging/canary Lambda alias first.
3. Verify per converter: output parity, latency (Bedrock cold-start + inference-profile routing), and cost (recorded via existing llm-cost tracking). Watch opus-page-converter specifically — it uses the Anthropic Files API, which Bedrock does not support; rework it to inline base64 / a different upload path before migrating, or leave it on direct API behind the flag and document the gap.
4. Canary % rollout → 100%. Keep the flag so we can revert instantly.

Phase 2 — Gemini & OpenAI conversion paths (depends on A/B/C decision) (1–2 weeks)

1. Apply the Phase-0 decision per task. For Option A, port the Gemini/OpenAI prompts in smart-cascade-converter, image-enhancer, image-ocr, visual-polisher, the WCAG checkers, etc. to Claude/Nova on Bedrock and re-baseline quality against the existing eval corpus.
2. For any task kept on Gemini/OpenAI, implement the chosen ZDR mechanism (Vertex data-governance, or OpenAI ZDR addendum) and document why it stayed.
3. Re-run the conversion benchmark suite; gate rollout on no quality regression.

Phase 3 — Course-map, remediate, support, campaigns, CLI/library (1 week)

1. workers/course-map-api is 100% Gemini — biggest Option-A re-prompt effort, but lower stakes than conversion. Migrate behind its own flag.
2. workers/remediate (CF Worker) — same posture as vpat-parse: it can’t use an IAM role, so it needs the aws4fetch + static-key approach, OR proxy through the Node API. Prefer proxying through workers/api so all Bedrock traffic flows through the IAM-role’d account and we don’t manage more static keys at the edge.
3. support-chat / campaigns (Haiku, CF Worker) — same edge decision.
4. CLI (segment-cli) and the exemption-review package — migrate via the same factory; low volume, do last.

Phase 4 — Enforce & verify (ongoing)

1. Make ZDR the default: flip LLM_BACKEND=bedrock as the baseline, keep anthropic direct as the break-glass fallback only.
2. Add a CI/lint check that forbids new inline new Anthropic() / new GoogleGenAI() / new OpenAI() outside the factory files — prevents regressions back to non-ZDR paths.
3. Add a startup assertion + /health field reporting the active backend and ZDR posture, so a misconfiguration is visible.
4. Update the public ZDR marketing claim (currently VPAT-only in apps/vpat) to cover the conversion product once Phases 1–2 ship — only after verification, not before.

---

Risks & open questions

• Model availability under mode: none — re-confirm each target model is ZDR-eligible on Bedrock (the Fable-5 provider_data_share trap). Do this in Phase 0, not at rollout.
• Files API gap (opus-page-converter) — no Bedrock equivalent; needs rework or a documented carve-out.
• Quality regression on Gemini→Claude re-prompts — the Gemini paths were tuned for Gemini; budget eval time, don’t assume parity.
• Two AWS accounts? — verify whether vpat-parse’s Bedrock keys and the Lambda/EC2 account are the same; ZDR posture is per-account.
• Edge workers (remediate, support-chat, campaigns) can’t use IAM roles — decide proxy-through-API vs static-key SigV4 early.
• Cost shift — Bedrock per-token pricing differs from direct API and from Gemini’s much cheaper rates; the consolidation may raise spend on high-volume paths (alt-text runs up to 80 parallel calls/page). Track via the existing cost ledger and confirm acceptable before 100% rollout.

Suggested issue breakdown

• #xxxx Phase 0: Bedrock account posture + IAM + anthropic-client.ts factory
• #xxxx Phase 1: Migrate Claude converters/quality-gates to factory + canary
• #xxxx Decision: Gemini/OpenAI ZDR strategy (A consolidate vs B Vertex vs C addendum)
• #xxxx Phase 2: Conversion Gemini/OpenAI paths
• #xxxx Phase 3: course-map / remediate / support / campaigns / CLI
• #xxxx Phase 4: enforce default + lint guard + health/marketing