Skip to content
AnglinAI Docs

Bedrock setup for ACR/VPAT parsing (#1192)

Brings the vpat-parse worker’s Bedrock path online with zero data retention. Zero retention is now always on for every account — the evaluator sends disableDataRetention: true on every parse, so with AWS creds present the worker routes all ACR traffic to Bedrock and fails closed (HTTP 409) if creds are missing. This runbook is the AWS-side + Cloudflare-side configuration; the creds in step 5 are therefore mandatory in production, not optional.

Region: us-east-1. Default eval model: amazon.nova-lite-v1:0.

Prereqs (already done)

  • Claude Haiku 4.5 enabled account-wide (use-case submitted + one playground invocation). Amazon Nova auto-enables on first invocation — nothing to do.

1. Set account-level zero data retention

No console UI — API only. Requires an admin credential with bedrock:PutAccountDataRetention (NOT the worker key).

Terminal window
curl -X PUT https://bedrock.us-east-1.amazonaws.com/data-retention \
  -H "Authorization: Bearer $AWS_BEARER_TOKEN_BEDROCK" \
  -H "Content-Type: application/json" \
  -d '{ "mode": "none" }'


# verify
curl https://bedrock.us-east-1.amazonaws.com/data-retention \
  -H "Authorization: Bearer $AWS_BEARER_TOKEN_BEDROCK"

With none, any model that requires provider_data_share (Fable 5 / Mythos 5) returns status: unavailable — the intended fail-safe. Nova + Haiku 4.5 are pre-Fable and support none, so they run normally.

Optional org-wide lock (SCP): deny setting any mode other than none on bedrock:PutAccountDataRetention.

2. Confirm model invocation logging is OFF

Console → Bedrock → Settings → Model invocation logging. Off by default; if on, it writes prompts to CloudWatch/S3 and defeats zero retention on the classic Converse path. Confirm disabled.

3. Create the worker IAM credential (least privilege)

Programmatic IAM user; attach workers/vpat-parse/bedrock-iam-policy.json (replace ACCOUNT_ID). It can ONLY invoke the eval models — not change retention or logging. Save the access key + secret.

Start with the Nova statement only. Add the Haiku inference-profile statement when you add Haiku to the eval. If a bare model ID returns ValidationException: ...must use an inference profile, use the us. form.

4. Validate locally BEFORE touching the worker

One Converse call with the new key confirms enablement + IAM in isolation:

Terminal window
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... \
aws bedrock-runtime converse \
  --region us-east-1 \
  --model-id amazon.nova-lite-v1:0 \
  --messages '[{"role":"user","content":[{"text":"Reply with the word OK."}]}]'

Expect a short text response. Repeat with --model-id us.anthropic.claude-haiku-4-5-20251001-v1:0 when validating Haiku.

5. Load creds into the worker

Terminal window
cd workers/vpat-parse
wrangler secret put AWS_ACCESS_KEY_ID --env production
wrangler secret put AWS_SECRET_ACCESS_KEY --env production

AWS_REGION and BEDROCK_MODEL_ID are already in wrangler.toml [env.production.vars].

Routing (important): zero retention is universal, so every request now carries disableDataRetention: true and routes to Bedrock whenever creds are present — LLM_PROVIDER no longer gates general traffic and can stay unset. The creds are required: without them the worker returns HTTP 409 rather than fall back to a retaining provider. The Gemini/Anthropic path remains only as the documented default for callers that don’t request zero retention (none today).

6. Deploy + end-to-end check

Terminal window
npm run deploy:production
  • Sign in with any account and run a real ACR → confirms it parses via Bedrock/Nova under zero retention (check the X-LLM-Provider: bedrock response header). Non-internal accounts always use Amazon Nova Lite.
  • Sign in with a @theaccessible.org account → the Model (eval) picker appears in Settings; switch it and re-parse to confirm the chosen model handles the request (verify via the X-LLM-Model header).

7. Run the model eval

Cycle BEDROCK_MODEL_ID across amazon.nova-lite-v1:0, amazon.nova-micro-v1:0, us.anthropic.claude-haiku-4-5-20251001-v1:0 and the current Gemini baseline; score extraction accuracy, valid-JSON rate, hallucination rate, latency, and $/doc. Publish the comparison here.

8. Cost & security guardrails

Scope the worker IAM key to only the eval models (least privilege):

Terminal window
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
aws iam list-attached-user-policies --user-name vpat-parse-bedrock   # detach any broad managed policy
aws iam put-user-policy --user-name vpat-parse-bedrock --policy-name bedrock-invoke \
  --policy-document "$(cat workers/vpat-parse/bedrock-iam-policy.json | sed "s/ACCOUNT_ID/$ACCOUNT/")"

The worker also allowlists the same three model IDs, so IAM scope and app scope agree.

Bedrock budget (the console service filter won’t list Bedrock until ~24h after first charges; the API accepts it immediately):

Terminal window
aws budgets create-budget --account-id "$ACCOUNT" \
  --budget '{"BudgetName":"Bedrock-Monthly","BudgetLimit":{"Amount":"25","Unit":"USD"},"TimeUnit":"MONTHLY","BudgetType":"COST","CostFilters":{"Service":["Amazon Bedrock"]}}' \
  --notifications-with-subscribers '[{"Notification":{"NotificationType":"ACTUAL","ComparisonOperator":"GREATER_THAN","Threshold":80,"ThresholdType":"PERCENTAGE"},"Subscribers":[{"SubscriptionType":"EMAIL","Address":"larry@anglin.com"}]}]'

Optional hard cap: add a Budget Action that auto-applies a Deny policy to vpat-parse-bedrock at the ceiling (mirror anthropic-spend-killswitch.md).

Status (2026-06-17)

  • ✅ data_retention_mode = none set account-wide.
  • ✅ Production live on the internal-only path (LLM_PROVIDER unset).
  • ⏳ Pivot to universal zero retention (every account, toggle removed) — pending deploy. After this ships, AWS creds are mandatory or all parsing 409s.
  • ✅ All three models verified end-to-end in prod: Nova Lite, Nova Micro, Claude Haiku 4.5 (via us. inference profile).
  • ✅ Bedrock budget + alerts configured.
  • ⏳ TODO: scope the IAM key to eval models; run the model bake-off; investigate the wrangler secret list env-topology discrepancy.

Rollback

Set LLM_PROVIDER back to gemini (or unset) and redeploy — instant revert to the Gemini-primary path. No data migration involved. To fully disable Bedrock, also remove the worker’s AWS secrets.