Skip to content
AnglinAI Docs

Silent Failure Audit — 2026-08-31

Automated read-only audit. No source code was modified.

Summary

MetricValue
Files scanned (changed, last 7 days)299
Files scanned (random unchanged sample)10
Total catch blocks analysed528
Catch blocks with no log / no throw / no propagation211
Of those, wrapping a critical operation (DB / R2 / AI / fetch / email / SQS)47
Confirmed findings after manual triage7

Findings by severity: 0 critical · 1 high · 4 medium · 2 low

Method

  1. git log --since="7 days ago" --name-only → 405 paths → 300 source files (.ts/.tsx/.js/.mjs/.py, excluding tests, dist/, .next/, node_modules) → 299 still present on disk.
  2. Brace-matched every catch body (not line-regex) and classified it as logging / throwing / propagating / silent.
  3. Cross-referenced each silent catch against the preceding try body for critical operations (supabase, .from(, insert(, rpc(, R2/S3 puts and deletes, fetch(, Anthropic/Bedrock/Gemini calls, recordCost, resend, SQS, execFile).
  4. Manually read every one of the 47 hits; discarded those that log via the bare log('error', …) helper (the initial regex only matched log. / console.) or that are documented deliberate fail-open paths.
  5. Separately scanned for empty catches, floating promises, .then() chains without .catch(), and console.* in place of the structured logger.
  6. Random unchanged sample: 10 files from workers/api/src/routes/ and workers/api/src/utils/.

Good news up front: there are zero empty catch blocks (catch {} / catch (e) {}) anywhere in the scanned set, and the random unchanged sample came back clean — its 3 silent catches are all JSON.parse of a KV cache falling back to a recompute, which is correct.

HIGH

H1 — ZIP export silently omits images that fail to load from R2

File: workers/api/src/routes/export.ts:1011-1021

const fetched = await Promise.all(
  imageRefs.map(async (filename) => {
    const r2Key = `${R2_PATHS.images(userId, fileId)}/${filename}`;
    try {
      const obj = await storage.objects.get(r2Key);
      if (!obj) return null;
      const buf = await obj.arrayBuffer();
      return { filename, data: new Uint8Array(buf) };
    } catch {
      return null;                      // ← swallowed
    }
  }),
);
zipImages = fetched.filter((f): f is NonNullable<typeof f> => f !== null);

zipImages is then zipped and returned with HTTP 200 and no comparison against imageRefs.length (line 1037 onward). A transient R2 error, or a key that has drifted, produces a ZIP whose index.html references images/foo.png that is not in the archive — broken images in the customer’s deliverable, with no error logged server-side and no signal client-side.

This is worse than a generic missing asset: the images carry the alt text that the product exists to produce, so a partial ZIP is an accessibility regression shipped as a success.

Tracked as: #2483

Severity rationale: silent, user-facing, quality-degrading, on the primary paid deliverable path, and undetectable after the fact because nothing is logged.

Recommended fix: log each failure with the R2 key, and compare zipImages.length to imageRefs.length after the Promise.all. On a mismatch either fail the request with a retryable error code, or include a MISSING-IMAGES.txt manifest in the ZIP so the gap is visible. At minimum, log('error', …) the key so the failure is greppable in Loki.

MEDIUM

M1 — Workshop registration page has no .catch() on the live catalog fetch

File: apps/home/src/app/register/page.tsx:50-52

fetch(`${API_BASE}/api/workshops/catalog?upcoming=1`, { headers: { accept: 'application/json' } })
  .then((r) => (r.ok ? r.json() : null))
  .then((j) => { … setOptions(buildOptions(ws, bs)); });
  // no .catch()

Falling back to the build-time WORKSHOPS constant is deliberate (it is set first, on line 48). The defect is the missing rejection handler: a network failure or a JSON parse error becomes an unhandled promise rejection, and the registration screen silently shows build-time workshop dates and prices on a Stripe checkout path. Stale pricing on a page that takes money is worth a console signal at minimum.

Fix: add .catch((err) => reportClientError(err, { context: 'register:catalog' })). Consider surfacing a “showing cached schedule” note when the live fetch fails.

M2 — pdf-inventory host discovery returns an empty list indistinguishable from “no hosts”

File: tools/pdf-inventory/src/discover.ts:79-81 (crt.sh), and :103-105 (Common Crawl)

} catch {
  return [];        // crt.sh
}
…
} catch {
  return null;      // Common Crawl
}

Both third-party discovery sources are rate-limited in practice (the Common Crawl helper’s own comment says crt.sh “tends to” rate-limit). When one fails, the crawl silently proceeds with a smaller host set. The resulting inventory reports fewer PDFs than exist, and the report gives the operator no way to tell a genuinely small domain from a discovery outage — which is precisely the wrong failure mode for a tool whose output is a compliance inventory.

Fix: return a discriminated result ({ hosts, sourceFailed?: string }) and note failed sources in the run summary so under-coverage is visible.

M3 — pdf-inventory fetcher discards the network error entirely

File: tools/pdf-inventory/src/fetcher.ts:150-157

} catch (err) {
  last = { ok: false, status: 0, text: '', challenged: false, retries: attempt };
  void err;          // ← explicitly discarded
  return last;
}

void err deliberately throws away the only description of the failure. Downstream the row records status: 0, which conflates DNS failure, TLS rejection, connection refused, and timeout. The sibling code path at fetcher.ts:105 gets this right — it preserves err.message and distinguishes an aborted signal as timeout after Nms.

Fix: mirror line 105 — carry err instanceof Error ? err.message : String(err) into the result’s error field.

M4 — tag-pdf CLI has no rejection handler on its entry point

File: tools/tag-pdf/tag-pdf.mjs:298

runCli(process.argv.slice(2)).then((code) => process.exit(code));

The file documents Exit codes: 0 ok · 1 usage/error · 2 one or more files not delivered. If runCli rejects, none of those apply — the promise rejects unhandled and the documented contract is not met. This CLI is invoked from action-runner.mjs, so a rejection surfaces to CI as an opaque failure rather than exit code 1 or 2.

Fix: .then((code) => process.exit(code), (err) => { console.error(err); process.exit(1); }).

LOW

L1 — Cached ACR report deletion failures are ignored, orphaning R2 objects

Files: workers/api/src/routes/files.ts:1189 and workers/api/src/routes/files.ts:1399

if (fileMeta.acrReportR2Key) {
  try {
    await storage.objects.delete(fileMeta.acrReportR2Key);
  } catch (_err) { /* ignore if already gone */ }
}

Immediately afterwards putFileMeta clears acrReportR2Key, so a delete that fails for any reason other than “already gone” leaks an R2 object with no remaining reference — unbounded storage growth that no counter tracks. The comment justifies exactly one of the failure modes it actually swallows.

Correctness is fine (stale-cache invalidation still succeeds because the pointer is dropped), hence LOW.

Fix: catch (err) { log('warn', \orphaned ACR report ${key}: …`) }` so leaked keys are greppable and a future sweep can reclaim them.

L2 — 1,459 console.error / console.warn calls in workers instead of the structured logger

Files: across workers/api/src and workers/batch/src — e.g. workers/api/src/index.ts:439, workers/api/src/server.ts:376, workers/api/src/routes/files.ts:1506.

The global standard is “no console.log in production code — use the structured logger” (@anglinai/logger). These are not silent — Promtail scrapes container stdout, so they do reach Loki — but they arrive as unstructured text, missing requestId, userId, service, and error fields, so they cannot be filtered or alerted on the way structured entries can, and they never reach EmailAlertTransport.

Deliberately rated LOW: this is pre-existing architectural debt at a scale (1,459 sites) that no single PR should address, and the output is not actually lost.

Fix: adopt @anglinai/logger in workers/api incrementally — convert server.ts and index.ts top-level handlers first, since those are the ones whose failures currently page nobody.

Notable non-findings

Worth recording so future runs don’t re-litigate them:

  • workers/api/src/scheduler/chunk-scheduler.ts — 20+ catches initially flagged, all clean. Every one calls the module’s bare log('error'|'warn', …) helper with the error message, and several go further (chunk-scheduler.ts:526 explicitly annotates the orphan-recovery consequence in the log line). This file is the best error-handling in the scanned set.
  • workers/api/src/routes/convert.ts:130 — validateAnthropicKey returning null on a network error is correct and commented: a key-validation probe failing must not block conversion.
  • workers/api/src/services/admin-alerts.ts:37 — returning { enabled: true, channel: 'both' } when the preferences lookup fails is deliberate fail-open; the alert still goes out. The correct direction for an alerting path.
  • workers/api/src/routes/files.ts:1494 — the refresh-pdf regen chain has both a .then that handles non-throwing failure outcomes and a .catch, with the issue-#1793 rationale in a comment. Model example.
  • apps/web void fn() calls (13 sites) — all React effect bodies where the invoked async function does its own try/catch and sets error state. Not fire-and-forget.

Resource usage

Start time2026-08-31T07:14:20Z
End time2026-08-31T09:13:00Z
Duration~1h 59m
Tool invocations20 (18 Bash, 2 setup/write)
Files read in full or part12
GitHub Issues created1 — #2483 (H1)