Skip to content
AnglinAI Docs

The Document Accessibility Census — Research Study Plan

Internal. Compiled 31 August 2026. Companion to the customer document scanning plan; same pipeline, pointed outward.

The premise

Nobody has measured document accessibility in American higher education or state government at scale. Web pages have been measured for years — Pope Tech has scanned every US higher-ed institution’s top-level site annually since 2019, and the WebAIM Million covers the top million home pages. There is no document analogue. Institutions are staring down ADA Title II deadlines that explicitly cover electronic documents, and nobody can tell them how big the problem is.

We can, because we already built the machine.

The finding worth publishing

Field experience suggests roughly half of failing PDFs have structural-only failures — the content is fine, the tag tree is broken or missing — and are therefore repairable by automation with no author involvement at all.

If that holds at scale, it is a genuinely newsworthy number and it reframes the entire remediation conversation. Institutions are budgeting as though every failing document needs a human. The claim to test:

Half of the inaccessible documents in American higher education can be fixed without anyone writing a word.

Treat this as a hypothesis, not a headline. If the real number is 30% or 70%, that is still the first measurement anyone has published and it is still the most useful thing in the report. Publish whatever we find. A study that confirms a vendor’s prior is worth nothing; a study that reports an inconvenient number is worth a great deal.

Why this is the highest-leverage marketing we can do

  1. The data does not exist. We would own the category-defining number.
  2. It runs on the product we’re building anyway. Marginal cost is compute and patience.
  3. It establishes authority on our exact differentiator — documents, and the structural/content split — rather than on generic WCAG scanning where we’re one of forty vendors.
  4. Every institution in the sample becomes a warm lead who wants to know their own number.
  5. Repeated annually it compounds. Longitudinal data is what gave Pope Tech durable standing. A one-off study is a press release; year three is an institution.

Sample design

Frame

Two populations, run as separate reports:

  • Higher education. Degree-granting US institutions from IPEDS. Roughly 4,000 in the frame.
  • State and local government. State agencies plus a sample of counties and municipalities, weighted toward the entities facing the April 2027 Title II deadline (population 50,000+).

Start with higher ed. It’s our primary market, the frame is cleaner, and Laura’s relationships are there.

Institution sample

Stratified random sample, not a census. A census of 4,000 institutions’ document libraries is a different order of magnitude from a census of their home pages, and the marginal information from institutions 500 through 4,000 is small.

Target 300–500 institutions, stratified by:

  • Control (public / private nonprofit / private for-profit)
  • Level (4-year / 2-year)
  • Enrollment band
  • Carnegie classification
  • Census region

Stratification matters because the interesting cross-tabs are the story: do better-resourced institutions actually do better? Do community colleges lag? Is there regional variation tracking state-level policy?

Document sample within an institution

This is where the methodology can go wrong and nobody will notice.

Do not take the first N documents the crawler finds. That biases toward homepage-adjacent marketing PDFs, which are newer, better-produced, and more likely to pass than the accreditation reports and course syllabi buried four levels deep.

Instead:

  • Crawl to a fixed depth, enumerate the full document set, then sample randomly across that set, stratified by site section and crawl depth
  • Cap at 300–500 documents per institution so one large university doesn’t dominate the aggregate
  • Weight the aggregate by institution, not by document, so a school with 40,000 PDFs doesn’t outvote forty schools with 1,000 each
  • Record the discovered count alongside the sampled count — the size of institutional document libraries is itself a publishable finding

Expected corpus: roughly 100,000–250,000 documents.

What we measure

Tier 0 and Tier 1 only. No Tier 2 at study scale — the cost doesn’t justify it, and machine-verifiable findings are the defensible part anyway. Be explicit about that limitation in the methodology.

Per document

FieldSourceWhy it matters
Has StructTreeRootTier 0Tagged at all
/MarkInfo /MarkedTier 0
Document /LangTier 0Trivially fixable, near-universally missing
Title + DisplayDocTitleTier 0Same
Text layer presentTier 0Image-only scans are Class C
Encrypted / permission-lockedTier 0Both a barrier and a scanning obstacle
Page count, file sizeTier 0Drives the cost model and the long tail
/Producer stringTier 0The sleeper finding — see below
Creation / modification dateTier 0Is it improving over time?
PDF/UA-1 pass/failTier 1Headline conformance rate
Failed clause listTier 1Input to classification
Assigned class (A/B/C/D)ClassifierThe whole point
Near-miss flagClassifierClass A but for 1–2 content items

Aggregate outputs

  • Overall PDF/UA-1 conformance rate
  • Structural vs content split — the headline
  • Near-miss share: how many documents are one alt text away from automation
  • Image-only share (the OCR problem)
  • Distribution of document library size per institution
  • Page-count distribution and the very-large-document tail
  • Producer analysis: failure rate by export toolchain
  • Age analysis: are newer documents better?
  • Cross-tabs by institution type, size, control, region
  • Estimated aggregate remediation exposure at market rates, and the share avoidable through automation

The producer analysis is the sleeper

If we can report that documents from one export path fail at 94% while another fails at 12%, that is actionable at the source. It tells institutions to fix the template and the workflow rather than remediating one document at a time. It is a genuine public good, it is the kind of finding that gets cited, and no vendor-neutral party is positioned to produce it.

It also generates a natural follow-on: publish guidance on the export settings that produce conformant output from the worst offenders.

Ethics, legality, and not becoming the story

We are the accessibility company. Getting called out for aggressive crawling or for publicly shaming universities would be a self-inflicted wound that outlasts the study.

Crawling conduct

  • Honour robots.txt without exception
  • Conservative rate limits, well below what the target can handle
  • Identify the crawler by user agent, with a URL explaining the study and a contact address
  • Publish an opt-out mechanism and honour it immediately, no questions

Data handling

  • Never retain the documents. Fetch, probe, validate, discard. Retain findings, hashes, and metadata only. This keeps us clear of copyright entirely — we are measuring structure, not reproducing content — and avoids accidentally warehousing whatever PII sits in a university’s public PDFs.
  • Publish no document text, no excerpts, no screenshots

Publication posture

  • Aggregate and category-level findings public. Individual institutions never named in the public report. Naming and shaming your own buyers is a strategy with one use.
  • Offer each sampled institution its own results privately, free. This is the ethical position and the lead-generation engine, and the two happen to coincide. Send it as a service, not a pitch.
  • Publish the full methodology, the sampling frame, the clause-to-class mapping table, and the aggregate dataset. The findings only carry weight if someone hostile can check them.
  • State our commercial interest plainly at the top. Everyone will work it out anyway; disclosing it first is the only version where we keep credibility.

Human subjects: none, so no IRB. Say so, and say why, so nobody wonders.

Cost and schedule

The cost is wall clock and politeness, not dollars.

  • Discovery. Link extraction rarely needs full Chromium — plain HTTP plus HTML parsing handles most sites. Reserve rendered crawls for JS-heavy sites detected by a cheap heuristic. This matters at 400 institutions.
  • Download. Roughly 100k–250k documents at a few MB each. Ingress is cheap; the binding constraint is the rate limit we impose on ourselves.
  • Tier 0. Milliseconds per document. Negligible.
  • Tier 1 veraPDF. A few CPU-seconds per document. At 250k documents that’s on the order of a couple hundred CPU-hours — tens of dollars on spot instances.
  • Tier 2. Not run.

Plan for a six-to-ten week run, not a weekend. Polite crawling of 400 institutions is inherently slow, and rushing it is exactly how we end up in someone’s abuse report.

Publication plan

  1. Methodology preprint and dataset release first, so the numbers land with their workings attached.
  2. The report itself — a short executive version and a full technical one. Both accessible, obviously, and both available as HTML rather than a PDF. (A PDF-only report about PDF accessibility would be the funniest possible own-goal. Offer a conformant PDF as a secondary format and let it be the demonstration.)
  3. Conference talks, not booths: Accessing Higher Ground, EDUCAUSE, CSUN. The study is the talk.
  4. Trade press: EDUCAUSE Review, Inside Higher Ed, Campus Technology.
  5. Private institution reports to all sampled institutions, in a coordinated wave after publication.
  6. Annual repeat. Commit to it publicly. The longitudinal series is the asset.

Dependencies

This study cannot start before:

  • The clause-to-class mapping table exists and is reviewed (customer plan, step 1)
  • Tier 0 and Tier 1 run at queue scale (steps 2–5)
  • The fixture corpus validates the classifier, because the classification is the finding — if the structural/content split is wrong, the study is worthless and publicly so
  • Large-document caps and the failure taxonomy are in place

Everything above is on the critical path for the customer product anyway. The study adds sampling logic, crawler etiquette, and analysis — not new pipeline.

Open decisions

  1. Sample size. 300 institutions or 500? The marginal precision from 500 is small; the marginal warm leads are not.
  2. Higher ed and government in one report or two? Two, probably — different audiences, different deadlines, and two publication moments instead of one.
  3. Do we name institutions that do well? A “top performers” list is defensible, generates goodwill, and creates an incentive. A bottom list does not. Consider recognising the top decile by name with permission.
  4. Timing against the April 2027 Title II deadline. Publishing roughly six to nine months ahead puts the number in front of institutions while they still have budget cycles left to act on it.