UKMOD statistics adapter (mode 1): Country Report validation tables

[vahid-ahmadi]

Closes #38. Fifth UK mode-1 adapter (parent #3) — the peer-model lane. UKMOD is the closest peer microsimulation (FRS-based, EUROMOD platform), so these are the highest-signal model-vs-model rows the UK side can produce.

What's here

• sources/ukmod-stats/raw/ — the UKMOD Country Report 2023–2030 PDF (CeMPA WP 8/26, March 2026, version B2026.01) as fetched from ISER
• sources/ukmod-stats/adapter.py — pypdf-based parser (the one UK adapter needing a non-stdlib dependency, because the source is a PDF; declared and guarded) → 1,131 tidy rows at data/externals/ukmod-stats.json
• source.json + 7 annotations; data/lanes.json — ukmod-stats: registered → ingested

Coverage

• Tables 4.5/4.6 — simulated caseloads (thousands; per-row unit: families / children / households / individuals) and expenditure (£m) for 30 instruments × policy years 2023–2030: UC, child benefit (both families and children-covered), pension credit, HB, CTR, legacy benefits, Scottish benefits, benefit caps, NICs (employee/self-employed/employer), and income tax with the full devolved band breakdowns (rUK savers/basic/higher/additional; Scottish starter→top incl. the advanced rate from 2026)
• Tables 4.7/4.8 — Gini, mean/median and quintile medians/shares of equivalised BHC disposable income, and poverty rates below 50/60/70% of median by age group and gender

Claim provenance is carried as the variant: ukmod (simulated), official_as_cited (the report's own transcription of DWP/HMRC/OBR/SFC estimates), input_data, hbai. Missing cells are absent claims — skipped, not suppressed.

Parsing + validation

Text extraction with an ordered row registry anchored on UKMOD variable names (bsauc_s, tin00_s, …) and band labels; every row must consume exactly its expected token count, and any anchor miss fails the run. 12 checks pinned to the report's printed values pass (UC 6.941M simulated vs 6.493M official caseload 2025, £79.5bn official UC expenditure, Gini 0.32 UKMOD vs 0.33 HBAI, child poverty 19% UKMOD vs 20% HBAI, Scottish advanced-rate payers, …).

Honesty boundary

The annotations pin the decomposition contract: UKMOD shares PE UK's survey base but differs in weighting (HBAI-adjusted FRS 2023/24), uprating, and take-up corrections frozen at 2021 outside Scotland/pension-age — the report's own health warning — so PE-vs-UKMOD wedges route through data/uprating/take-up channels before indicting either engine. official_as_cited rows must resolve to primary publications before scoring (several are now sibling lanes: HMRC #45, OBR #46), with transcription mismatches becoming external-issue seeds. The 4.7/4.8 tables use the same BHC/modified-OECD construction as the hbai-poverty lane (#44), making the PE/UKMOD/HBAI triangle well-posed. Case-level adjudication stays with the mode-3 JRC-connector lane (#41). UKMOD is open source, so confirmed external-model issues here have an upstream path — unique among UK lanes.

🤖 Generated with Claude Code

[vahid-ahmadi]

Correction: 40 of Table 4.7's values were wrong

A review of this PR found a real data-corruption defect, now fixed in d6a1a5a/1f2e0f4. Recording it plainly.

What was wrong. parse_rows matched each row's anchor as a substring of whitespace-flattened page text. Table 4.7's quintile rows are anchored on bare digits ("2", "3", "4"), and those matched digits inside preceding numbers — the "2" anchor hit the trailing 2 of 1352, so quintile 2's window started two tokens early and swallowed the row label. The over-long token run was then silently truncated, because the length guard only checked the too-few direction.

cell	report	was emitted
q2 median, ukmod 2024	2,333	2.0 ← the row label itself
q2 median, ukmod 2023	2,238	1,527 (q1's HBAI value)
q3 median, ukmod 2023	2,869	8.0
q4 median, ukmod 2023	3,779	91.0
q3 share, all years	0.17	0.13

40 of Table 4.7's 130 values. All 12 spot-checks printed OK the whole time, because none of them touched those rows — that is the real lesson here, not the regex.

The fix. Anchors now match whole tokens, never substrings, and a numeric anchor must sit at the start of its line (a bare 2 mid-line is a data value, not a row label). Values are read from the anchor's own line and continue onto later lines only while those lines hold nothing but values — which keeps Table 4.6's wrapped benefit-cap rows working without letting a row run on into the next labelled row.

Tables 4.5, 4.6 and 4.8 are byte-identical after the rewrite (they had been machine-diffed against the PDF at 0 mismatches, and stay that way); only the 40 corrupt cells changed, and each now matches the report exactly, including the input-data and HBAI reference columns.

Defences added, since hand-picked spot-checks structurally cannot see a shifted row:

• structural_invariants() — per variant and year: quintile shares sum to 1, quintile medians increase, the middle quintile's median equals the population median, median ≤ mean. 40 invariants, all pass. They fire 27 times on the pre-fix data.
• tests/test_ukmod_adapter.py puts those identities plus hand-read PDF ground truth in CI's path (CI runs pytest and never touches sources/, so the adapter's own checks were invisible to it). 5 of the 7 new tests fail on the pre-fix output.
• pypdf is now imported lazily so the invariants are importable, and testable, without it.

Two smaller items from the same review, also fixed: expenditure rows carried the caseload registry's counting unit (families on a £ value) and now carry gbp_nominal; the 325 absent (-/na) cells are tallied and reported rather than silently skipped.

Downstream: PR #48 ingests this file, and 32 of the wrong values had reached the claim DB through it. No change to #48 is needed — it re-reads this output — but it should be re-run after this merges.