HMT distributional analysis (mode 2, distributional): Budget 2025 external side — chart-only scope, honestly

[vahid-ahmadi]

First slice of the HMT DA lane. The headline finding shapes the whole PR: HMT publishes no machine-readable decile values. The Budget 2025 "Impact on households" document is a 20-page PDF whose Figures 1.A–1.C are unlabeled chart bars; the supporting-documents page carries XLSX only for the costings tables (4.1/4.2), and no chart-data file exists. So this PR deliberately ships the external side without value claims rather than digitizing charts by eye.

What's here

• sources/hmt-distributional/raw/ — the real publication, fetched 2026-08-17 from the gov.uk assets URL, SHA-256 9f7e68f3…480ea pinned in README, source.json, the registry and the tests.
• data/uk/hmt_da_packages.yaml — the Budget 2025 package entry: all 30 in-scope measures verbatim from pp. 7–9, each with channel (16 tax / 9 welfare / 5 benefits-in-kind) and computability triage (3 expressible / 9 partial / 18 not_expressible, each partial naming what's missing and each not_expressible why), the published exclusions (employer NICs incidence, NLW, devolved decisions, behavioural effects…), the counterfactual (no policy changes since AB2024), horizon (2028-29) and income concept (equivalised net BHC). No invented reform dicts: expressible components reference measure families for the OBR policy costings (mode 2): Policy Measures Database + EFO 3.17 → PE-UK static counterparts #54/OBR policy costings (mode 2): registry, offline certified compute, staging, descriptive comparison #56 registry, so one measure keeps one executable home.
• sources/hmt-distributional/adapter.py — verifies rather than parses: SHA gate, page count, figure inventory, and every registry component title verbatim-anchored in the document text (normalized for PDF line breaks and typographic quotes), so the registry cannot drift from the publication. Emits a meta artifact recording value_claims_emitted: 0 and the value-availability rule. Runs clean against the vendored PDF.
• tests/test_hmt_da_adapter.py — raw-file identity, provenance files, the no-value-emission contract, meta honesty; registry schema checks via importorskip("yaml") (CI's bare-pytest env skips those five assertions; the adapter enforces the same contract on every execution).

What this defers, and to what

• Per-decile external values: enter only if HMT releases data tables or a documented chart digitization lands with per-bar provenance (the registry's value_availability_rule states this as a contract).
• PE-side decile counterparts: blocked on the Adjudicate the UK ingest architecture: #48 vs #32, then re-gate the blocked UK lanes #57 ingest adjudication and the OBR policy costings (mode 2): Policy Measures Database + EFO 3.17 → PE-UK static counterparts #54 measure registry for package composition — the components list here is the composition spec they'll consume.

Suite: 167 passed / 5 skipped; ruff format clean.

Builds #61.

🤖 Generated with Claude Code

[DTrim99]

Review — HMT distributional analysis (mode 2, chart-only)

The central call here — shipping the external side without value claims because HMT publishes no machine-readable decile data — is the right one, and it holds across every file: no fabricated or eyeballed decile value appears anywhere. Every number in hmt_da_packages.yaml sits inside a verbatim HMT measure title (rates, dates, "450,000 children" — HMT's own text), never a per-decile £/% chart read, and value_claims_emitted: 0 is asserted in tests with no emit path for value rows by construction.

Verified:

• SHA-256 pin is consistent — one 64-hex hash across meta.json:10, yaml:63, README:394, test:465 (and source.json deliberately omits it, which test_lane_provenance asserts). It's actually checked: adapter.py:319-321 recomputes-and-compares and test:470 recomputes the raw file hash.
• Counts sum both ways — channel 16 tax / 9 welfare / 5 BIK = 30; triage 3 expressible / 9 partial / 18 not_expressible = 30 (both match meta.json).

Should address

• The strongest honesty guarantees aren't in CI. The verbatim-title, figure-presence, and "20-page" checks live only in adapter.py (needs pypdf) and run outside the default pytest path — so tests/test_hmt_da_adapter.py doesn't enforce that titles are actually verbatim or figures actually present; only a manual uv run of the adapter does. A pypdf-gated test (importorskip) mirroring the registry-schema test at :499 would catch drift in CI rather than locally.

Minor

• meta.json:32 has no trailing newline (cosmetic). The measure_family strings referencing OBR policy costings (mode 2): Policy Measures Database + EFO 3.17 → PE-UK static counterparts #54/Baseline as a first-class attribute of every score #13 are unverified pointers — expected for a first slice.

Reviewed with Claude Code assistance.

[vahid-ahmadi]

Addressed in the latest commit:

• Honesty guarantees now in pytest — test_document_anchoring_in_ci_when_pypdf_available loads the adapter and enforces the verbatim-title anchoring (all 30 components), figure anchors, and the 20-page identity against the real committed PDF, gated with importorskip("pypdf")/importorskip("yaml") exactly like the schema test. Verified passing locally under uv run --with pypdf --with pyyaml (6/6 in the file); it skips in CI's bare env — same limitation as the yaml-gated test, and the reason the adapter's run() remains the local backstop. If CI ever grows pypdf/pyyaml, the guarantee turns on with no further change.
• meta.json trailing newline added.

Full suite: 167 passed / 6 skipped; ruff clean. The unverified measure_family pointers stay as-is per your note — they resolve when the #54 registry merges.

[DTrim99]

Re-review — addressed ✅

The pypdf-gated CI test I asked for is in: test_document_anchoring_in_ci_when_pypdf_available importorskips pypdf+yaml and enforces all three guarantees in CI — the 20-page identity, every FIGURE_TITLES anchor present, and verbatim titles (empty unanchored set). Skips gracefully in a bare env with the adapter run as local backstop.

SHA-256 pin, value_claims_emitted: 0, and the 16/9/5 = 30 / 3/9/18 = 30 counts all still hold. Nothing left from my side.

[DTrim99]

Re-reviewed — addressed: the pypdf-gated CI test now enforces the verbatim/figure/20-page guarantees, and the SHA pin + counts still hold. Approving.