Skip to content

Add datacard_validator — standalone validator library for MODCON Datacard v1 - #10

Open
jeanbez wants to merge 1 commit into
mainfrom
datacard-validator-library
Open

Add datacard_validator — standalone validator library for MODCON Datacard v1#10
jeanbez wants to merge 1 commit into
mainfrom
datacard-validator-library

Conversation

@jeanbez

@jeanbez jeanbez commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

What this is

A rule-based validator that flags incomplete, incorrect, inconsistent, and misleading information in datacards produced by the datacard-generator skill (or handwritten to the same template). It was built as a standalone tool — deliberately not part of the skill — so cards can be checked after the fact, in bulk, and from other services.

Recovered from an uncommitted working tree; it had never been committed to any branch.

Contents

Path What
datacard_validator/__init__.py Library: check implementations, Finding/Datacard model, HTML + human renderers
datacard_validator/__main__.py CLI — python -m datacard_validator
scripts/validate_datacard.py Shim keeping the historical python scripts/validate_datacard.py card.md invocation working
tests/test_validator.py 59 tests over the check matrix
pyproject.toml Editable install, so downstream code can import datacard_validator

Public API: validate_file(path, offline=True, level=None) -> dict, build_summary(results) -> dict, plus load_datacard, run_checks, get/walk, the harvest_* helpers and all renderers.

Checks

  • Identifiers — ORCID format + ISO-7064 MOD-11-2 checksum, ROR format, DOI format and resolution, OSTI record lookup
  • Structure — YAML frontmatter parses; no surviving [!TODO] / <REPLACE:…> template placeholders
  • Values — ISO YYYY-MM-DD dates and date monotonicity, enum membership (level, sensitivity tier, access level, release status, stewardship, update frequency, OSTI dataset type), SPDX license whitelist
  • Completeness — per-level required-field matrices for L1/L2/L3, and an L3 over-claim check (declares L3 with no L3 content)
  • Consistency — YAML title vs body H1, access_level: open without a license, and a critical finding when nothing in the card points at the actual data (no repository, DOI, or URL)

Severity ladder is critical > error > warning > info. A card with no declared level is validated as L1 and the omission is itself reported.

Output modes

--offline (skip network), --json, --sidecar (<card>.validation.json), --html (self-contained per-card report), --summary-out for multi-file runs, --level N to override.

Multi-file runs print a per-file roll-up plus a "top issues" block ranked by files affected, and with --html write a validation_summary.html dashboard (totals, per-card table linking to each sidecar, issue bars).

Exit codes: 0 clean, 1 errors present, 2 script/input error.

Verification

$ python -m pytest tests/ -q
...........................................................   [100%]   59 passed

CLI, python -m and library entry points all verified to produce identical payloads.

Notes for review

  • Template version. This validator targets MODCON Datacard v1, the template the skill shipped when it was written. main has since moved the skill to Genesis v1.2 with its own Pydantic/LinkML-based skills/datacard-generator/scripts/validate_datacard.py. The two are independent — different paths, different templates, no collision — but a follow-up should decide whether this one gets ported to v1.2 or kept as the v1-era checker.
  • Nothing in the skill is touched by this PR.
  • Downstream consumer: the integrate-validator branch of BASE-data-render-cards/datacard-explorer imports this package (its own PR is separate).

…ard v1

Adds a rule-based validator that flags incomplete, incorrect, inconsistent,
and misleading information in datacards produced by the datacard-generator
skill (or handwritten to the same template).

- datacard_validator/ — importable library: checks, findings model, HTML and
  human renderers. Public API: validate_file(), build_summary(), load_datacard(),
  run_checks(), plus the harvest_*/lookup helpers.
- datacard_validator/__main__.py — CLI (python -m datacard_validator), with
  --offline, --json, --sidecar, --html, --summary-out, --level.
- scripts/validate_datacard.py — shim keeping the historical invocation working.
- tests/test_validator.py — 59 tests covering the check matrix.
- pyproject.toml — installs as an editable package so downstream consumers
  (datacard-explorer) can import it instead of shelling out.

Checks cover: ORCID format + ISO-7064 checksum, ROR, DOI and OSTI lookups,
ISO dates and date monotonicity, enum/SPDX values, surviving template
placeholders, per-level required-field matrices (L1-L3), L3 over-claim, and
missing data-access pointers. Severity ladder is critical > error > warning >
info; a card with no declared level validates as L1 and reports the omission.

Multi-file runs emit a roll-up with per-check "top issues" ranked by files
affected, and an optional validation_summary.html dashboard.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant