From 82118489b487d35072f201a75731e9d1af9fa94b Mon Sep 17 00:00:00 2001 From: Braden Date: Tue, 15 Sep 2026 13:02:14 -0400 Subject: [PATCH] experiment: add independent Core Profile challenge --- challenges/core-profile-independent/README.md | 146 ++++++++++++++++++ .../REPORT-TEMPLATE.md | 47 ++++++ .../core-profile-independent/check_results.py | 43 ++++++ .../expected/conditional.json | 1 + .../expected/context-emergency.json | 1 + .../expected/context-normal.json | 1 + .../expected/deny.json | 1 + .../expected/inheritance-conflict.json | 1 + .../expected/permit.json | 1 + .../expected/scope-mismatch.json | 1 + .../expected/unknown-required-obligation.json | 1 + .../fixtures/README.md | 16 ++ 12 files changed, 260 insertions(+) create mode 100644 challenges/core-profile-independent/README.md create mode 100644 challenges/core-profile-independent/REPORT-TEMPLATE.md create mode 100644 challenges/core-profile-independent/check_results.py create mode 100644 challenges/core-profile-independent/expected/conditional.json create mode 100644 challenges/core-profile-independent/expected/context-emergency.json create mode 100644 challenges/core-profile-independent/expected/context-normal.json create mode 100644 challenges/core-profile-independent/expected/deny.json create mode 100644 challenges/core-profile-independent/expected/inheritance-conflict.json create mode 100644 challenges/core-profile-independent/expected/permit.json create mode 100644 challenges/core-profile-independent/expected/scope-mismatch.json create mode 100644 challenges/core-profile-independent/expected/unknown-required-obligation.json create mode 100644 challenges/core-profile-independent/fixtures/README.md diff --git a/challenges/core-profile-independent/README.md b/challenges/core-profile-independent/README.md new file mode 100644 index 0000000..f6b0939 --- /dev/null +++ b/challenges/core-profile-independent/README.md @@ -0,0 +1,146 @@ +# SPP Core Profile — Independent Implementation Challenge + +## What this is + +This is a small, **non-normative interoperability challenge** for the +experimental SPP Core Profile. It asks a practical question: can an engineer +who has not worked on PlaceAuth implement the profile from its public contract +and eight shared fixtures, then obtain the same normalized results? + +Use any language, runtime, policy engine, or architecture. OPA, XACML, Cedar, +and custom code are all reasonable choices. This is not a benchmark contest, +certification program, or claim that SPP 0.1 has changed. A disagreement is a +useful result: it may expose an ambiguous rule, an unnecessary field, an +engine-dependent assumption, or a better existing-policy mapping. + +For the first pass, implement independently. Do not reuse PlaceAuth evaluator +code or the existing Rego implementation. Those are reference implementations, +not the challenge specification. + +## What you are given + +Use only these materials during your first independent attempt: + +1. [The experimental SPP Core Profile](../../docs/experimental/spp-core-profile.md). +2. The eight canonical JSON files linked from [`fixtures/`](fixtures/). +3. The normalized expected outputs in [`expected/`](expected/). + +`fixtures/` contains portable relative links to the one canonical fixture set +in [`experiments/core-profile/fixtures/`](../../experiments/core-profile/fixtures/); +the JSON is intentionally not duplicated. The expected files are the `expected` +objects already established for those canonical fixtures; they do not add policy +semantics. + +To preserve the value of the exercise, first derive outcomes from the profile +and fixtures, then compare them with the expected outputs. This is +interoperability testing, not a blind test: expected results are deliberately +available for review. + +## Your task + +Build a small evaluator that accepts each fixture/profile input and produces +exactly this normalized JSON object: + +```json +{"decision":"PERMIT|DENY|CONDITIONAL","obligations":[],"reason_codes":[]} +``` + +Your implementation must handle the supported-versus-unsupported required +obligation distinction, governed-scope and context semantics, and deterministic +results. Use the Core Profile as the source of meaning. In particular, +`CONDITIONAL` is not permission, and an unrecognized required obligation must +fail closed using the profile-defined result. Do not silently invent behavior +when the profile is unclear—record the ambiguity instead. + +## Rules for the independent pass + +**Do:** + +- use a language and policy engine of your choice; +- interpret the public Core Profile literally; +- retain the requested normalized fields and deterministic JSON values; and +- document any ambiguity, assumption, or disagreement. + +**Do not:** + +- copy `experiments/core-profile/native/`; +- copy `experiments/core-profile/opa/`; +- inspect those implementations before completing the first pass; or +- silently supply semantics that the profile does not state. + +If something is unclear, write it down. Discovering unclear wording is part of +the experiment. + +## Fixtures + +| Fixture | Purpose | +| --- | --- | +| `permit` | Tests inherited permission from a governed parent scope. | +| `deny` | Tests a nearer scope denying an otherwise permitted action. | +| `conditional` | Tests a conditional result with a required authorization and obligation. | +| `unknown-required-obligation` | Tests fail-closed reliance when the deployment does not support a required obligation. | +| `scope-mismatch` | Tests denial when the requested scope is outside the supplied governed chain. | +| `context-normal` | Tests the normal-context branch of context-sensitive rules. | +| `context-emergency` | Tests the emergency-context branch of context-sensitive rules. | +| `inheritance-conflict` | Tests nearest-scope precedence over a broader wildcard rule. | + +The names are for reporting only. They do not replace the Core Profile's +contract or disclose a required implementation approach. + +## Optional self-check + +After your evaluator writes one normalized JSON output per fixture, compare +your result directory without exposing any PlaceAuth policy logic: + +```sh +python challenges/core-profile-independent/check_results.py path/to/my/results +``` + +The directory must contain files named `permit.json`, `deny.json`, and so on. +The script compares parsed JSON values only; it does not evaluate SPP rules, +provide a reference implementation, or normalize your output. + +## How to report results + +Open a GitHub issue or discussion using this suggested title: + +```text +Independent Core Profile Implementation — +``` + +Include the information in [REPORT-TEMPLATE.md](REPORT-TEMPLATE.md): language, +runtime, policy engine and version, fixtures matched, disagreements, +ambiguities, missing information, unnecessary rules, and useful mappings to +existing standards. Categorize the feedback as an **ambiguity**, **semantic +disagreement**, **implementation result**, or **existing-standard mapping**. +For an issue that is security-sensitive, use `security@placeauth.org` rather +than a public report. + +Blunt feedback is welcome. An 8/8 match alone is not the only successful +outcome. It is at least as valuable to find ambiguity, redundant data, +engine-specific behavior, or an existing standard that implements the profile +cleanly. + +## What this does not prove + +This challenge is not certification, production security validation, proof of +robot safety, proof of enforcement, IEEE endorsement, proof that SPP should +become a standard, or a modification to normative SPP 0.1. It does not test +identity, PKI, geometry, mapping, planning, task allocation, evidence, +admission, lifecycle, or physical enforcement. + +## After you finish + +**REFERENCE IMPLEMENTATIONS — DO NOT CONSULT UNTIL AFTER YOUR FIRST +INDEPENDENT PASS** + +Once you have recorded your independent result, you may compare it with the +repository's reference implementations: + +- [`experiments/core-profile/native/`](../../experiments/core-profile/native/) +- [`experiments/core-profile/opa/`](../../experiments/core-profile/opa/) + +Those implementations are useful comparison points, not authoritative proof +that an interpretation is correct. If they reveal a difference from your +implementation, report the difference and the Core Profile wording that led to +it. diff --git a/challenges/core-profile-independent/REPORT-TEMPLATE.md b/challenges/core-profile-independent/REPORT-TEMPLATE.md new file mode 100644 index 0000000..bd73ec6 --- /dev/null +++ b/challenges/core-profile-independent/REPORT-TEMPLATE.md @@ -0,0 +1,47 @@ +# Independent Implementation Report + +Implementation: + +Language/runtime: + +Policy engine, if any: + +Version: + +Repository/link: + +## Results + +| Fixture | Result | Match? | +| --- | --- | --- | +| permit | | | +| deny | | | +| conditional | | | +| unknown-required-obligation | | | +| scope-mismatch | | | +| context-normal | | | +| context-emergency | | | +| inheritance-conflict | | | + +## Ambiguities encountered + +## Information missing from the profile + +## Semantics that felt redundant + +## Existing standards or policy concepts reused + +## Suggested profile changes + +## Overall assessment + +Choose one: + +- implementable as written +- implementable with minor clarification +- underspecified +- unnecessary because an existing profile already solves it +- other + +Please be direct. A disagreement or negative assessment is useful feedback for +this experimental, non-normative profile. diff --git a/challenges/core-profile-independent/check_results.py b/challenges/core-profile-independent/check_results.py new file mode 100644 index 0000000..0e2c80e --- /dev/null +++ b/challenges/core-profile-independent/check_results.py @@ -0,0 +1,43 @@ +"""Compare participant-provided normalized Core Profile outputs to expected JSON.""" +from __future__ import annotations + +import json +from pathlib import Path +import sys + + +ROOT = Path(__file__).resolve().parent +EXPECTED = ROOT / "expected" + + +def main() -> int: + if len(sys.argv) != 2: + print("usage: python check_results.py PATH_TO_RESULTS") + return 2 + + results = Path(sys.argv[1]) + matched = 0 + expected_paths = sorted(EXPECTED.glob("*.json")) + for expected_path in expected_paths: + result_path = results / expected_path.name + if not result_path.is_file(): + print(f"{expected_path.stem}: missing") + continue + try: + expected = json.loads(expected_path.read_text(encoding="utf-8")) + actual = json.loads(result_path.read_text(encoding="utf-8")) + except json.JSONDecodeError as error: + print(f"{expected_path.stem}: invalid JSON ({error.msg})") + continue + if actual == expected: + matched += 1 + print(f"{expected_path.stem}: match") + else: + print(f"{expected_path.stem}: mismatch") + total = len(expected_paths) + print(f"summary: {matched}/{total} fixtures matched") + return 0 if matched == total else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/challenges/core-profile-independent/expected/conditional.json b/challenges/core-profile-independent/expected/conditional.json new file mode 100644 index 0000000..027cb17 --- /dev/null +++ b/challenges/core-profile-independent/expected/conditional.json @@ -0,0 +1 @@ +{"decision":"CONDITIONAL","obligations":[{"type":"recording.disabled","value":true}],"reason_codes":["required_authorization_missing"]} diff --git a/challenges/core-profile-independent/expected/context-emergency.json b/challenges/core-profile-independent/expected/context-emergency.json new file mode 100644 index 0000000..9fbf3e5 --- /dev/null +++ b/challenges/core-profile-independent/expected/context-emergency.json @@ -0,0 +1 @@ +{"decision":"PERMIT","obligations":[],"reason_codes":["matched_rule"]} diff --git a/challenges/core-profile-independent/expected/context-normal.json b/challenges/core-profile-independent/expected/context-normal.json new file mode 100644 index 0000000..e4ac2af --- /dev/null +++ b/challenges/core-profile-independent/expected/context-normal.json @@ -0,0 +1 @@ +{"decision":"DENY","obligations":[],"reason_codes":["matched_rule"]} diff --git a/challenges/core-profile-independent/expected/deny.json b/challenges/core-profile-independent/expected/deny.json new file mode 100644 index 0000000..e4ac2af --- /dev/null +++ b/challenges/core-profile-independent/expected/deny.json @@ -0,0 +1 @@ +{"decision":"DENY","obligations":[],"reason_codes":["matched_rule"]} diff --git a/challenges/core-profile-independent/expected/inheritance-conflict.json b/challenges/core-profile-independent/expected/inheritance-conflict.json new file mode 100644 index 0000000..9fbf3e5 --- /dev/null +++ b/challenges/core-profile-independent/expected/inheritance-conflict.json @@ -0,0 +1 @@ +{"decision":"PERMIT","obligations":[],"reason_codes":["matched_rule"]} diff --git a/challenges/core-profile-independent/expected/permit.json b/challenges/core-profile-independent/expected/permit.json new file mode 100644 index 0000000..9fbf3e5 --- /dev/null +++ b/challenges/core-profile-independent/expected/permit.json @@ -0,0 +1 @@ +{"decision":"PERMIT","obligations":[],"reason_codes":["matched_rule"]} diff --git a/challenges/core-profile-independent/expected/scope-mismatch.json b/challenges/core-profile-independent/expected/scope-mismatch.json new file mode 100644 index 0000000..f3ff8a4 --- /dev/null +++ b/challenges/core-profile-independent/expected/scope-mismatch.json @@ -0,0 +1 @@ +{"decision":"DENY","obligations":[],"reason_codes":["scope_mismatch"]} diff --git a/challenges/core-profile-independent/expected/unknown-required-obligation.json b/challenges/core-profile-independent/expected/unknown-required-obligation.json new file mode 100644 index 0000000..7aa7a71 --- /dev/null +++ b/challenges/core-profile-independent/expected/unknown-required-obligation.json @@ -0,0 +1 @@ +{"decision":"DENY","obligations":[],"reason_codes":["required_obligation_unsupported"]} diff --git a/challenges/core-profile-independent/fixtures/README.md b/challenges/core-profile-independent/fixtures/README.md new file mode 100644 index 0000000..fade195 --- /dev/null +++ b/challenges/core-profile-independent/fixtures/README.md @@ -0,0 +1,16 @@ +# Canonical shared fixtures + +The challenge deliberately references, rather than copies, the engine-neutral +Core Profile fixtures. This prevents a challenge copy from drifting from the +fixtures evaluated by the native and OPA reference implementations. + +- [permit](../../../experiments/core-profile/fixtures/permit.json) +- [deny](../../../experiments/core-profile/fixtures/deny.json) +- [conditional](../../../experiments/core-profile/fixtures/conditional.json) +- [unknown required obligation](../../../experiments/core-profile/fixtures/unknown-required-obligation.json) +- [scope mismatch](../../../experiments/core-profile/fixtures/scope-mismatch.json) +- [normal context](../../../experiments/core-profile/fixtures/context-normal.json) +- [emergency context](../../../experiments/core-profile/fixtures/context-emergency.json) +- [inheritance/conflict](../../../experiments/core-profile/fixtures/inheritance-conflict.json) + +Use these files unchanged as the input corpus for an independent implementation.