Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #179 +/- ##
==========================================
+ Coverage 72.76% 81.80% +9.03%
==========================================
Files 37 66 +29
Lines 6004 10565 +4561
Branches 1307 2318 +1011
==========================================
+ Hits 4369 8643 +4274
- Misses 1187 1357 +170
- Partials 448 565 +117
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Pushed increments 22–24 ( What this stretch added. The layer stopped being only defended and became connected: the explorer config surface ( Two things need your call, neither blocking:
The substantive change worth reading first: network identity is now bound to CONTENT, not the file stem. A Two related fixes came out of it: Green: 938 fast pdep tests (~29 s) plus 105 in |
|
Increment 25 — What landedPersistence is now round-trippable.
Two findings worth your attentionTests that passed for the wrong reason. A mutation sweep of A Still openOne round-37 P1 is deliberately not yet fixed: in Still needs your call (unchanged, neither blocks)
|
Increment 26 — the layer's tests made true, then its inputs made untrustedBranch is at The one you should look at first: a persisted selection could forge its way past the budget gate Twelve commits, in three groups. Coverage. A mechanical sweep — rewrite each Correctness. The exactly-zero case is a judgement call worth flagging for you: an all-ILT network reports zero TS Hardening. Threshold validation (a Still waiting on you, and neither blocks the work (both also in
|
…tion Third and last part of the gate -> ranking reframe. 354476f gave T3's in-run queueing the whole field to decide with; e43349d ranked that field against a budget. Both stopped at T3's own wiring: explore_pdep_network(), the public entry point, still treated selection.qualified as the sole admission authority, so a caller that had ranked a network and chosen to spend on it had no way to say so. It now takes a keyword-only admission_policy. Under 'caller_admitted' BOTH qualification checks stand aside -- the unqualified skip and the not-evaluated raise -- because both are about using `qualified` AS A GATE, and a caller that admitted the network elsewhere is not using it as one. Under the default, 'qualified_selection', nothing changes: with no external admission there is no positive evidence and no spend decision, so a missing evaluation is still a missing verdict rather than a negative one. What does NOT stand aside is provenance. method, network_id and network_source_hash stay unconditional, and evaluation_status is now checked for validity unconditionally too -- admitting a network is a budget statement, and no budget statement makes a stale or unreadable decision current. That is the whole difference from selection=None, which drops the binding along with the gate. Keyword-only because logger is the fourth positional parameter and an argument inserted before it would land on callers' logger. The result records what admitted it, since a 'succeeded' result carrying an unqualified selection is otherwise indistinguishable from a bypassed gate. A third value, 'ungated', covers selection=None: recording the argument default there would have the record assert that a qualified selection admitted a run for which no selection existed -- a false provenance claim, and the one an auditor of an expensive QM run would lean on. It is derived rather than requested, so a caller cannot claim 'ungated' while passing a selection, and 'skipped' is cross-checked against it because only the gate can decline a run. A results file written before this field is DERIVED, not refused and not blanket-defaulted: nothing predating the field could have been caller-admitted, so a record with a selection was gate-admitted and one without was ungated. That reconstructs the true value instead of guessing, and keeps EXPLORATION_RESULT_SCHEMA_VERSION 1 meaning one loadable shape. Public API change; PR #179's description needs it. T3's own path does not use the new policy yet -- t3.main still offers only qualified selections to the budget -- so in-run behaviour is unchanged. Also de-drifts prose that still called this a budget gate, including two claims in selector.py that stated flatly that explore_pdep_network refuses a selection that did not qualify. Verified: 1264 fast tests, and 70 in test_main_wiring.py against this exact tree. Fourteen mutations applied, fourteen killed, all against the final tree: the skip ignores the policy; the raise ignores it; the hash check becomes policy-conditional; the evaluation_status check becomes policy-conditional; the recorded value echoes the argument instead of being derived; 'ungated' becomes requestable; the loader defaults, or blanket-defaults, instead of deriving; the succeeded and failed sites drop the field; an unknown policy falls back to the default; caller_admitted with no selection is allowed as a no-op; the result skips policy validation; the skipped cross-check is removed; as_dict drops the field. Codex round 44 rejected the bare require_qualified bool this started as -- a flag naming what is not enforced hides what did admit the run -- and round 45 caught the false 'qualified_selection' on the selection-less path.
Update: the per-network qualification GATE is now a budget-limited RANKINGThree commits implement the reframe you asked for. Still DO NOT MERGE.
Two things to look at specifically. Behaviour is NOT preserved by
Design points worth your disagreement:
Verification: 1264 fast tests, 70 in |
Increment 32 — a budget refusal is now a durable record (2 commits, pushed)
Until now, a network that qualified for QM refinement and then went unrefined because the budget was The plan I inherited for this was built on a false premise, and probing killed it. It said to Reusing the existing TS-join sidecar was considered and rejected on review, for a reason I had not What landed instead: a Two defects worth naming, both found before the tests were green:
The builder originally re-derived the budget's grouping, ranking and remaining-budget bookkeeping. Verification. Fast suite 1317 passed; One caveat worth flagging: the record has no in-repo consumer today. Its intended reader is human Still DO NOT MERGE — this is for your review. |
Follow-up on the comment above — adversarial review of the write path found six defects
Two of the six are worth calling out, because they were real and neither was caught by a green suite: The write followed a symlink. A symlink sitting at The write was not atomic, and a truncated record still validates. A partial file can parse and The other four: version gates compared with Verification. Fast suite 1324 passed; One disclosure: the Still DO NOT MERGE — for your review. |
57dbb4a to
2c91568
Compare
…tion Third and last part of the gate -> ranking reframe. 354476f gave T3's in-run queueing the whole field to decide with; e43349d ranked that field against a budget. Both stopped at T3's own wiring: explore_pdep_network(), the public entry point, still treated selection.qualified as the sole admission authority, so a caller that had ranked a network and chosen to spend on it had no way to say so. It now takes a keyword-only admission_policy. Under 'caller_admitted' BOTH qualification checks stand aside -- the unqualified skip and the not-evaluated raise -- because both are about using `qualified` AS A GATE, and a caller that admitted the network elsewhere is not using it as one. Under the default, 'qualified_selection', nothing changes: with no external admission there is no positive evidence and no spend decision, so a missing evaluation is still a missing verdict rather than a negative one. What does NOT stand aside is provenance. method, network_id and network_source_hash stay unconditional, and evaluation_status is now checked for validity unconditionally too -- admitting a network is a budget statement, and no budget statement makes a stale or unreadable decision current. That is the whole difference from selection=None, which drops the binding along with the gate. Keyword-only because logger is the fourth positional parameter and an argument inserted before it would land on callers' logger. The result records what admitted it, since a 'succeeded' result carrying an unqualified selection is otherwise indistinguishable from a bypassed gate. A third value, 'ungated', covers selection=None: recording the argument default there would have the record assert that a qualified selection admitted a run for which no selection existed -- a false provenance claim, and the one an auditor of an expensive QM run would lean on. It is derived rather than requested, so a caller cannot claim 'ungated' while passing a selection, and 'skipped' is cross-checked against it because only the gate can decline a run. A results file written before this field is DERIVED, not refused and not blanket-defaulted: nothing predating the field could have been caller-admitted, so a record with a selection was gate-admitted and one without was ungated. That reconstructs the true value instead of guessing, and keeps EXPLORATION_RESULT_SCHEMA_VERSION 1 meaning one loadable shape. Public API change; PR #179's description needs it. T3's own path does not use the new policy yet -- t3.main still offers only qualified selections to the budget -- so in-run behaviour is unchanged. Also de-drifts prose that still called this a budget gate, including two claims in selector.py that stated flatly that explore_pdep_network refuses a selection that did not qualify. Verified: 1264 fast tests, and 70 in test_main_wiring.py against this exact tree. Fourteen mutations applied, fourteen killed, all against the final tree: the skip ignores the policy; the raise ignores it; the hash check becomes policy-conditional; the evaluation_status check becomes policy-conditional; the recorded value echoes the argument instead of being derived; 'ungated' becomes requestable; the loader defaults, or blanket-defaults, instead of deriving; the succeeded and failed sites drop the field; an unknown policy falls back to the default; caller_admitted with no selection is allowed as a no-op; the result skips policy validation; the skipped cross-check is removed; as_dict drops the field. Codex round 44 rejected the bare require_qualified bool this started as -- a flag naming what is not enforced hides what did admit the run -- and round 45 caught the false 'qualified_selection' on the selection-less path.
…ture lock Two CodeQL alerts on PR #179, one of which I twice reported to Alon as a pure false positive and which turns out to be only mostly one. `py/unused-import` flagged the five names re-exported from `t3.utils.network_thermo`. Four of them ARE imported from this module elsewhere, so for those the query is wrong: it asks only whether a name is used within its own file, follows no cross-module re-export, and does not read the per-line ruff suppressions. `__all__` says the same thing in the language instead of in a linter pragma, and the query honours it. The list is not hand-guessed -- it is every name any other module imports from `t3.pdep.parser`, taken by parsing the repo's own import statements. That sweep is also what corrected me: `NetworkThermoCeiling` was re-exported here and imported from here by nobody. It is deleted rather than declared. Importers take it from `t3.utils.network_thermo`, which is where it lives. The capture lock file drops from 0o644 to 0o600. It is written by `_acquire_capture_lock` and read only by `_read_capture_lock_holder`, both inside one user's own capture directory, to recover the PID of a possibly-dead holder. There is no cross-user reader to serve, so the narrow mode is the correct one on its own merits rather than a concession to the scanner.
|
Dropped the Blocking: RMG-Py #2990 must land first. T3's CI clones Not blocking, but worth knowing before review:
Rebased onto |
There was a problem hiding this comment.
Pull request overview
Adds end-to-end, network-level PDep selection and QM refinement while keeping ARC species/reaction scoped.
Changes:
- Adds sensitivity/uncertainty assessment, budgeting, persistence, and queueing.
- Adds Arkane exploration/ME adapters, artifact capture, and hybrid input generation.
- Extends schemas, runner validation, documentation, fixtures, and regression coverage.
Reviewed changes
Copilot reviewed 68 out of 110 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
.gitignore |
Tracks required PDep fixtures. |
docs/input_reference.md |
Documents PDep sensitivity settings. |
t3/common.py |
Adds shared PDep constants. |
t3/logger.py |
Adds PDep reporting. |
t3/main.py |
Integrates the PDep workflow. |
t3/pdep/__init__.py |
Initializes the PDep package. |
t3/pdep/api.py |
Exposes selection and exploration APIs. |
t3/pdep/assessment.py |
Models and persists assessments. |
t3/pdep/budget.py |
Applies network-level QM budgets. |
t3/pdep/cache.py |
Validates SA cache provenance. |
t3/pdep/capture.py |
Captures and verifies ARC artifacts. |
t3/pdep/discovery.py |
Discovers PDep networks. |
t3/pdep/energy_settings.py |
Resolves energy settings. |
t3/pdep/explorer/__init__.py |
Initializes explorer adapters. |
t3/pdep/explorer/adapter.py |
Defines the explorer interface. |
t3/pdep/explorer/arkane.py |
Implements Arkane PES exploration. |
t3/pdep/explorer/config.py |
Defines exploration configuration. |
t3/pdep/explorer/factory.py |
Constructs explorer adapters. |
t3/pdep/explorer/input_file.py |
Generates exploration inputs. |
t3/pdep/explorer/result.py |
Represents exploration results. |
t3/pdep/hashing.py |
Provides artifact hashing. |
t3/pdep/hybrid.py |
Generates hybrid Arkane inputs. |
t3/pdep/join.py |
Joins networks with ARC results. |
t3/pdep/me_success.py |
Validates ME output success. |
t3/pdep/mesolver/__init__.py |
Initializes ME adapters. |
t3/pdep/mesolver/adapter.py |
Defines the ME interface. |
t3/pdep/mesolver/arkane.py |
Implements Arkane ME solving. |
t3/pdep/mesolver/factory.py |
Constructs ME adapters. |
t3/pdep/parser.py |
Safely parses network inputs. |
t3/pdep/reason_codes.py |
Centralizes refusal reasons. |
t3/pdep/selector.py |
Implements network qualification. |
t3/pdep/yaml_safe.py |
Restricts YAML deserialization. |
t3/runners/rmg_runner.py |
Strengthens Arkane success checks. |
t3/schema.py |
Adds PDep configuration fields. |
t3/utils/network_thermo.py |
Adds network thermo helpers. |
t3/utils/uncertainty.py |
Classifies kinetics uncertainty. |
t3/utils/writer.py |
Supports ME method rewriting. |
tests/data/pdep_arkane_log/arkane.log |
Supplies Arkane log data. |
tests/data/pdep_energy_settings/composite_level_project/calcs/statmech/thermo/input.py |
Composite-level input fixture. |
tests/data/pdep_energy_settings/composite_level_project/output/output.yml |
Composite-level output fixture. |
tests/data/pdep_energy_settings/missing_model_chemistry_project/calcs/statmech/kinetics/input.py |
Missing-chemistry input fixture. |
tests/data/pdep_energy_settings/missing_model_chemistry_project/output/output.yml |
Missing-chemistry output fixture. |
tests/data/pdep_energy_settings/xl1001_project/calcs/statmech/kinetics/input.py |
XL1001 kinetics fixture. |
tests/data/pdep_energy_settings/xl1001_project/calcs/statmech/thermo/input.py |
XL1001 thermo fixture. |
tests/data/pdep_energy_settings/xl1001_project/output/output.yml |
XL1001 output fixture. |
tests/data/pdep_hybrid/arc_ts/TS1.py |
Hybrid TS fixture. |
tests/data/pdep_hybrid/arc_ts/TS2.py |
Hybrid TS fixture. |
tests/data/pdep_hybrid/arc_ts/TS_colliding.py |
Collision fixture. |
tests/data/pdep_hybrid/arc_ts/TS_dup_path_text.py |
Duplicate-path fixture. |
tests/data/pdep_hybrid/arc_ts/TS_missing_log.py |
Missing-log fixture. |
tests/data/pdep_hybrid/arc_ts/logs/freq_a1235/output.out |
Frequency-log fixture. |
tests/data/pdep_hybrid/arc_ts/logs/scan_a1236/output.out |
Scan-log fixture. |
tests/data/pdep_hybrid/arc_ts/logs/sp_a1234/output.out |
Single-point fixture. |
tests/data/pdep_hybrid/arc_ts/logs/ts1_freq.out |
TS1 frequency fixture. |
tests/data/pdep_hybrid/arc_ts/logs/ts1_scan.out |
TS1 scan fixture. |
tests/data/pdep_hybrid/arc_ts/logs/ts1_sp.out |
TS1 energy fixture. |
tests/data/pdep_hybrid/arc_ts/logs/ts2_freq.out |
TS2 frequency fixture. |
tests/data/pdep_hybrid/arc_ts/logs/ts2_sp.out |
TS2 energy fixture. |
tests/data/pdep_me/hard_failure/output.py |
Hard-failure ME fixture. |
tests/data/pdep_me/overlapping_channels/input.py |
Overlapping-channel input. |
tests/data/pdep_me/overlapping_channels/output.py |
Overlapping-channel output. |
tests/data/pdep_me/soft_failure_cse/output.py |
Soft-failure ME fixture. |
tests/data/pdep_me/success/output.py |
Successful ME fixture. |
tests/data/pdep_me/success_multi/input.py |
Multi-channel ME input. |
tests/data/pdep_me/success_multi/output.py |
Multi-channel ME output. |
tests/data/pdep_network_variants/network_arity_asymmetry.py |
Arity edge-case fixture. |
tests/data/pdep_network_variants/network_explicit_products.py |
Explicit-product fixture. |
tests/data/pdep_real_networks/README.md |
Documents real fixtures. |
tests/data/pdep_real_networks/network21_1/arkane.log |
Network 21 log. |
tests/data/pdep_real_networks/network21_1/network21_1.py |
Network 21 input. |
tests/data/pdep_real_networks/network21_1/sensitivity/sa_coefficients.yml |
Network 21 sensitivity data. |
tests/data/pdep_real_networks/network21_1/sensitivity/t3_sa_cache.yml |
Network 21 cache metadata. |
tests/data/pdep_real_networks/network799_1/arkane.log |
Network 799 log. |
tests/data/pdep_real_networks/network799_1/network799_1.py |
Network 799 input. |
tests/data/pdep_real_networks/network799_1/sensitivity/sa_coefficients.yml |
Network 799 sensitivity data. |
tests/data/pdep_real_networks/network799_1/sensitivity/t3_sa_cache.yml |
Network 799 cache metadata. |
tests/data/pdep_sa/network4_2_MSC/sa_coefficients.yml |
SA parsing fixture. |
tests/test_main.py |
Tests main workflow integration. |
tests/test_pdep/_wiring_helpers.py |
Provides wiring-test helpers. |
tests/test_pdep/test_api.py |
Tests public PDep APIs. |
tests/test_pdep/test_arkane_mesolver.py |
Tests Arkane ME adapter. |
tests/test_pdep/test_assessment.py |
Tests assessment logic. |
tests/test_pdep/test_assessment_persistence.py |
Tests assessment persistence. |
tests/test_pdep/test_budget.py |
Tests QM budgeting. |
tests/test_pdep/test_cache.py |
Tests cache validation. |
tests/test_pdep/test_capture.py |
Tests artifact capture. |
tests/test_pdep/test_discovery.py |
Tests network discovery. |
tests/test_pdep/test_energy_settings.py |
Tests energy resolution. |
tests/test_pdep/test_explorer_arkane.py |
Tests Arkane exploration. |
tests/test_pdep/test_explorer_config.py |
Tests explorer configuration. |
tests/test_pdep/test_explorer_factory.py |
Tests explorer construction. |
tests/test_pdep/test_explorer_input_file.py |
Tests explorer input generation. |
tests/test_pdep/test_explorer_result.py |
Tests explorer results. |
tests/test_pdep/test_hybrid.py |
Tests hybrid generation. |
tests/test_pdep/test_join.py |
Tests network-result joining. |
tests/test_pdep/test_main_funnel.py |
Tests workflow funneling. |
tests/test_pdep/test_main_wiring.py |
Tests orchestration wiring. |
tests/test_pdep/test_me_success.py |
Tests ME success validation. |
tests/test_pdep/test_mesolver_factory.py |
Tests ME construction. |
tests/test_pdep/test_parser.py |
Tests safe parsing. |
tests/test_pdep/test_real_networks.py |
Tests real network chains. |
tests/test_pdep/test_selector.py |
Tests network selection. |
tests/test_pdep/test_selector_diagnostics.py |
Tests selection diagnostics. |
tests/test_pdep/test_yaml_safe.py |
Tests restricted YAML loading. |
tests/test_runners/test_rmg_runner.py |
Tests Arkane artifact gates. |
tests/test_schema.py |
Tests new schema fields. |
tests/test_utils/test_libraries.py |
Tests append idempotency. |
tests/test_utils/test_network_thermo.py |
Tests network thermo helpers. |
tests/test_utils/test_uncertainty.py |
Tests uncertainty classification. |
tests/test_utils/test_writer.py |
Tests writer changes. |
Suppressed comments (1)
t3/pdep/cache.py:366
- This metadata path is derived from the caller-supplied
sa_path, but it is deserialized with ARC'sread_yaml_file/yaml.FullLoaderbefore the cache can be rejected. That bypasses the restricted-loader guarantee of the public PDep API for a file the caller controls. Use the restricted loader already imported by this module.
metadata = read_yaml_file(metadata_path) or dict()
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Add three tests pinning rank_pdep_networks' documented three-tier ordering + tiebreak contract, none of which was previously exercised: - test_rank_pdep_networks_orders_qualified_by_delta_ln_k_descending kills Mutation C4 (max_delta_ln_k = max(...) -> max_delta_ln_k = 0.0): with the mutation applied, the test fails with assert ['aLowDelta', 'zHighDelta'] == ['zHighDelta', 'aLowDelta']. - test_rank_pdep_networks_not_evaluated_ranks_before_unqualified kills Mutation C3 (tier = 1 -> tier = 2 on the not_evaluated branch): with the mutation applied, the test fails with assert ['aUnqualified', 'zMissing'] == ['zMissing', 'aUnqualified']. - test_rank_pdep_networks_ties_broken_by_network_id_ascending kills Mutation C5 (dropping `selection.network_id or ''` from the _rank_key tuple): with the mutation applied, the test fails with assert ['zzz', 'aaa'] == ['aaa', 'zzz']. Each mutation was applied directly to t3/pdep/api.py, confirmed to make the corresponding new test fail, then restored from a scratchpad copy and confirmed green again. Full fast suite: 984 passed.
Add test_resolve_direction_key_multiple_canonical_matches_is_ambiguous,
constructing an SA dict with two distinct keys ('(A) + (B) <=> C' and
'(B) + (A) <=> C') that both canonicalize to the same
(('A', 'B'), ('C',)) tuple as network_reaction='[A] + [B] <=> C', so
the multi-match branch is genuinely exercised (verified via
_canonical_reaction in a scratch run first).
- Kills Mutation B4 (`if len(matches) > 1:` -> `if False:`): with the
mutation applied, the test fails with
assert None == '(A) + (B) <=> C' (falls through to "not found").
- Kills Mutation B5 (`return matches[0], True, ...` -> `..., False,
...` on the multi-match branch): with the mutation applied, the test
fails with assert False is True (ambiguity flag no longer set).
Both mutations applied directly to t3/pdep/selector.py, confirmed to
fail the new test, then restored from a scratchpad copy and confirmed
green again.
B8 (dropping the `key != STRUCTURES_KEY` filter from `candidates`) is
deliberately NOT tested: _canonical_reaction('structures') returns the
single-sided tuple (('structures',),), which can only collide with a
network_reaction whose own canonicalization is also single-sided (no
' <=> ' separator). Every real network_reaction in this codebase is
either produced by the two-sided 'A + B <=> C' format or supplied by a
caller in that same form, so a genuine collision with 'structures'
cannot arise without feeding a malformed, non-reaction string as
network_reaction -- not a realistic path worth enshrining in a test.
Full fast suite: 985 passed.
Mutation-sweep survivors A1, C6, C7 all shared the property that disabling
the guard does not change the ultimate verdict (still a refusal/not_evaluated),
only the diagnosis message -- so each test below pins the specific message
text a caller relies on for troubleshooting, not the shared outcome.
- A1 (selector.py:525, select_from_sa_dict's non-dict sa_dict guard): updated
test_evaluation_status_not_evaluated_for_malformed_sa_dict to assert the
warning is select_from_sa_dict's OWN diagnosis ("criterion (b)") rather than
the different message resolve_direction_key would produce further downstream
for the same non-dict input. Disabling the guard made the test fail with
"cannot locate reaction A + B <=> C" in place of "criterion (b)", confirming
the kill.
- C6 (api.py:571, _unpack_network_entry's dict-form missing-network_path
guard): added test_rank_pdep_networks_dict_entry_missing_network_path_names_the_key.
Disabling the guard made the test fail with "'NoneType' object has no
attribute 'path'" in place of "missing a 'network_path' key", confirming
the kill.
- C7 (api.py:577, _unpack_network_entry's tuple/list-form short-entry guard):
added test_rank_pdep_networks_short_tuple_entry_names_the_required_shape.
Disabling the guard made the test fail with "tuple index out of range" in
place of "must have at least (network_path, sa_path)", confirming the kill.
No code defects found: every mutation produced exactly the predicted failure,
confirming both the tests' discriminating power and the correctness of the
underlying refusal logic.
explore_pdep_network() had three sequential first-raise-wins guards
(no source hash, hash mismatch, unevaluated-and-unqualified). A
selection that failed more than one of these at once reported only
whichever guard ran first, silently discarding the rest: a caller who
fixed the first problem and re-ran was then hit with a second,
previously unmentioned problem. Accumulate every applicable diagnosis
(the hash and evaluation-status guards are independent and can
co-occur; no-hash and hash-mismatch are mutually exclusive, since a
None hash short-circuits the mismatch check) into one ValueError,
enumerating each existing message's exact original text verbatim so
every current substring-matching test still passes unmodified.
TDD: added test_explore_pdep_network_reports_every_reason_a_selection_cannot_gate
(a selection with both a stale hash and evaluation_status='not_evaluated'/
qualified=False). Confirmed it failed against the unfixed code with:
AssertionError: assert 'not_evaluated' in "selection.network_source_hash
('sha256:eee...') does not match the content hash ...: the network file
has changed since the decision was made. ..."
i.e. only the hash-mismatch diagnosis was reported. After implementing the
accumulation, the full pdep suite (minus test_main_wiring.py) passed 1001/1001.
Mutation test: reverted api.py to the original three independent
first-raise-wins guards (scratchpad backup, not git checkout) and reran the
new test -- it reproduced the identical original failure above verbatim,
confirming the test is pinned to the accumulation behaviour and not
incidentally green. Restored the fix from the scratchpad backup and
reconfirmed 1001/1001 passing.
fix(pdep): fold the method and network-id guards into the accumulated report
explore_pdep_network's selection guard raised immediately on the first of
selection.method != config.method or selection.network_id != parsed_network.network_id,
before the reasons=[] accumulation block that already folded the hash and
evaluation-status diagnoses together. A caller who fixed only the first-reported
problem, re-ran, and was then handed a second, previously unmentioned problem had
been actively misled about how much was actually wrong with the selection.
Folds both checks into the same accumulator. network_id mismatch already proves
this is a decision about a different network entirely, so it short-circuits (via
else:) the hash-diagnosis branch: reporting "the network file has changed since
the decision was made" alongside "this is not even the same network" would be a
misleading second diagnosis for evidence that doesn't apply.
New test test_explore_pdep_network_reports_method_and_network_id_alongside_evaluation_status
builds a selection that fails method, network_id, AND evaluation_status
simultaneously and asserts a SINGLE error naming all three ("for 3 independent
reasons"), with the hash-mismatch wording ("has changed since") absent since it's
short-circuited by the network_id mismatch.
Mutation testing: neutralizing the method check's fold-in (`if False and
selection.method != ...`) drops the message back to a bare 2-reason report
missing "method" and fails the new test's `for 3 independent reasons` and
`'method' in message` assertions. Neutralizing the network_id check's fold-in
similarly produces a message missing 'network_id', failing the new test's
`'network_id' in message` assertion. Both mutations were applied, confirmed to
fail the new test for the right reason, then reverted (restored from a scratchpad
backup, never via git checkout) with the full suite reconfirmed green after each.
Full fast suite: 1002 passed, 1 warning (was 1001 baseline + 1 new test).
_make_fake_run_arkane_job (tests/test_pdep/test_explorer_arkane.py) accepted a required_artifact argument and silently ignored it, returning the caller's declared `success` unconditionally. The real run_arkane_job (t3/runners/rmg_runner.py) deletes any stale artifact up front and only reports success if Arkane itself (re)wrote it -- so the stub was more permissive than reality, exactly the "fixtures testify" failure mode that has bitten this branch before. Added test_fake_run_arkane_job_honours_required_artifact, which declares success=True while omitting the required 'output.py' from output_files. Against the old stub it failed for the wrong reason (only _resolve_artifacts' redundant "missing expected outputs" check caught the gap; the job-failure reason text that should come from run_arkane_job's own gate never appeared). Tightened the stub to require os.path.isfile(artifact_path) alongside `success`, mirroring the real gate. Full fast suite re-run after tightening: 1035 passed (was 1034 + this test), nothing else broke -- _resolve_artifacts already independently re-verifies output.py's existence on disk, so no production fail-open was hiding behind this stub in the explorer path. The mesolver test's own fake (_fake_run_arkane_job_writing in test_arkane_mesolver.py) always writes output.py unconditionally before returning True, so it does not exhibit the same gap and was left unchanged.
validate_sa_cache() rejected an otherwise-trustworthy cached sensitivity YAML (right hashes, right method, right contract version, parseable) whenever its recorded max_abs_ts_coefficient sat below the absolute floor. That conflated two different questions: whether the CACHE is trustworthy, and whether the SA DATA is useful for criterion (b). The first real-data trial (T3-pdep-qm-trial-001, network13_2) landed exactly this case -- genuine Arkane output with real, non-zero-but-below-floor TS coefficients (max 2.454e-14 mol/J observed, floor ~1.195e-7 mol/J) -- and validate_sa_cache() would have thrown it away and regenerated forever, even though nothing was wrong with the cache itself. The usefulness judgment now lives where it belongs: per-reaction-key, in t3.pdep.selector.select_from_sa_dict, not as a whole-dict scan in the cache layer. max_abs_ts_coefficient is still recorded in the sidecar, but only as human-readable provenance. Tests changed in tests/test_pdep/test_cache.py: - test_cache_with_only_structural_zero_ts_coefficients_is_rejected -> ..._is_valid: a below-floor TS coefficient no longer invalidates the cache; the guarantee that below-floor TS data doesn't produce a false QM-qualification now lives in select_from_sa_dict (FIX2, next commit). - test_cache_with_no_ts_coefficient_recorded_is_rejected -> ..._is_valid: same reasoning; an absent max_abs_ts_coefficient is no longer a cache-validity gate either. - Stale comments in test_network_file_changed_after_write_is_rejected and test_sidecar_with_mismatched_sa_cache_contract_version_is_rejected updated to stop citing the removed floor check as the reason a realistic (non-stub) SA YAML is used. Mutation evidence: re-inserting the removed recorded_max_abs_ts_coefficient-vs-floor block (with a distinctive 'MUTATION-INJECTED-REJECTION' warning) kills both renamed _is_valid tests, confirming they exercise the removed gate and not just the verdict. Fast suite at tip: 1035 passed.
…cord
Round-41 finding: the T-grid clamp was recorded only as a logger.warning. The
sidecar already carried the network hash, SA hash, method and perturbation, but
nothing durable said the sensitivity analysis had been computed over a NARROWED
temperature domain. A saved PDepNetworkSelection could not be told apart from
one resting on the network's original grid -- and a coefficient measured over
[700, 3000] K is not the same measurement as one over [700, 3200] K. A log line
is gone the moment the run ends, so this violated the rule that a decision's
evidence must be reconstructible from the persisted record.
New TGridClampRecord (t3/utils/network_thermo.py) carries the requested Tmax,
the thermo ceiling, the Tmax actually written, whether the explicit Tlist was
dropped and its original highest entry, and which species were skipped when
computing the ceiling. The writers emit it, write_sa_cache_metadata persists it
in the SA sidecar, and it rides on PDepNetworkSelection through the YAML round
trip (dict-shaped, no tuples, per the settled round-trip constraint).
The distinction that matters is THREE-way, not two:
None -> provenance UNKNOWN (sidecar predates this, or
the SA was produced outside T3)
{'clamped': False, ...} -> positively known NOT to have been clamped
{'clamped': True, ...} -> clamped, with both the requested and the
written Tmax recoverable
Collapsing "unknown" into "not clamped" would silently reintroduce the exact
defect being fixed, so absence is never read as a negative. Unknown provenance is
a disclosure, not a disqualification: it does NOT make a decision not_evaluated
and does NOT refuse, since over-refusal is this branch's recurring failure mode.
SA_CACHE_CONTRACT_VERSION deliberately stays 1. The key is purely additive: a
sidecar written before this change is not MISREAD by the new reader, it simply
reads as unknown, which is a true statement about it. A bump would wrongly
invalidate every existing cache to record something no old cache ever claimed.
tests/test_pdep + tests/test_utils: 1185 passed.
tests/test_pdep/test_main_wiring.py: 66 passed (28 min).
fix(pdep): record which RMG-Py actually ran the PDep sensitivity analysis
The SA sidecar has always had somewhere to put the provenance of the RMG-Py
that produced the sensitivity coefficients -- write_sa_cache_metadata accepts
rmg_py_path and stores it alongside get_git_commit()'s answer. Nothing ever
passed it. The only production caller, in determine_species_from_pdep_network,
omitted the argument, so every sidecar a real T3 run has ever written records
rmg_py_path: null and rmg_py_commit: null. The field was not unreliable; it was
dead.
That is not a cosmetic gap. A real PDep trial run was carried out against an
Arkane predating the fix that makes sensitivity meaningful for ILT-based path
reactions, so its transition-state coefficients were numerical noise nine orders
of magnitude below the selection floor. The run looked merely negative rather
than invalid, and it stayed that way for a week -- because the one field that
would have identified the Arkane it came from was empty. Provenance earns its
keep only when something fills it in.
Resolve the path from sys.modules['arkane'] rather than from configuration.
Arkane runs in-process (rmg_runner imports arc.statmech.arkane and calls it), so
after a successful job the interpreter already holds the exact module that ran;
its __file__ names the checkout with no guessing and no second source of truth
to drift. A dict lookup is not an import, so T3 still never imports arkane or
rmgpy. When Arkane has not run in this process there is nothing truthful to say
and the answer is None, which is what the sidecar recorded before and what it
will go on recording -- the difference is that null now means "no Arkane ran"
instead of "nobody asked".
The commit is deliberately recording-only. Rejecting a cache whose commit no
longer matches the checked-out RMG-Py is a tempting next step and the wrong one:
it would invalidate every cached network on any RMG-Py commit, including commits
touching nothing this code depends on, and mass re-running master-equation jobs
to satisfy a hash is the over-refusal failure this package keeps having to
resist. A human reading the sidecar is the consumer here, as with the budget
record.
The test that matters is the wiring one. The fast suite passes with the new
call-site argument deleted -- 1329 tests, all green, against code that had
silently resumed writing nulls -- because the existing coverage exercises
write_sa_cache_metadata directly and never the caller that was broken. The added
test drives determine_species_from_pdep_network down the Arkane-succeeded path
with a stub arkane module installed, and fails on that deletion.
fix(pdep): read the RMG-Py provenance from Arkane's log, not from this process
The previous commit resolved the RMG-Py behind a sensitivity analysis by looking
up sys.modules['arkane'], on the stated premise that Arkane runs in-process.
That premise is wrong. ARC's run_arkane builds a bash script and invokes
micromamba run -n rmg_env python -m arkane input.py, so the work happens in a
subprocess in a different conda environment, and nothing of the RMG-Py that did
it is ever loaded here. T3 cannot import it either, by design and in practice
alike. The resolver therefore returned None on every real run, and the sidecar
went on recording the same null it recorded before -- the fix was ineffective
rather than harmful, but it was ineffective everywhere it mattered.
The mistake was not visible from the call site. rmg_runner imports run_arkane
from arc.statmech.arkane, which reads as an in-process call and is in fact a
wrapper around a shell-out. The tests did not catch it because they installed a
stub arkane module in sys.modules and then confirmed that the mechanism worked
against the stub. Mutation testing did not catch it either: deleting the
argument at the call site does fail the test, so the wiring was genuinely
covered. What went untested was the premise underneath the wiring, which no
amount of testing the caller can reach.
Arkane records its own provenance in the log it writes into the output directory
T3 already owns, which makes that log the only first-hand witness to what ran.
Read the commit from there. The value sits on the line after its label, and the
line after that is a commit date, so only the first following non-empty line is
considered and only if it actually looks like a hash; recording an arbitrary log
line would be worse than recording nothing, since it would read as real
provenance to whoever audits the sidecar later. write_sa_cache_metadata takes
the commit directly now, keeping the derive-from-a-checkout path as the fallback
for callers holding a checkout rather than a log.
Checked against the runs themselves rather than only against fixtures. The two
networks re-run today report e720866a, the fork tip carrying both ILT
sensitivity fixes. The trial whose transition-state coefficients were nine
orders of magnitude below the selection floor reports dfc4df86, which predates
both. That is the distinction the field existed to make and had never once made.
Queueing a P-dep network for QM refinement was an immediate, per-network
decision taken inside determine_species_from_pdep_network()'s loop: the
`if selection.qualified:` branch queued the network's uncertain transition
states on the spot. Whether a network is worth its QM cost is a question
about the whole field of candidates, so it cannot be answered while
candidates are still being discovered -- ranking networks against one
another is impossible from inside that branch.
The branch now records a frozen PDepQueueCandidate instead, carrying the
three loop-local values the queueing step needs and that do not otherwise
survive the loop (the parsed network, the Arkane structures block, and the
file the network was parsed from). A second pass after the loop does the
queueing, in the order the candidates were found. No budget or ranking is
applied yet; every qualified network is still queued, so which networks are
refined is unchanged.
This is NOT a pure no-op, and the difference is worth stating plainly.
queue_pdep_transition_states() calls add_reaction(), which cascades into
add_species() for any of the reaction's species T3 does not already know.
The sensitive-well branch of the same loop also calls add_species(). So
deferring the queueing reorders species-key assignment between those two
paths, and since add_species() assigns `key = len(self.species)` and
get_species_with_qm_label() derives the ARC-facing QM label from that key,
species keys and QM labels differ from what the old interleaving produced.
Consequently species.yml, reactions.yml and restart state are not
bit-for-bit comparable across this change. No work is lost: the SET of
species queued for QM is the same, and every one of them is still computed.
One consequence is a contract question rather than mere numbering. A
species reachable from both paths gets a key back from whichever path
reaches it first and None from the second, and only the well path appends
to the returned species_keys. With queueing deferred, the well analysis now
always reaches such a species first, so a species justified by a sensitive
well is always reported -- which is the intended reading of a return value
documented as "species determined to be calculated based on SA". The second
test below pins that deliberately.
Both new tests were mutation-verified against the same mutation, putting
the queue call back inside the loop:
- test_queueing_is_deferred_until_every_network_has_been_evaluated
explores two entries and asserts every queue call saw both selections
already recorded ([2, 2]; the mutation yields [1, 2]).
- test_a_species_justified_by_a_sensitive_well_is_reported_even_when_a_
queued_ts_also_needs_it fails under the mutation because the queued
TS1 reaction claims C4rad(5) first.
Green: 1191 (tests/test_pdep + tests/test_utils, ignoring test_flux.py and
test_main_wiring.py), 67 (test_main_wiring.py, 29m47s), and
tests/test_main.py -k pdep.
Whether a network is worth its quantum chemistry is a question about the
whole field of candidates, and the previous answer was a per-network gate:
every qualified network was refined, however many there were and however
much they cost. This adds the other half -- rank the qualified networks
against one another and refine as many as the configured budget allows,
most deserving first.
Two knobs, both defaulting to None (no limit), so the budget is opt-in and
setting nothing refines exactly the networks T3 refined before:
t3.sensitivity.pdep_QM_max_transition_states -- the primary bound, since
ARC jobs are per-species and per-transition-state;
t3.sensitivity.pdep_QM_max_networks -- a secondary cap on whole networks.
Both are strict ints: bool is an int subclass, so without that a YAML
`pdep_QM_max_networks: true` would validate as 1 and silently cap the run
at a single network per iteration.
The ranking already existed in t3.pdep.api.rank_pdep_networks as a local
_rank_key. It is now t3.pdep.selector.selection_rank_key, so the in-run
queueing and the public ranking API cannot drift apart into two different
ideas of "most deserving".
t3/pdep/budget.py is pure: it reads decisions and returns indices. That is
not tidiness. Projecting a network's cost by actually queueing it would
mutate T3's species, reactions and QM state, which is precisely what a
budget has to be able to decide against before committing to it. Three
further properties are load-bearing and are argued in the module docstring:
It never slices a network. uncertain_ts_labels() is sorted by LABEL, not
by sensitivity, so taking "the first k" of them would choose quantum
chemistry alphabetically. A network that does not fit is skipped whole
and the walk continues, so a smaller, lower-ranked network can still be
admitted rather than leaving budget unspent.
Its cost is an upper bound. Some uncertain transition states will not
become ARC jobs (unsafe label, missing structures, a shared transition
state, a reaction T3 already knows), and they are charged anyway, so the
budget can only over-estimate the spend it authorizes.
A network offered several times in one iteration -- the normal case,
since several sensitive reactions can belong to one network -- is charged
once, for the UNION of what its offers name, and ranked by the best of
them. Two offers are not one decision reached twice: each answers "which
transition states is THIS observable reaction's rate sensitive to?", so
they legitimately name different transition states, and queueing both
produces their union.
A network no offer could evaluate is refused before it is ranked. It is not
a cheap candidate, it is an absent measurement: there is nothing to rank it
by and nothing to justify spending on it. A network whose cost exceeds the
entire budget is refused with a distinct reason naming the limit to raise,
because no remaining budget is ever larger than the whole of it and no
later iteration will change that on its own -- without that message it
would simply starve, quietly, forever.
Nothing is dropped silently. Every refusal is recorded in the returned
budget decision and logged as a WARNING naming the network, its cost and the bound that refused
it, because a run that quietly shrank its own work would be
indistinguishable in the logs from a run that had nothing left to do.
Note what this commit does NOT do. main.py still offers only QUALIFIED
networks to the budget, so the hard gate survives one level up, and
explore_pdep_network still refuses an evaluated-but-unqualified network
outright. Reframing that gate is a public-API change and is deliberately
left to its own commit. Also, a budget refusal is durable only in the log,
not on the persisted selection; recording it there bumps the selection
schema version and is likewise left separate.
Ten mutations were applied and all ten killed: charge the first offer
instead of the union; rank by the first offer instead of the best; group
unnamed selections together; delete the not-evaluated refusal; delete the
exceeds-the-whole-budget refusal; drop strict=True; hardcode the knobs to
None at the call site; queue every candidate ignoring admitted_indices;
drop the sort; turn skip-and-continue into stop-at-first-miss.
Green: 1247 (tests/test_pdep + tests/test_utils + tests/test_schema,
ignoring test_flux.py and test_main_wiring.py) and 70 (test_main_wiring.py,
30m23s).
…tion Third and last part of the gate -> ranking reframe. 354476f gave T3's in-run queueing the whole field to decide with; e43349d ranked that field against a budget. Both stopped at T3's own wiring: explore_pdep_network(), the public entry point, still treated selection.qualified as the sole admission authority, so a caller that had ranked a network and chosen to spend on it had no way to say so. It now takes a keyword-only admission_policy. Under 'caller_admitted' BOTH qualification checks stand aside -- the unqualified skip and the not-evaluated raise -- because both are about using `qualified` AS A GATE, and a caller that admitted the network elsewhere is not using it as one. Under the default, 'qualified_selection', nothing changes: with no external admission there is no positive evidence and no spend decision, so a missing evaluation is still a missing verdict rather than a negative one. What does NOT stand aside is provenance. method, network_id and network_source_hash stay unconditional, and evaluation_status is now checked for validity unconditionally too -- admitting a network is a budget statement, and no budget statement makes a stale or unreadable decision current. That is the whole difference from selection=None, which drops the binding along with the gate. Keyword-only because logger is the fourth positional parameter and an argument inserted before it would land on callers' logger. The result records what admitted it, since a 'succeeded' result carrying an unqualified selection is otherwise indistinguishable from a bypassed gate. A third value, 'ungated', covers selection=None: recording the argument default there would have the record assert that a qualified selection admitted a run for which no selection existed -- a false provenance claim, and the one an auditor of an expensive QM run would lean on. It is derived rather than requested, so a caller cannot claim 'ungated' while passing a selection, and 'skipped' is cross-checked against it because only the gate can decline a run. A results file written before this field is DERIVED, not refused and not blanket-defaulted: nothing predating the field could have been caller-admitted, so a record with a selection was gate-admitted and one without was ungated. That reconstructs the true value instead of guessing, and keeps EXPLORATION_RESULT_SCHEMA_VERSION 1 meaning one loadable shape. Public API change; PR #179's description needs it. T3's own path does not use the new policy yet -- t3.main still offers only qualified selections to the budget -- so in-run behaviour is unchanged. Also de-drifts prose that still called this a budget gate, including two claims in selector.py that stated flatly that explore_pdep_network refuses a selection that did not qualify. Verified: 1264 fast tests, and 70 in test_main_wiring.py against this exact tree. Fourteen mutations applied, fourteen killed, all against the final tree: the skip ignores the policy; the raise ignores it; the hash check becomes policy-conditional; the evaluation_status check becomes policy-conditional; the recorded value echoes the argument instead of being derived; 'ungated' becomes requestable; the loader defaults, or blanket-defaults, instead of deriving; the succeeded and failed sites drop the field; an unknown policy falls back to the default; caller_admitted with no selection is allowed as a no-op; the result skips policy validation; the skipped cross-check is removed; as_dict drops the field. Codex round 44 rejected the bare require_qualified bool this started as -- a flag naming what is not enforced hides what did admit the run -- and round 45 caught the false 'qualified_selection' on the selection-less path.
A network that qualified for QM refinement and then went unrefined because the budget was spent left nothing behind but two log lines. That is the one outcome a long unattended campaign most needs to be able to reconstruct afterwards, and it was the only decision in this package that vanished with the process. The record covers admitted and refused networks alike, not refusals alone. Recording only refusals would collapse "no file", "empty file", "nothing was refused", "nothing was a candidate" and "no budget was configured" into a single ambiguous silence, and a reader could not tell a network that was admitted from one that was never considered. Refusal reasons gain stable codes alongside their prose. The prose is written to be read by a human in a log and is free to be reworded; anything that has to branch on why a network was refused needs a key that is not a sentence. The decision now carries the walk it already made -- the ranked identities, their offers, costs and remaining budget -- and the record is built from that alone. Deriving them a second time from the selections would have meant two independently maintained copies of the same grouping and ranking, and on drift the failure is not a crash but a record that confidently attributes the wrong refusal to the wrong network. Since the builder no longer takes the selections, handing it a decision and a field that never met is now unconstructible rather than merely unlikely. A network the budget identifies positionally because it has no id is recorded that way too, rather than as a repeated empty id the record then rejects; the budget accepts such networks, so refusing to describe them would have made the record fail on input the decision itself allows. Both a schema version and an algorithm version, following the split already in selector.py: this record's shape can stay fixed while what the budget means by ranking and refusal changes, and a reader has to be able to tell those apart.
The record type existed but nothing wrote it, and a save/load pair with no production caller is not durability -- it is the same silence in a different place, which is precisely why writing the outcome onto the selection was rejected: T3 never persists selections either. So t3/main.py now writes the record itself, into the iteration directory, alongside the other per-iteration sidecars. It is written whenever the budget runs, including when nothing was refused and when there was nothing to consider at all, so that the absence of the file carries a meaning that is actually true: the feature was off this iteration, not that it had nothing to say. For that to stay true across a re-run, a record left by an earlier run of the same iteration is cleared on the path where the feature is off, the way a superseded ARC finalization marker already is -- otherwise a stale file would go on describing a decision this run never took. It is written before the queueing loop rather than after. The record describes what the budget decided, not what queueing later managed to do, so it should outlive a crash in the step it authorizes. That only holds if the file is never half-written, since a truncated record can still parse and still validate while under-reporting what was refused, so it is staged beside its destination and moved into place, as the capture manifest already is. Replacing the path rather than writing through it also means a symlink sitting where the record belongs is replaced instead of followed. The loader is strict in the manner of the two loaders already here: an unversioned file, a version it does not recognize, a malformed envelope and a malformed record are all refused rather than guessed at. Versions are compared as integers, since Python would otherwise read `true` as version 1. The record carries its own schema and algorithm versions and is never nested inside another persisted structure, so those fields serve as the file's version markers and no second envelope-level copy is introduced to disagree with them. One existing wiring test had hand-built a budget decision that reported a refusal while carrying no account of it. That is exactly the incoherence the record refuses to render, so the test now builds a decision that is consistent with itself.
The gate had never been tested against data it did not generate. Vendor two complete unmodified chains from a real run and assert what each decision MEANS, not just whether it qualified. The pair is chosen because it disagrees. network21_1 (CSE) responds five times more strongly than network799_1 (MSC) and still does not qualify, because all three of its path reactions come from a reaction library and none of its kinetics are uncertain; network799_1 qualifies with every one of its selected transition states uncertain. Qualification is sensitivity TIMES uncertainty, and a gate that ranked on sensitivity alone would invert both verdicts. A hand-built fixture would almost certainly make the two agree and would pass against that broken gate. Assertions are on the semantics -- the (transition state, path reaction, uncertainty) triples, the counts, the thresholds, the cache status, the absence of warnings -- because the boolean alone is nearly worthless here. Mutation testing, applied to production code and reverted: qualified ignores uncertainty killed by 2 tests absolute coefficient floor removed killed by 5 tests, BOTH BOOLEANS UNCHANGED path-reaction join shifted by one killed by 2 tests, BOTH BOOLEANS UNCHANGED reaction-library certainty marker removed killed by 2 tests The two mutations that leave `qualified` untouched are the argument for this shape: a fixture asserting only the verdict would have shipped green against both. The floor assertions are arithmetic rather than vibes: the raw SA files offer 36 transition-state rows each, of which 7 and 16 fall below the floor, and the selector keeps exactly 29 and 20. What survived is what was offered minus what the floor excluded, so a gate discarding rows for some other reason breaks the equality even if the survivor count coincides. Provenance is recorded rather than asserted into existence. Both arkane.logs name RMG-Py e720866ae, which carries the two ILT sensitivity fixes; without them these coefficients would be noise nine orders below the floor, which is exactly how an earlier experiment looked merely negative for a week. The t3_sa_cache.yml sidecars still record rmg_py_commit: null because they predate 2c91568 -- that null is now asserted by a test so nobody "fixes" it by inventing a commit. Nothing binds a log to the SA file beside it, and the test says so instead of overclaiming. A method-level branch of the uncertainty predicate survives mutation. That is structural, not a gap in the fixture: selector.py:1082 passes only kinetics_comment, and the network parser records kinetics_type (the callee name), never a kinetics method -- so no real-network fixture can reach it. Left to unit tests, where it belongs. Test-only; no production code changed. Fast suite 1331 -> 1345.
… qualified
Until now the only in-run trace of a PDep network's fate was appended at one site deep
inside the success path, so every fail-closed exit before it left the network with no
record at all. On a real 12-network trial that meant 7 networks -- the majority -- vanished
silently while the log's "3 qualified" read like an unqualified success. Fail-closed was
and remains correct; the defect is that refusing to answer looked identical to never having
been asked.
This is the first half of that work: the record type and the diagnosis that populates it.
The main.py funnel that produces the records follows separately.
PDepNetworkAssessment is a new OUTER type rather than a reused PDepNetworkSelection.
A selection is rankable, and selection_rank_key sorts a not_evaluated one into tier 1,
ahead of evaluated negatives -- right for a selection, wrong for a placeholder, since it
would make "we never found out" outrank "we checked and it does not qualify" for every
future consumer. The selection is kept as an optional NESTED payload instead.
The record's load-bearing property is that it cannot disagree with itself. Each of the 21
reason codes implies exactly one status and one rule about the nested selection, and
construction refuses any combination that violates either. A durable file is believed, so
one claiming a selector verdict for a network whose SA never parsed would mislead with the
full authority of provenance.
The vocabulary lives in its own module with no t3 imports. It is shared by a producer (the
selector, reporting which refusal site fired) and a consumer (the assessment, persisting
it), and putting it in either would force a dependency between them -- with the codes in
assessment, selector had to import persistence, which in turn stopped assessment from
importing PDepNetworkSelection to type-check its own nested payload. That left `selection`
typed as `object` and duck-checked, so any value at all passed construction and only failed
later at serialization. The neutral module lets the dependency run selector -> reason_codes
<- assessment, and the nested payload is now type-checked.
Two malformed-payload codes exist because one could not serve both callers: T3's funnel
inspects `structures` before it can build a network reaction string, so a non-mapping SA
payload is caught there and never reaches the selector, while the selector's public entry
points can be handed one directly. A single code would have forced the funnel to either
fabricate a selection or violate the type.
An internal error gets its own status rather than sharing not_evaluated. It is a bug, not
an outcome, and counting it alongside legitimately unevaluable networks would make a T3
defect read as the ordinary cost of doing business. It is also the one category for which a
nested selection is optional -- a crash after the selector returned should keep the evidence
already obtained.
Where two selector refusals both apply they are now BOTH recorded. Discarded rows and an
unassessed transition-state provenance are independent defects in the evidence -- a
transition state can fail to join a path reaction whether or not other rows were unreadable
-- so reporting only the first would undercount the second wherever these are tallied.
The record is frozen and the nested selection is deep-copied. Freezing alone would not be
enough: a selection is mutable, so a caller still holding the original could flip
`qualified` afterwards and leave a validated record quietly self-contradictory.
Verified by mutation, each failing at least one test: dropping the status/reason cross-check,
the selection-presence check, the selection type check, the deep-copy snapshot, the frozen
decorator, the bare-string guard (tuple('MSC') is ('M','S','C')), the bool guards on
iteration and schema_version, the first-cause suppression, moving sa_output_malformed
between categories, collapsing internal_error into not_evaluated, and swapping or inverting
reason codes at four selector sites.
Fast suite: 1906 passed. The 4 failures in tests/test_main.py and tests/test_functional.py
are pre-existing at d251414 -- confirmed by running them in a detached worktree at that
commit -- and are stale set_paths expectations from the 'PDep QM budget' key plus a cantera
environment issue, not regressions from this change.
Freezing the record turned out to freeze the reference and not the object behind it, so
every invariant above could be undone the instant construction returned -- and it was the
undone state that got serialized. The deep copy shut the caller's alias and was mistakenly
believed to shut the record's own copy too, which it hands out freely through
record.selection. The rendered snapshot taken at construction is what makes the claim true:
as_dict() serializes it, so no later mutation reaches the file, and it refuses outright if
the live selection has drifted rather than quietly persisting one story while the record in
memory tells another.
The nested selection must also now be about the SAME network as the record. Six fields are
carried by both, and nothing checked any of them, so a record could nest another network's
evidence -- exactly the misattribution t3/pdep/reason_codes.py's own docstring says the
category rules exist to prevent. network_id must match exactly, since that is the identity
claim itself; the other five need only not conflict, because absence is not disagreement and
the funnel legitimately knows things the selector never recorded. That distinction is not
cosmetic: requiring both halves to be populated made 26 tests fail, since it refuses
ordinary records.
Adds ASSESSMENT_ENVELOPE_SCHEMA_VERSION, used by the persistence commit that follows.
…use to read one that lies The record type from the previous commit had nowhere to live. This adds the two functions that put it on disk and take it back off, so the next commit's funnel has somewhere to write the fate of every network it looks at. The write is atomic and durable -- staged in a private directory beside the destination, flushed, renamed, and the destination directory flushed in turn -- which save_pdep_network_selections is not. The difference is not stylistic. The funnel rewrites this file once per network as an iteration progresses, so a crash or a full disk part-way through a write is an ordinary event rather than a theoretical one, and this is precisely the file whose job is to survive the failure that interrupted it. A truncated but still parseable record would be read back as authoritative and would under-report exactly the networks whose silent absence this increment exists to fix. The rename also closes the symlink write-through vector, as the budget record's write already did. Atomic is not the same as durable: os.replace orders the rename but says nothing about whether the bytes behind it ever reached the disk, so without the two flushes a power loss can leave the rename applied and the contents gone -- the file present, and empty or torn. Staging inside a fresh 0700 directory rather than beside the target (as the budget writer does) also closes the window between creating the staged file and reopening it by path to write, during which anything else with access to that directory could substitute a file at the staged path. The list is wrapped in a versioned envelope for the reason the other list-savers give: an iteration in which T3 finds no P-dep networks at all is an ordinary outcome, and an empty list has no record of its own to carry a version. The envelope's version is its OWN constant under its own key rather than a second copy of the per-record one. save_pdep_network_selections does reuse a single number for both roles, but that makes each version a hostage of the other: adding a field to a record would force the envelope to claim a change it never underwent, and renaming the list key would force every record ever written to be re-stamped. A version number that has to lie is worse than none. The loader checks the two separately, so a file cannot claim one shape at the top level and hold another underneath. The loader needs no companion cross-field validator, unlike _selection_from_dict. Every invariant that makes an assessment trustworthy already lives in PDepNetworkAssessment's __post_init__, so constructing one re-checks the whole record: a hand-edited file claiming a selector verdict for a network whose SA never parsed is refused by the same rule that refused the impossible combination at the site that would have written it. Construction is wrapped only so the refusal names the file and the entry -- the type's own message names the network, which suffices where it is written but not where it is read, with a dozen iterations on disk. The per-field helpers already name both and are left unwrapped rather than given a second prefix. An absent 'selection' key is refused rather than read as null. as_dict() always writes the key, rendering it null where the reason code forbids a nested selection, so absent and null are different claims: guessing null would quietly convert a corrupt record into a plausible "never evaluated" one, which is the misreading the file exists to prevent. cache_status is read as a free optional string even though the selection loader enum-restricts its own, because PDepNetworkAssessment accepts any string there. A loader stricter than the constructor that feeds it would make a record T3 legitimately wrote unreadable by T3 -- the one failure a durable record must never have. Strictness not shared with the writer is not safety; it is a bug with a delay on it. Nested selection versions are now bool-guarded like every other version on this branch: True == 1, so `selection_schema_version: true` was being read as correctly versioned. One gap is pinned by test rather than fixed, and deliberately. PDepNetworkSelection is mutable by design -- the selector builds it across some twenty assignments -- so it validates almost nothing, while _selection_from_dict is strict; a selection mutated into e.g. a bare-string `warnings` therefore saves and then cannot be loaded back. It is not reachable from T3's own paths, and validating at construction would not help since the mutation happens afterwards. The real fix is one validation contract shared by writer and loader, which is its own increment. The test exists so the gap stays visible and cannot silently widen. Also drops an unused dataclasses.field import left in assessment.py. 78 new tests. Beyond the round trips, the ones that discriminate rather than merely describe: a save whose underlying write raises leaves the previous record intact rather than truncated, a save onto a symlink replaces the link instead of writing through it, both flushes happen in the right order around the rename, and the staging directory is unreachable by anything else. Verified by mutation, each failing at least one test: dropping either fsync, staging beside the target instead of in a private directory, collapsing the envelope key back onto the record key, dropping either bool guard on the nested selection versions, and reading an absent 'selection' key as null. pdep suite: 1433 passed, up from 1355 at d251414.
Adds the 'PDep network assessments' iteration path key and clears a stale record when the
feature is off, so the funnel that follows has somewhere to write and cannot inherit another
run's answers. No funnel yet -- nothing writes this file at runtime.
It is a sibling of 'PDep QM budget' rather than part of it, because the two answer different
questions over different populations: the budget describes the networks that QUALIFIED --
which were admitted, which refused, and why -- while this describes every network that was
looked at at all. A network that failed before the selector ran never reaches the budget, and
those are precisely the ones that used to vanish without trace.
Stale-clearing mirrors _clear_pdep_budget_record and matters more here. This record's whole
purpose is to be believed about which networks were never evaluated, so a file left over from
an earlier run of the same iteration would not merely be stale, it would answer that question
with another run's networks. Clearing is the only thing standing between "no record" and "a
confident wrong record". It is narrow in the same way: the path is always the single
set_paths-derived key, never constructed or caller-supplied, and removal happens only after
os.path.isfile confirms a regular file.
Fixes the three long-standing failures in tests/test_main.py, which were failing before this
branch and are now green. Two of them turned out NOT to have the cause they had been recorded
as having. Only test_set_paths was stale on the path key ('PDep QM budget', added in
increment 32 and never added to the test's literal). test_as_dict and test_args_and_attributes
were stale on increment 32's SCHEMA additions instead -- pdep_QM_max_transition_states and
pdep_QM_max_networks were missing from the shared expected sensitivity dict. Same increment,
different omission; the diff says so plainly once read rather than assumed.
Fast suite: 1987 passed, 1 failed. The single remaining failure is
tests/test_functional.py::test_computing_thermo, a cantera environment issue unrelated to
this branch, down from 4 before this commit.
…ng one file end the run
On a real twelve-network run, seven networks -- the majority -- left no trace of what T3 decided
about them, because the only record was appended deep inside the success path. "Assessed and found
not worth refining" and "never assessed at all" were the same silence afterwards, and the second is
the one an operator has to act on. Four of the paths producing that silence were uncaught
exceptions that ended a multi-day RMG+ARC campaign over a single unreadable file.
Route every offer through one funnel, `_assess_pdep_network_candidate()`, which has no bare
`continue`: every path out of it produces a `PDepNetworkAssessment`, so a network cannot be dropped
by adding an exit site. The four crash paths become recorded outcomes -- writing the Arkane input,
reading the sensitivity YAML (missing and malformed), and mapping species labels -- and the version
arithmetic that used to raise on a stray `network4_backup.py` now ignores anything that is not a
versioned network file rather than either dying or discarding the real network beside it.
The catches are narrow and the line runs between DATA and CODE, not between convenient and
inconvenient. A missing or unreadable network file is a fact about this run's data at either site
that reads it, so the input writer's `OSError` is an outcome exactly as the parser's already was;
which of the two happens to touch the file first must not decide whether the campaign survives. A
`TypeError` is a fact about the code: it is recorded under its own status, never counted among the
networks that legitimately could not be evaluated, and re-raised. Recording it is guarded, because
this handler runs when something is already wrong enough that the record may be unbuildable, and a
complaint about writing the diagnosis down must not replace the diagnosis.
Two failures that were per-METHOD were being treated as per-network. The sensitivity artifact is now
opened inside the master-equation loop: a method that reports success and leaves nothing readable
behind says nothing about whether the next method would, and it no longer spends their turn.
Where methods break in different ways those are independent defects in different artifacts, so all
of them reach the record rather than whichever ran last.
The record is written after every network and marked incomplete until the pass ends. A file holding
four of twelve networks is not a smaller truth, it is a different claim, and it is the one
corruption that cannot be detected by inspecting the records, since each of them is individually
valid. The loader refuses an unfinished file unless asked for one. The pass also stakes its claim on
the file before doing any work, so a `complete: true` record from an earlier run of the same
iteration cannot survive a crash and answer "which networks were never evaluated?" with another
run's networks.
Both verdict lists now describe one iteration. Carrying them over was never a decision, and a
network re-examined in three iterations was reported three times, against a model that had changed
underneath each one.
fix(pdep): stop an unreadable file ending the campaign, and check the cache before writing
Three defects on one path, found by probing the last open backlog item -- "write_pdep_network_file
runs before validate_sa_cache, so a ValueError there denies a network its cached SA". The ordering
was the least of it.
`validate_sa_cache` promised to return a status and did not. It guarded both content hashes with
`os.path.isfile`, which answers whether a file exists, not whether this process can read it, and
which does not close the window between the check and the open. An unreadable file, or one
unlinked by a concurrent run, threw an OSError out of a function documented to return
`'cached_valid'` or `'cached_rejected'`. At the `t3/main.py` call site that is not covered by the
surrounding `(OSError, ValueError)` handler, so it reached the outer `except Exception`, which
records an `internal_error` and re-raises: one unreadable file ended the whole campaign.
`main.py` already states the governing rule at its own read sites -- a network file that is missing
or unreadable is a fact about this run's DATA, not a bug. Both hashes now honour it. So does
`_read_pdep_sa_output`, which runs immediately afterwards on the same path and had the same hole;
`sa_output_unreadable` ("exists but could not be read") was already the right code for it, and
catching the error in the validator while letting it escape two lines later would only have moved
the crash.
The Arkane input is now rendered only when it is going to be used. It was written before the cache
was consulted, which cost two things. A network whose output directory could not be written --
read-only, full, quota -- lost a cached SA that was perfectly valid and that using required writing
nothing at all, because `os.makedirs` and `shutil.copyfile` run before any parsing and are facts
about the destination, not the source the cache is validated against. And on a cache hit the
rewrite left an `input.py` that need not be the one which produced the `sa_coefficients.yml` beside
it -- a divergence neither `network_file_hash` nor `sa_file_hash` covers. On a hit nothing is
written now; the isomer labels the writer used to return are read from the network file's own AST
instead, which agrees with the writer's quote-scan on all seven real network fixtures.
That read is opportunistic and never fatal. It feeds only `executed_networks`, which is appended in
two places and read nowhere, and the authoritative parse downstream deliberately records
`network_parse_failed` while still letting the well analysis run, since that analysis needs the SA
and the label map rather than the parsed network. Returning early here would have denied a network
holding a valid cached SA the very species that analysis would have refined -- to populate a list.
Finally, the `not_evaluated` placeholder built when a cache is rejected no longer carries
`t_grid_clamp` provenance read from that same rejected sidecar. The field means "the T grid this
decision rests on", and this decision rests on nothing. `tests/test_pdep/test_api.py` pinned the
old behaviour deliberately and is inverted here, on the owner's call.
Reads of the network file are assumed not to race with a concurrent writer; T3 owns the RMG output
tree for an iteration. Closing that window means hashing the source once and threading the bytes
through the writer, the parser and the sidecar -- a change to the cache contract, noted in
`_assess_pdep_network_candidate`'s docstring rather than attempted here.
… back `_read_persisted_yaml_file` already asserted the invariant -- "any file containing [a Python object tag] is not a file T3 wrote" -- but that was an assertion about the writers made on the read side, and nothing enforced it on the write side. `arc.common.save_yaml_file` renders with a full representer, so a single non-plain value in a record (a `pathlib.Path` handed to a `str`-annotated field is the realistic way in) is written out as a `!!python/object/apply:` tag, the write reports success, and every loader here refuses that file from then on. The failure being prevented is TOTAL LOSS, not a bad record: `yaml.safe_load` fails before any per-record check runs, so one bad nested field costs the whole iteration's assessments and reports a YAML tag rather than the field at fault. `_refuse_content_that_would_not_parse_back` renders through `to_yaml` -- the same function the write uses, custom string representer included -- and parses that back. Checking with `yaml.safe_dump` instead would approve bytes other than the ones written, which is the shape of the bug itself. Called by all four writers, before staging in the two that stage, so a refusal leaves no droppings. It does NOT make the loaders accept the file: a `str` where a list belongs is good YAML, refused later per record with a message naming the field. `PDepNetworkSelection` is the only record type written here that does not type-check its own fields in `__post_init__`; closing that is its own work. The `except yaml.YAMLError` catch stays narrow on purpose. A `RecursionError` from a self-referential record is a defect in the code that built it, not a fact about the data, and filing it as a refusal would hide a bug inside a provenance record.
`PDepNetworkSelection` and `SensitiveTransitionState` were the only record types in `t3/pdep` that validated nothing, while `_selection_from_dict` and `_sensitive_transition_state_from_dict` have always been strict. The gap between a permissive constructor and a strict loader is a record that can be built and written but never read back -- and a selection is nested inside both an exploration result and a network assessment, so one bad field cost whichever file carried it. Both now hold the contract their loader enforces. The selection's lives in `validate()`, called from `__post_init__` AND from `as_dict()`, because a constructor check alone would be a guarantee in name only: the record is mutable and `select_from_sa_dict` builds it blank and fills it in across ~20 assignments, so construction sees an empty record. `as_dict()` is the one funnel every persisted copy passes through. Two things the suite proved wrong on the way: `network_id` is NOT required. `rank_pdep_networks` records a decision for an entry too malformed to name a network, and `t3.pdep.budget` counts two such records as two distinct networks rather than collapsing them. The LOADER was relaxed to match -- so a file written by `rank_pdep_networks` + `save_pdep_network_selections`, two public API functions that pair, is no longer refused outright by `load_pdep_network_selections`. The assessment record's `network_id` stays required: it is a statement about a named network. `combine()` then raised `TypeError` instead of its intended `ValueError` on a set mixing `None` with a label, because `sorted` cannot order them. Sorted by `repr` now -- the mixed case is exactly what that refusal is for. Also: `VALID_CACHE_STATUSES` is now shared with the loader rather than respelled there; the `network_id` annotation says `str | None` and its docstring records that distinctness of unnamed records is positional, not identity-based, so a consumer joining on it collapses them all.
…m a file `PDepNetworkSelection.t_grid_clamp` is documented as a `TGridClampRecord.as_dict()` rendering, and the only check anywhere was "a dict or None". It is also the one field whose value is read off disk -- `t3.pdep.api` calls `read_t_grid_clamp_record(sa_path)` unconditionally and copies whatever it returns into up to four live selections, which then persist it as their own provenance. Nothing in t3/ or tests/ reads a key back out of that dict. That makes a malformed value worse rather than more tolerable: there is no failing consumer to reveal it, so the only symptom is someone opening a saved decision much later and believing what it says about the solve behind it. `TGridClampRecord` now type-checks its own fields, and `t_grid_clamp_shape_error` asks the same question of a plain dict, returning a reason rather than raising because its callers need opposite outcomes: `read_t_grid_clamp_record` collapses a malformed sidecar dict to None (unknown provenance -- its documented contract is to disclose, never to gate), while `PDepNetworkSelection.validate()` and the selection loader refuse one. What makes that leniency safe is the other end: `write_sa_cache_metadata` now refuses to record provenance its own reader would discard, so a possibly-foreign file is read leniently and a file T3 writes is written strictly. Only `clamped` is required and unrecognized keys are carried. Sidecars and selection files outlive the version that wrote them in both directions, and refusing either an older writer's missing key or a newer writer's extra one would discard honest provenance -- the loss this record exists to prevent. Tolerating a key means saying nothing about its value, which is what leaves the write-time plain-YAML backstop its remaining live reach. Also from the round-61 review: `nan`/`inf` are refused as temperatures (a `nan` renders, reloads, and then compares unequal to itself, so a record carrying one can never be shown to have round-tripped); a `list` passed for `skipped_species` is normalized to a tuple, since keeping it would let a caller append after construction and render entries the contract never saw; and `combine()` no longer adopts the first component's provenance without comparing, matching what `method` and `cache_status` beside it already do. Cross-field invariants were considered and declined: both temperatures are read from the same parsed line, so the real writers cannot produce an incoherent record, and a semantic rule would fire only on a foreign one -- where, on the sidecar path, it would silently drop provenance. The docstring states that boundary and a test pins it.
…nothing can use `T3Sensitivity.check_me_methods` compared `entry.lower()` against the three method names and then returned the value unchanged, so `ME_methods: ['cse']` in a user's YAML validated. Nothing downstream of it is case-insensitive. `t3/main.py` reads the list straight out of `InputBase(...).model_dump()` and hands each entry to `write_pdep_network_file`, which reaches `METHOD_MAP[method]` in `rewrite_arkane_method_line` -- a bare `KeyError: 'cse'` that the `(OSError, ValueError)` handler around that call does not catch, so a lowercase entry ended the whole run with a traceback naming neither the input file nor the field. The same string is also used verbatim as the per-method output directory name, as the `method` written into and compared against the SA cache sidecar, and as `requested_me_methods` provenance. The validator now returns the canonical `METHOD_MAP` key. Case-insensitive acceptance is kept rather than tightened away: it is deliberate, it matches `global_observables` beside it whose consumers genuinely do read case-insensitively, and no working configuration exists today with lowercase -- it crashed. The repetition check moved after canonicalization, so `['CSE', 'cse']` is still refused as one method spelled two ways. `METHOD_MAP` moves from `t3/utils/writer.py` to `t3/common.py`. Validating input must not drag in the writer's dependency stack (Mako, ARC species perception, the generator, the thermo reader), and `t3/pdep/api.py` imports `T3Sensitivity` only for its defaults, so it would have inherited all of it. `t3.utils.writer` re-exports the constant, leaving every existing `from t3.utils.writer import METHOD_MAP` call site untouched. `write_arkane_network_input_file` now refuses an unknown method before it touches the disk. `rewrite_arkane_method_line` already failed on one, but only after the destination directory had been created and the source copied into it -- leaving a plausible `<network>/<bad-method>/input.py` still carrying the SOURCE file's method, a different solve from the one its own directory name claims. The four other sites that render a method already check up front; this one was the exception.
`_extract_payload_numeric_leaves` tested `_get_call_name(node) in NESTED_KINETICS_CALL_NAMES`
against a name that was never defined anywhere in the repo, so every kinetics payload containing a
nested constructor call raised NameError out of the parser. Its callers handle OSError and
ValueError; a NameError is neither, so it reached the outer `except Exception` in `t3/main.py`,
was recorded as an `internal_error` and re-raised, ending the campaign. The trigger is ordinary,
not exotic: `PDepArrhenius(pressures=..., arrhenius=[Arrhenius(...), ...])` is what Arkane emits
for `interpolationModel = ('pdeparrhenius',)`.
Every fixture in `tests/data/pdep_me/` uses a flat `Arrhenius` (46) or `Chebyshev` (41) with no
nesting, so 2267 tests ran over a guaranteed crash without touching it. Ruff's F821 found what the
suite structurally could not, which is the argument for the lint job being green.
The constant is a whitelist rather than "any call", because the alternative fails open: recursing
into an unrecognized call would let `array([[1.0, 2.0]])` or `float('nan')` donate their arguments
to the ME-success gate as if they were fitted rate coefficients. Unlisted calls keep falling
through to the `else`, which surfaces them as a None leaf. Being wrong toward "unrecognized" costs
a regeneration; being wrong the other way passes a network whose rates are not real.
Both directions are pinned by new tests. Also drops a duplicate `RECOGNIZED_TOP_LEVEL_CALLS`
binding that sat in the same hunk, and two unused imports ruff removed from the test module.
`ruff check .` reported 51 errors on this branch and 0 on main, so the Lint job had been failing on this branch alone for ten days while I read only the test job. The one live bug among them is fixed in the previous commit; what remains is dead weight. Mostly unused imports, 23 of them in `assessment.py` alone. They look like a re-export surface but are not one: every consumer, including `t3/pdep/__init__.py`, imports those constants straight from `t3.pdep.reason_codes`, so nothing resolves them through this module. Two comments in `cache.py` contradicted each other about whether the derived coefficient floor is used later; it is not, and the docstring already said so. The call stays -- it is there to raise on a bad `min_delta_ln_k`/`perturbation` before any early return can mask it -- but its result is now visibly discarded and both comments say the same thing. `budget.py`'s `unnamed_offer_index` local is removed as redundant, not as a fix: I first read it as a dropped field and said so, then found `build_pdep_budget_record` recomputes the same expression inline at the sole construction site, so the persisted record was always correct. The rest: a discarded `_write_manifest` return, five test locals bound and never read (each of those tests asserts against files on disk, so no assertion was missing), and four f-strings with no placeholders.
…ted constant The three `except ...: pass` clauses CodeQL flags are each correct, and now each says why. Two are label-rendering failures inside a scan over every known reaction: a reaction that cannot produce a SMILES or Chemkin label is not the reaction being looked up, so it is skipped and the scan continues to the next candidate. The third is a lock release that finds the lock already gone, which satisfies the post-condition the function promises and must not raise, since it runs from a caller's `finally` where raising would mask the real exception already unwinding. `energy_settings._MODEL_CHEMISTRY_CALL_NAMES` was a second copy of `hybrid._MODEL_CHEMISTRY_CALL_NAMES`, unread, and held in agreement by a comment saying it was "kept in sync". Nothing enforced that, and this module already imports from `t3.pdep.hybrid`, so the copy could only ever drift into a second answer to a question that has one. Deleted rather than wired up: no caller wanted it. Two of the flagged clauses are pre-existing on main and surfaced here only because earlier commits moved their line numbers. Not addressed, both false positives, left for a human to dismiss: - `py/unused-import` on `t3/pdep/parser.py:25`. Those names ARE used -- `t3/pdep/hybrid.py:39` imports them from `t3.pdep.parser`. The query asks only whether a name is used within its own file, follows no cross-module re-export, and does not read the `# noqa: F401` on every line. Acting on this alert would break `hybrid.py` at import time. - `py/overly-permissive-file` on the `0o644` capture lock. The file holds a PID.
`determine_species_from_pdep_network` bound `arkane = None` and, four lines later, offered an Edge-species fallback behind `elif arkane is not None:`. Nothing reassigned the name in between, so the branch was unreachable from the day it was written. `main` has the same shape (`sa_coefficients_path, arkane = None, None`, tested identically), so this is fallout from the RMG-Py API removal rather than something the PDep work introduced. What is being lost is real and should not be discovered later from an empty diff. The comment on that branch said "this is an Edge species which is missing from the Core rmg_species list": a well whose label is absent from `labels_map` was meant to be resolved against Arkane's `species_dict` instead of dropped. It has always been dropped. Deleting the branch changes no behaviour -- it cannot, being unreachable -- but it removes the last written trace of the intent, so the intent now lives in a comment at the drop site, which explains what used to be attempted and why it cannot be put back in place: T3 no longer imports arkane at all. Reaching those species again means threading something T3 parses itself into this loop, which is design work rather than a repair. Suite unchanged at 2269 passed, 1 failed (the pre-existing cantera test_computing_thermo), as it must be for a deletion of dead code. Found by CodeQL py/unreachable-statement, which reported it as new only because earlier commits on this branch moved the line numbers.
…ture lock Two CodeQL alerts on PR #179, one of which I twice reported to Alon as a pure false positive and which turns out to be only mostly one. `py/unused-import` flagged the five names re-exported from `t3.utils.network_thermo`. Four of them ARE imported from this module elsewhere, so for those the query is wrong: it asks only whether a name is used within its own file, follows no cross-module re-export, and does not read the per-line ruff suppressions. `__all__` says the same thing in the language instead of in a linter pragma, and the query honours it. The list is not hand-guessed -- it is every name any other module imports from `t3.pdep.parser`, taken by parsing the repo's own import statements. That sweep is also what corrected me: `NetworkThermoCeiling` was re-exported here and imported from here by nobody. It is deleted rather than declared. Importers take it from `t3.utils.network_thermo`, which is where it lives. The capture lock file drops from 0o644 to 0o600. It is written by `_acquire_capture_lock` and read only by `_read_capture_lock_holder`, both inside one user's own capture directory, to recover the PID of a possibly-dead holder. There is no cross-user reader to serve, so the narrow mode is the correct one on its own merits rather than a concession to the scanner.
Status
Review gate lifted; the blocking marker that used to head this description has
been removed, from the title as well. One hard prerequisite remains, so do not merge on green
checks alone:
RMG-Py@main,so until it does there is zero CI coverage of the PDep SA path — the critical path of
everything this PR adds. Merging before then puts ~18.7k lines of new
t3/source intomainwith its main path unexercised by CI. #2990 is open and mergeable as of this writing.
Checks:
Lint (ruff),Analyze (python)andCodeQLgreen;Build and Test T3green on theprevious head and re-running after the rebase. Rebased onto
main, 127 commits, no conflicts.Two things a reviewer should weigh, neither of them blocking:
capture_ts_artifacts, the join, and the hybrid rewrite are still exercised only by fixtures.See Testing below for what the live trial did and did not reach.
What this adds (I-008 / D-020)
A T3 layer that decides which whole pressure-dependent (PDep) reaction networks deserve
expensive QM (PES exploration + master-equation refinement), and then computes them. The selection
criterion is observable-sensitivity × uncertainty.
The key architectural choice: this reasoning is network-level (ME solver, PES explorer). ARC
stays per-species / per-reaction and never learns that networks exist. Nothing in ARC was
touched;
arc.*is consumed read-only.The end-to-end capability, and the property the whole capture layer exists to provide:
T3.process_arc_run()writes hybrid Arkane P-dep network inputs — QM/RRKM for transition statesthat were successfully computed, RMG/ILT for the rest — from the capture alone, with the ARC
project directory absent entirely.
New module
t3/pdep/:discovery,join,capture,hybrid,selector,parser,energy_settings,me_success,cache,yaml_safe,api, andmesolver/(adapter + Arkaneimplementation + factory).
Constraints held throughout
rmgpyand noarkaneimports in T3.arc.*only;arc.moleculereplacesrmgpy.molecule..pyfiles are parsed withast.parse, neverexec/eval. Parsing an RMGnetwork.pywould otherwise mean executing the RMG/Arkane DSL.Testing
Full suite: 2269 passed, 1 failed (~55m). The single failure is
tests/test_functional.py::test_computing_thermo(CanteraError) — pre-existing onmain, notfrom this branch.
The suite is not the evidence that matters most here. A
NameErrorsat on the hot path of theArkane-output parser under 2267 passing tests, because every fixture in
tests/data/pdep_me/uses aflat
ArrheniusorChebyshevand the crashing branch only fires on a nested kinetics call —PDepArrhenius(arrhenius=[Arrhenius(...)]), which is exactly whatinterpolationModel = ('pdeparrhenius',)emits. A linter found it, not a test.So the funnel was run end to end on real data (trial-005): RMG → Cantera SA → a real PDep network
discovered and assessed → assessment and budget records written. Real Arkane output parsed cleanly;
the CSE→MSC fallback fired for real, CSE having failed macroscopic equilibrium on that network; and
a refusal was recorded with its rationale. What that run did not reach: nothing qualified, so
capture/join/hybrid never ran live — the funnel correctly had nothing to send, but roughly a third
of the new surface is still fixture-only.
Roughly 1000 lines of the diff are tests. Beyond assertions, the new guards were mutation-tested:
each guard was individually neutered and a test confirmed to fail, then reverted. Where a mutation
was reported without evidence, it was re-run independently before being believed.
Review-driven hardening
This branch went through 65 adversarial review rounds. That process found — and this PR fixes —
thirteen instances of the same fail-open / vacuous-truth defect, where a guard silently passed
on the case it existed to catch. Representative examples:
verify_capture()never read thestatusfield at all, so a manifest entry claimingstatus: usablewith a null artifact path verified clean, counted zero artifacts, pruned theoutput tree, returned success, and let the finalization marker be written.
verify_capture()and then separately re-read the same manifest, so pathconfinement applied to one parse while the use applied to another.
use_atom_corrections='False'(a non-empty, therefore truthy, string) slipped past a falsy checkwhile the generated file rendered a bare
useAtomCorrections = False.os.replace'd whileinput.pywas a plainopen(...,'w')—the marker was more durable than the output it certifies, so a torn file could be certified
as complete.
Known limitation, recorded rather than hidden
The versioned finalization marker relies on every step of
process_arc_run()being idempotent. TheRMG library append's contribution to that is dedup-by-label: verified to add no duplicates, but
it would silently keep stale values if a source library ever carried the same labels with different
values. No path in this feature produces that, it is pre-existing behavior, and it is documented at
the claim site rather than left implicit.
Open questions for you
state is "no
input.pyplus a finalization marker" — indistinguishable later from not-selected,zero-artifact, manually deleted, or pruned. The reason exists only in logs. Review flagged this
twice. I did not design it unilaterally because it interacts with the prune preflight and I
think the shape should be your call.
t3/utils/libraries.pyhas two pre-existing defects I deliberately did not fix here, sincethey are outside I-008 and deserve their own PR:
description_to_appendis extracted from thedestination's own
longDescand checked against it, so it never consults the source and iseffectively a no-op; and the shared-library lock path is keyed by
library_name, so twoprojects writing the same
shared_library_nameunder different locallibrary_names takedifferent locks — a real concurrency hole in shared-library writes.
— no longer true; this PR nowexplore=Truestill raisesNotImplementedErrorbuilds that piece. The standalone PES exploration entry point
explore_pdep_network()and aconcrete
ArkaneExplorerAdapterare included (t3/pdep/api.py,t3/pdep/explorer/), alongwith the QM budget (
t3/pdep/budget.py,pdep_QM_max_transition_states/pdep_QM_max_networks).select_pdep_network()itself never explores: the deprecatedexplore=True/how=...arguments now raise aValueErrorredirecting the caller toexplore_pdep_network().Related
Depends on RMG-Py PR ReactionMechanismGenerator/RMG-Py#2990 (open, awaiting review) for the
P-dep sensitivity / ILT support this consumes.