feat: implement durable output knowledge and overlay architecture - #674
Draft
ty-everett wants to merge 204 commits into
Draft
ty-everett wants to merge 204 commits into
ty-everett wants to merge 204 commits into
Conversation
…ify durable replay
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Problem and resulting behavior
Applications need durable UTXO-derived state across independent overlay hosts, progressive ingestion and resumable live updates, while keeping Bitcoin evidence, source membership and application decisions separate. This PR implements the application-neutral BRC-192–199 specifications with additive SDK, output-knowledge, wallet, Overlay, Overlay Express and LCH components. Existing finite lookup, ordinary STEAK, wallet actions and standalone BRC-170 defaults remain supported. Current-channel query semantics remain a separate contract.
Checkpoint two is pending and this PR remains draft. Exact published head:
63ed4ca72b205ff20be4fc2f5ad5dbd52fe4a96c. Downstream application migrations follow its human review. No merge, package publication, deployment or funded operation is included.Implementation
Runtime follow-ups preserve ownership, byte ceilings, error identities, fresh custody, sealing, authority, trusted-clock, Script and transaction boundaries. Bounded serialized-envelope reads preserve subclass/instance/prototype dispatch and original receiver. The latest native header shortcut captures fields once and constructs an owned address only for bounded supported ASCII metadata; every other representation retains the original validation path. Numeric byte-index lookup preserves every descriptor and range guard. No private value, key or authorization decision is cached. Two unsuccessful SDK optimization experiments were rejected without adoption.
The fixture follow-up expresses startup as a joined promise chain and starts its three real Mongo members sequentially, preserving all storage/election/write settings, original launch/hook deadlines and full test selections. Each original start settles before the next probe or failure cleanup; existing members retain original restart ports. Three deterministic startup/failure/restart regressions pass. The small class stays in the existing fixture file for direct Node TypeScript and bundled consumers. No retry or swallowed failure is introduced. The preceding inline-loop form raised two Sonar S9382 findings; the promise-chain correction preserves the same ordering and failure identity without a dismissal, accepted classification or suppression. Its original three regressions and complete20-native/all-three-Chrome selection pass again.
Validation and remaining qualification
The acceptance inventory maps each complete reference workflow to reproducible selectors and guides. The implementation record retains measured comparisons, original controls, failed attempts and assurance limits. Synthetic public fixtures and loopback services broadcast nothing; they do not establish production mining or a production audit.
Complete local application coverage passes 3,457 tests/229 suites, zero failures or skips, in962.684 seconds (98.34/96.83/98.63/98.86%). All original300-case properties, seed3242026, blank replay and150/2100-second bounds remain. The native selection passes292/17, including four new ownership, malformed-representation, ordered-field-access and live-kind/Unicode regressions. Affected builds pass. Complete installed controls pass494 tests/52 test files, zero failures, cancellations or skips. Every captured authored/configuration byte and all 4,424 compiled paths across seven roots remain fixed through independently verified physical process drains.
The final inline fixture passes all 20 native reference tests/five files and all three actual Chrome profiles in84.168 seconds. The private-state profile has exactly two Opens, zero exceptions, native IndexedDB, owned HTTPS, producer restart and proposal expiry. The preceding complete core replay passes1,079 tests with its one unchanged pre-existing skip; final-head hosted coverage independently reruns that union. Failed optional-type and direct-source-path attempts are retained with exact inputs and physical drains.
The complete-file mutation subdivision retains every original test, source tuple, runner option, additional input, property case, deadline, worker setting, canonical score and zero-invalid/uncovered/unexecuted gate. The actual pinned protected-ledger inventory843 divides into complete store485, scalar-codec167 and payload-codec191 files. Registry141 canonical targets now schedules376 execution rows [256,120]. Native-clock1481, SDK authentication/HTTP890 and wallet-recovery-encoding167 inventories remain unchanged. Installed-engine checks prove exact duplicate-free tuple coverage.
Published predecessor c413 passes SDK, hosts, native/Chrome/mobile/packed/documentation and strict security component checks. Its CI37195017159 nevertheless fails the original buyer150-second property and application35-minute ceiling; those passes do not qualify this source. The prior blank-selector campaign37190660296 failed an uncovered unused private initializer, subsequently fixed, and was cancelled as obsolete. The latest grouped codec local campaign also retained its2700-second timeout after all 252 original tests passed. Subsequent cc492 CI exposed a concrete reference-fixture port collision and cleanup race; fullcampaign37204952974 was cancelled as obsolete for that correction, retaining its immutable partial results without score borrowing. No failed or partial run supplies a passing score.
The final campaign must bind caller SHA, run, attempt, immutable archive identities and SHA-256 before extraction, reject stale/incomplete/diagnostic evidence, and perform the independent complete canonical recheck. No source exclusion, suppression, advisory dismissal, accepted finding, test skip, deadline increase or score borrowing is introduced.
BotBoard772 records authorized coordination. Checkpoint two will be presented for human review only after every required item is complete; checkpoint three remains the downstream application integration phase.
Current-head hosted CI/security/conformance/container qualification has been triggered after the fixture-only push. The new complete blank-selector campaign will be dispatched after applicable CI passes, avoiding another expensive campaign on a source that still fails its primary tests. Checkpoint two remains open until both independent qualifications are complete.
The focused local scalar-codec replay on5e instruments167 mutations and passes all252 original initial tests, then is deliberately interrupted at340.630 seconds for the concrete consumed-fixture Sonar follow-up. Actual group85954 is independently absent and all captured inputs/4,424 compiled paths are unchanged; this incomplete replay supplies no score. Final published source remains subject to one fresh complete blank-selector same-run campaign.