Skip to content

Feat(newstate): prune new state in runtime - #4192

Merged
NazariiDenha merged 2 commits into
mainfrom
feat/newstate-runtime-pruning
Oct 27, 2026
Merged

NazariiDenha merged 2 commits into
mainfrom
feat/newstate-runtime-pruning

Conversation

@NazariiDenha

@NazariiDenha NazariiDenha commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

PR Type

Enhancement, Tests


Description

  • Add new-state layout detection and history pruning to PruneUpto

  • Delete superseded per-key history entries via supersededPruner

  • Parametrize pruner tests over deprecated and new layouts

  • Assert history read answers unchanged after pruning


File Walkthrough

Relevant files
Enhancement
accessors.go
Add new-state history pruning to pruner                                   

pruner/accessors.go

  • Add hasNewStateHistory to detect whether state history uses the
    new-state layout
  • Add pruneNewStateHistoryFromUpdate deleting each key's superseded
    entry below the block for storage, nonce, and class-hash history
    buckets
  • Introduce supersededPruner that reuses one iterator per bucket to
    locate and delete the latest entry below a block
  • Thread the layout flag through PruneUpto and pruneHashKeyedUpto to
    pick the appropriate pruning path
+139/-2 
Tests
accessors_test.go
Run pruner tests over both layouts                                             

pruner/accessors_test.go

  • Add layouts var and storeBlocks helper storing blocks in either layout
    via testutils.StoreBlockWithLayout
  • Parametrize TestPruneUpto, no-op, resume-after-cancel, and rolling
    carve-out tests over both layouts
  • Assert HistoryAnswers are identical before and after pruning to verify
    pruning preserves history semantics
+107/-80

@NazariiDenha
NazariiDenha added this pull request to stack #4193 October 14, 2026 11:10
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @NazariiDenha's task in 36m 3s —— View job


Review of PR #4192 — prune new state in runtime

  • Gather context (diff, changed files, CLAUDE.md)
  • Review node/node.go
  • Review pruner/accessors.go
  • Review pruner/pruner.go
  • Review test files
  • Post inline comments for issues found
  • Post final summary

Blocker

  • node/node.go:399,546 — The new pruner.WithNewState(cfg.NewState) wiring is unreachable in a running node. node/node.go:228-231 (unchanged by this PR) still rejects cfg.Prune && cfg.NewState outright, and both call sites for WithNewState are inside if cfg.Prune { ... } blocks — so cfg.NewState is guaranteed false whenever this code executes. The pruneNewStateHistoryFromUpdate path in pruner/accessors.go is only exercised by tests calling pruner.PruneUpto(..., true) directly, never from the actual node runtime. See the inline comment for details — if this is intentional groundwork ahead of a follow-up PR that relaxes the guard, it'd help to note that in the PR description.

Important

None beyond the blocker above — the core pruning logic in pruner/accessors.go (pruneNewStateHistoryFromUpdate, supersededPruner) looks correct: it matches the new-state history semantics (entries record the post-block value, reads take the latest entry at-or-before), correctly uses Seek+Prev to find and delete only the superseded entry, and is consistent with the write path in core/state/state.go (e.g. deploys intentionally don't get nonce-history pruning since writeHistory never writes nonce history for DeployedContracts either).

Nit

  • The new TestPruneUpto_NewStateHistory test fixture (pruner/testutils/helpers.go) never populates StateDiff.DeployedContracts, so the DeployedContracts branch in pruneNewStateHistoryFromUpdate's class-hash loop is untested. Not a blocker since the logic is simple and symmetric with ReplacedClasses, but worth adding coverage if convenient.

@codecov

codecov Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.32787% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.82%. Comparing base (f17fbb9) to head (5055241).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
pruner/accessors.go 80.32% 12 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4192      +/-   ##
==========================================
- Coverage   80.89%   80.82%   -0.08%     
==========================================
  Files         509      509              
  Lines       39102    39161      +59     
==========================================
+ Hits        31632    31650      +18     
- Misses       7464     7505      +41     
  Partials        6        6              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread node/node.go Outdated

@EgeCaner EgeCaner left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The range-delete suggestion isn't a blocker, but I'd like to see the test coverage addressed before merge.

Comment thread pruner/accessors.go
Comment thread pruner/accessors_test.go Outdated
@NazariiDenha
NazariiDenha force-pushed the feat/newstate-runtime-pruning branch from f47f546 to 82130d2 Compare October 20, 2026 12:57
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Review in progress

  • Gather context (diff vs origin/feat/newstate-pruning, changed files)
  • Review pruner/accessors.go changes
  • Review pruner/accessors_test.go changes
  • Post inline comments for issues found
  • Post final summary

@NazariiDenha
NazariiDenha force-pushed the feat/newstate-runtime-pruning branch from 82130d2 to 300c01b Compare October 23, 2026 10:48
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @NazariiDenha's task in 19m 41s —— View job


Review of latest changes (vs origin/feat/newstate-pruning)

  • Gather context (diff vs origin/feat/newstate-pruning, changed files)
  • Review pruner/accessors.go changes
  • Review pruner/accessors_test.go changes
  • Verify whether mixed on-disk layout is reachable at runtime
  • Post inline comments for issues found
  • Post final summary

This update addresses both points from the earlier review round:

  1. Blocker resolved — the unreachable pruner.WithNewState(cfg.NewState) wiring and the node.go/pruner.go Option it depended on have been removed entirely (300c01b9). PruneUpto now auto-detects the on-disk layout via the new hasNewStateHistory (pruner/accessors.go:321-340) instead of relying on a config flag, so the new-state pruning path is reachable in production as soon as history is actually stored in the new-state buckets — no dead code.
  2. Test coverage addressed — the four existing TestPruneUpto* cases are now parameterized over both layouts via the shared layouts/storeBlocks helpers (pruner/accessors_test.go:18-33), replacing the separate TestPruneUpto_NewStateHistory happy-path test, matching Ege's request to cover both layouts through the existing suite rather than bolting on new-state-only tests.

Correctness check: layout detection vs. mixed-layout safety

hasNewStateHistory only checks for presence of new-state entries and, unlike migration/historyprunner/layout.go's detectLayout, doesn't also check for leftover deprecated-layout entries (no "mixed" error). I verified this is safe in practice: migrateIfNeeded (node/node.go:845) runs the historyprunner migration to full completion — and detectLayout there does refuse to proceed on a mixed DB — strictly before any service, including the background pruner, is started (node/node.go:890-892). So pruner.PruneUpto never runs against a mixed-layout database in the real node; this is a non-issue, just worth knowing it relies on that startup ordering rather than an explicit check in accessors.go itself.

Nit (unresolved from last round, no longer a blocker)

The DeployedContracts branch in pruneNewStateHistoryFromUpdate's class-hash loop (pruner/accessors.go:396-402) is still untested — the shared test fixture (pruner/testutils/helpers.go:256) always writes an empty DeployedContracts map. Low risk since it's symmetric with the exercised ReplacedClasses path through the same supersededPruner, but still a gap if someone wants full coverage.

No blockers. Logic in pruner/accessors.go (pruneNewStateHistoryFromUpdate, supersededPruner) is correct and consistent with the new-state write path in core/state/state.go.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @NazariiDenha's task in 9m 41s —— View job


Review of latest changes (vs origin/feat/newstate-pruning)

  • Gather context (diff vs origin/feat/newstate-pruning, changed files)
  • Review pruner/accessors.go changes
  • Review pruner/accessors_test.go changes
  • Re-verify correctness of supersededPruner/pruneNewStateHistoryFromUpdate
  • Post final summary

No code changes since the last review round. The two files in this PR's diff (pruner/accessors.go at blob 27320b19, pruner/accessors_test.go at blob fbe7b5f4) are byte-identical to what was reviewed on 2026-10-23 and subsequently approved by @EgeCaner — this trigger landed on the same commit (6916ab97), with no new pushes to re-review.

Re-checked the logic once more for this pass:

  • supersededPruner.prune (pruner/accessors.go:427-438): Seek(prefix+blockNumber) followed by Prev() correctly lands on the entry immediately preceding the current block's own entry for a given key prefix, since BigEndian-encoded suffixes preserve numeric ordering. The one-iterator-per-bucket reuse across keys in arbitrary map-iteration order is safe because Seek is an absolute repositioning, not a relative scan.
  • No aliasing bug in the binary.BigEndian.AppendUint64(keyPrefix, ...) call — keyPrefix comes from db.Bucket.Key(...), which allocates an exact-length buffer (buf := make([]byte, size)), so cap == len and append always reallocates rather than mutating a shared backing array.
  • Both reviewer threads from prior rounds are resolved: the unreachable WithNewState wiring was removed in favor of runtime layout auto-detection (hasNewStateHistory), and test coverage now runs the full TestPruneUpto* suite over both layouts via the shared layouts/storeBlocks helpers, with HistoryAnswers equality assertions pre/post-prune.

Nit (carried over, non-blocking)

pruneNewStateHistoryFromUpdate's DeployedContracts branch (pruner/accessors.go:396-402) is still not exercised directly by a test fixture that populates StateDiff.DeployedContracts — low risk since it shares the same supersededPruner path as the tested ReplacedClasses case.

No blockers. Nothing further to act on for this commit.
· branch: feat/newstate-runtime-pruning → feat/newstate-pruning

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Title

(Describe updated until commit 5055241)

Feat(newstate): prune new state in runtime


PR Type

Enhancement, Tests


Description

  • Detect state-history layout before running pruning

  • Add pruneNewStateHistoryFromUpdate deleting superseded entries

  • Introduce supersededPruner with seek-based deletion

  • Extend pruning tests to cover both layouts


File Walkthrough

Relevant files
Enhancement
accessors.go
Add new-state history pruning with layout detection           

pruner/accessors.go

  • Add hasNewStateHistory to detect the state-history layout
  • Pass layout flag through PruneUpto into pruneHashKeyedUpto
  • Add pruneNewStateHistoryFromUpdate deleting the key's previous entry
  • Add supersededPruner reusing one iterator per history bucket
+139/-2 
Tests
accessors_test.go
Test pruning across deprecated and new layouts                     

pruner/accessors_test.go

  • Parameterize storeBlocks helper with a history layout
  • Run main pruning tests as subtests over both layouts
  • Assert HistoryAnswers unchanged before and after pruning
+107/-80

Comment thread pruner/accessors.go
Comment thread migration/historyprunner/migrator.go
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

(Review updated until commit 5055241)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
⚠️ Risk level: Medium
📂 Priority files

  • pruner/accessors.go
🏅 Score: 82
🧪 PR contains tests
🔒 No security concerns identified

Base automatically changed from feat/newstate-pruning to main October 27, 2026 10:20
@NazariiDenha
NazariiDenha force-pushed the feat/newstate-runtime-pruning branch from 6916ab9 to 44886bb Compare October 27, 2026 10:21
@NazariiDenha
NazariiDenha force-pushed the feat/newstate-runtime-pruning branch from 44886bb to 5055241 Compare October 27, 2026 10:22
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @NazariiDenha's task in 3m 12s —— View job


Review of latest changes (vs origin/main)

  • Gather context (diff vs origin/main, changed files)
  • Review pruner/accessors.go changes
  • Review pruner/accessors_test.go changes
  • Post inline comments for issues found
  • Post final summary

No new code to review. The two changed files (pruner/accessors.go, pruner/accessors_test.go) are byte-identical to commit 300c01b9, which was already reviewed in depth on 2026-10-23/2026-10-27 and approved by @EgeCaner. This trigger landed on 50552411 / 505524116, a rebase of the same content — no new diff to analyze.

Summary of prior findings (still accurate, nothing new)

  • Blocker (resolved): the earlier unreachable pruner.WithNewState(cfg.NewState) wiring was removed; PruneUpto now auto-detects the layout via hasNewStateHistory (pruner/accessors.go:321-340), so the new-state pruning path is live in production.
  • Important: none outstanding. pruneNewStateHistoryFromUpdate/supersededPruner logic (pruner/accessors.go:341-438) is correct — Seek+Prev finds the superseded entry via BigEndian-ordered suffixes, key-prefix buffers aren't aliased (db.Bucket.Key allocates exact-length, so append always reallocates), and it's consistent with the new-state write path in core/state/state.go.
  • Nit (non-blocking, carried over): pruneNewStateHistoryFromUpdate's DeployedContracts branch (pruner/accessors.go:396-402) is still not exercised by a fixture that populates StateDiff.DeployedContracts — pruner/testutils/helpers.go:256 always writes an empty map. Low risk since it shares the same supersededPruner path as the tested ReplacedClasses case.
  • Noted, not a bug: hasNewStateHistory only checks for new-state entries and doesn't explicitly reject a mixed-layout DB (unlike historyprunner.detectLayout). This is safe today because migrateIfNeeded (node/node.go:845) runs the history-pruning migration — which does reject mixed layouts — to completion before any service, including the background pruner, starts (node/node.go:890-892). Worth keeping in mind if that startup ordering ever changes.

No blockers. Nothing further to act on for this commit.
· branch: feat/newstate-runtime-pruning → main

@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 5055241

@NazariiDenha
NazariiDenha merged commit 77f38a6 into main Oct 27, 2026
18 checks passed
@NazariiDenha
NazariiDenha deleted the feat/newstate-runtime-pruning branch October 27, 2026 10:38

This branch was successfully deployed

1 active deployment
Development — 50552411 Deployed Oct 27, 2026 by NazariiDenha via dev_deploy_test / deploy #6199
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants