fix FlatKV cache memory leak

[cody-littley]

Describe your changes and provide context

fix a (slow) memory leak in the FlatKV cache layer

[cursor]

PR Summary

Medium Risk
Touches concurrent read-cache completion and MVCC retire paths; incorrect channel handling could strand readers or corrupt cache invariants, though behavior is covered by new tests.

Overview
Fixes a slow memory leak in the FlatKV read-through cache where retired entries could keep a buffered valueChan (and its stale read result) alive after setRetiredLocked / deleteRetiredLocked updated the entry.

Retirement now sets entry.valueChan = nil when a key is retired or tombstoned, so the cache stops holding old single-slot channels indefinitely. In-flight reads are unchanged: resolve and batch scheduling capture the channel at schedule time, and injectValue sends on that bound ch instead of e.valueChan, so a retire cannot break waiters or cause unsafe dereferences.

Adds integration and fuzz tests that the bound-channel path still delivers results and that no terminal entry keeps a channel buffer that disagrees with entry.value, including races between retire and late injectValue.

^{Reviewed by Cursor Bugbot for commit 1842b15. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	✅ passed	Sep 3, 2026, 8:23 AM

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.16%. Comparing base (b68026f) to head (1842b15).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4084      +/-   ##
==========================================
- Coverage   61.23%   60.16%   -1.07%     
==========================================
  Files        2177     2069     -108     
  Lines      190632   178137   -12495     
==========================================
- Hits       116729   107181    -9548     
+ Misses      62892    60987    -1905     
+ Partials    11011     9969    -1042     

Flag	Coverage Δ
sei-chain-pr	88.91% <100.00%> (?)
sei-db	69.80% <ø> (ø)
sei-db-state-db	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-db/db_engine/view/read_cache.go	97.83% <100.00%> (+0.03%)	⬆️

... and 108 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[seidroid]

The channel-binding refactor is sound: entry.valueChan is only read in lookupLocked's statusScheduled branch, and both retire paths move the entry to a terminal status under the same lock, so nil'ing the field cannot strand a waiter or hand a reader a nil channel — while it does release the stale buffered value that caused the leak. One non-blocking note on the new fuzz test's effective coverage.

Findings: 0 blocking | 1 non-blocking | 1 posted inline

Blockers

• None at the file/PR level.

Non-blocking

• None at the file/PR level.
• 1 suggestion(s)/nit(s) flagged inline on specific lines.

[seidroid]

[suggestion] With maxSize = 1<<30 and 1-byte keys / ≤8-byte values, gcQueue.GetTotalSize() never exceeds the budget, so case 4: evict is a permanent no-op. Since eviction is the only transition back to statusUnknown, each of the four keys reaches a terminal status once and stays there: after the first few iterations case 0 returns immediate, case 1 finds nothing pending, and case 4 does nothing. racedCompletions is therefore capped at 4 (one per key), and the bulk of the 1000 iterations exercise nothing.

The test still catches the bug it was written for (the non-raced retire-after-resolve path fails the invariant pre-fix), so this isn't blocking — but a small maxSize (e.g. a few tens of bytes, matching newTestConfig's style) would make evictions real, recycle entries back through statusUnknown, and let the loop actually explore the read/retire/evict interleavings the doc comment claims. Note entryLocked recreates evicted entries with a fresh channel and pending[key] is overwritten on the new needsSchedule outcome, so no channel receives two injectValue sends and the loop stays hang-free under eviction.

While there: randomValue never returns nil, so injectValue's statusDeleted branch is never hit — occasionally injecting a nil value would cover it.

[blindchaser]

it looks the detach logic is something each caller (multiple callers there not limited to ones we addressed) has to remember rather than an invariant. could we fold all fields into one function under cacheEntry struct ?

something like:

func (e *cacheEntry) settleLocked(status valueStatus, value []byte) {
	e.status = status
	e.value = value
	e.valueChan = nil
}