Skip to content

fix(core): keep cachefs: across app launches for the session - #1301

Merged
lvcabral merged 6 commits into
masterfrom
fix/cachefs-persist-1252
Oct 31, 2026
Merged

lvcabral merged 6 commits into
masterfrom
fix/cachefs-persist-1252

Conversation

@lvcabral

@lvcabral lvcabral commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Closes #1252

Problem

cachefs: was emptied on every launch. Its buffer was allocated inside the app worker (BrsDevice.getCacheFS()), and both hosts throw that worker away on every run. On a device, cachefs keeps its contents until a reboot or until the space is needed.

Change

  • The browser API (src/api/index.ts) and Node executeApp (src/node/host.ts) hosts now own the cachefs SharedArrayBuffer and pass it to each app worker as DeviceInfo.cacheFSBuffer. Task threads already mount the app's buffer, so they needed no change.
  • One volume for all apps: the host keeps a single cachefs: volume that every app it runs shares, whatever its developer ID. It is replaced with an empty one only when cacheFSVolSize changes. Reloading the page or exiting the process acts as the reboot. This is a deliberate difference from a device, which keeps cachefs per developer ID (file-system.md). Switching developer ID on a device is rare, and hosts that give each app its own developer ID (e.g. brs-desktop's "Registry by App") would otherwise hide one app's cache from another.
  • tmp: is still emptied on every launch.
  • Browser integrators can't inject cacheFSBuffer (it's added to invalidKeys). Both hosts share one getCacheVolume helper in core/common.ts.
  • Compaction on launch: on app launch, before any Task thread exists, BrsDevice calls FileSystem.compactCacheFS before mounting. It always clears a metadata write lock left by a worker that was hard-terminated mid-write. Without that, the next launch would block forever in Atomics.wait. zenfs SingleBuffer allocates by bumping and never reclaims space when a file is rewritten or deleted. So once that wasted data reaches a quarter of the volume, the volume is rebuilt from its live files, and only the used part of the buffer is zeroed. A fresh volume is not compacted at all. If the buffer can't be read, the volume starts empty, like a device eviction. Task threads only mount the volume, so they never reset it.
  • Eviction when full: the same launch-time pass also rebuilds the volume when it is over 75% full. It evicts the least recently modified files (only then; reclaiming wasted space alone keeps every file) until it is back to half, which is how a device frees cache space. Kept files keep their modification times, so later launches evict in the right order. Eviction only happens at launch. While an app runs, a write that doesn't fit fails with ENOSPC, and the app sees false. Unreadable entries are dropped one by one instead of wiping the volume.
  • Memory volumes (tmp:, cachefs:) use a MemoryVolume backend:
    • Space guard in the store: its SingleBuffer store rejects an allocation that doesn't fit (ENOSPC) before reserving any space. zenfs reserves space before copying the data, so one oversized write used to leave the volume unable to take any further write. Because the guard sits in the store, it covers every path: file writes, mkdir and the compaction restore. A rejected write leaves no file behind.
    • File name case: the volume keeps names as written and matches them regardless of case, as on a device. zenFS case folding now applies only to common:/pkg:. This replaces the per-thread original-case map, so names keep their case across threads and launches.
  • Renames: a rename within the same volume is native, so a case-only rename (a.txt → A.txt) keeps the file instead of deleting it.
  • brs-cli runs one app per process, so it still starts with an empty cachefs: on every run.

Device behavior

  • Confirmed on a device: re-sideloading a dev channel does not clear cachefs. This matches the engine, where re-running an app keeps the volume for the session.

Tests

  • test/node/host.test.js: two executeApp runs in one process. The second run, under a different developer ID, sees the cachefs: file, and tmp: is empty again.
  • test/interpreter/FileSystem.reliability.test.js: files and directories survive a relaunch; a corrupted buffer mounts empty while common: still works; a stale write lock no longer hangs the next launch; rewriting a 400 KB file over 5 launches fits in a 1 MB volume; with five 180 KB files in a 1 MB volume, the next launch keeps only the newest two, and a launch after that evicts nothing more; a write too large for the volume leaves no file behind and later writes still work; reclaiming wasted space keeps files that fit; file name case survives a relaunch, lookups ignore case, and a case-only rename keeps the file. I checked that the last two fail without compaction: the launch hangs, and the write fails with "offset is out of bounds".
  • npm run lint is clean. Full npm test passes: 238 files, 3185 tests.

Known limitations

  • A file being written when the app is terminated can be left half-written.
  • A full volume rejects writes that don't fit for the rest of that run. Space freed by deleting or rewriting files only comes back at the next launch. Evicting during a write, as a device does, would mean rebuilding the buffer while Task threads have it mounted. This is documented in docs/limitations.md.

Browser repro (issue example)

Ran markwpearce/brighterscript-game-engine examples/loading (Remote assets scene) in the example web app built from this branch:

  1. First launch: every asset reports network (loaded in 3.1 s).
  2. Home, then relaunch: every asset reports cache (0.17 s). There were no requests, which matches a device.
  3. Page reload (the simulated reboot): every asset reports network again.

🤖 Generated with Claude Code

lvcabral and others added 6 commits October 30, 2026 20:36
The cachefs: buffer was allocated inside the app worker, which both hosts
discard on every run, so each launch started with an empty volume. The
browser API and Node executeApp hosts now own the buffer, keyed by
developer ID (per the Roku file-system spec), and pass it to each app
worker; Task threads already share the app's buffer. tmp: stays per run.

cachefs: is now mounted separately so a buffer left unreadable by a
mid-write terminate is reset to empty instead of failing the whole
file-system setup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-ups for the session-persistent cachefs: volume:
- On app launch (no other thread mounted), rebuild the volume from its live
  files: reclaims space SingleBuffer never frees on rewrite/delete, and
  clears a write lock left by a worker terminated mid-write (which made the
  next launch block forever in Atomics.wait). Task threads only mount, so
  they can no longer wipe the volume on a transient checksum mismatch.
- Node host assigns the per-developer buffer instead of `??=`, since the
  payload device object is the shared DefaultDeviceInfo.
- Share getCacheVolume from core/common, keep cacheFSBuffer out of
  BrsDevice.deviceInfo, warn on a host buffer size mismatch.
- Move the cachefs: tests below the file header; add stale-lock and
  space-reclaim regressions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- compactCacheFS is a separate step BrsDevice runs before setup on app
  launch (no positional flag on setup). It skips a fresh volume and only
  rebuilds once rewritten/deleted data wastes a quarter of it, always
  clearing a stale write lock; it zeroes just the used prefix and drops a
  per-file copy and per-entry stat.
- Hosts keep a single cachefs: volume (replaced on a new developer ID or
  size), like a device evicting another developer's cache.
- Drop the unreachable worker-side size check; share memoryFSMount
  between tmp: and cachefs:.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the per-developer-ID keying: the host keeps a single cachefs: buffer
for every app it runs, replaced only when cacheFSVolSize changes. Switching
developer ID is rare on a device, and per-app developer IDs (e.g. a host's
registry-per-app mode) made one app's cache invisible to another.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… is full

- compactCacheFS also rebuilds when cachefs: is over 75% full, evicting the
  least recently modified files until it is back to half, as a device frees
  cache space. Kept files keep their modification times.
- A failed write to tmp:/cachefs: removes the file it left behind: the
  memory volume records the size before the data, so it would be unreadable
  but still reported by Exists().
- Document that eviction only happens at launch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… file name case

- tmp: and cachefs: use a MemoryVolume backend: a SingleBuffer store that
  rejects an allocation that doesn't fit (ENOSPC) before reserving it, so
  an oversized write no longer leaves the volume unable to take any write
  (covers every path: writeFile, mkdir, compaction restore).
- The memory volumes keep file names as written and match them regardless
  of case (MemoryVolumeFS resolves paths to the stored names), replacing
  the per-thread original-case map: the case is now shared by all threads
  and survives app launches.
- Same-volume renames are native, so a case-only rename keeps the file.
- compactCacheFS only evicts when the volume is over 75% full, drops
  unreadable entries instead of wiping the volume, reads data only for the
  files kept, restores directory times, and no longer counts metadata
  overhead as waste.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@lvcabral
lvcabral merged commit 5e90856 into master Oct 31, 2026
3 checks passed
@lvcabral
lvcabral deleted the fix/cachefs-persist-1252 branch October 31, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cachefs: is cleared on every channel launch; a Roku keeps it between launches

1 participant