Repository navigation
Codex usage collector rereads GB of history and saturates CPU #12411
Description
System details
Intel Core Ultra 7 165U (14 logical CPUs), Intel Meteor Lake-P graphics, Omarchy 4.0.4-1, Linux 7.2.5-3-omarchy.
What's wrong?
omarchy-agent-usage-codex can saturate one CPU core for several minutes and make the desktop lag when Codex/Pi session history is large. The stock agents widget starts a normal collection immediately (triggeredOnStart: true) and repeats it on every refresh. A normal collection only reuses the local scan cache for 20 seconds, so each 15-minute refresh can walk the session history again.
Observed session data sizes:
~/.pi/agent/sessions: 409 MB~/.omp/agent/sessions: 1.1 GB~/.codex/sessions: 4.7 GB
Observed during one stock collector run:
omarchy-agent-usage-codex: 98-99% CPU for more than 3 minutes- process I/O counters: about 5.9 GB
rcharand 546k read syscalls - system load: 13-17; desktop visibly laggy
- memory pressure: zero; CPU temperature: 56 C
After terminating the collector, the machine returned to 98% idle. The configured widget is a user clone of omarchy.agents, but its Main.qml is byte-for-byte identical to the packaged stock plugin, and the heavy process is the packaged /usr/share/omarchy/bin/omarchy-agent-usage-codex collector.
Steps to reproduce:
- Accumulate several GB of Codex/Pi/OMP JSONL session history.
- Enable the stock
omarchy.agentswidget, or runomarchy-agent-usage-codexdirectly after its 20-second cache window expires. - Observe the collector consuming one core while recursively scanning/parsing session files for minutes.
- Wait for the widget's normal refresh interval; the full scan happens again.
Relevant implementation details in 4.0.4-1:
SCAN_REUSE_SECONDS = 20scan_pi_sessions()runsrgacross all Pi and OMP session roots without a date cutoff.scan_native_codex_sessions()opens and parses every JSONL file modified in the last 30 days.- The normal widget timer calls
runUpdate("normal"), including at startup.
Expected: periodic usage refresh should not rescan gigabytes of unchanged history or degrade desktop responsiveness. Possible approaches: incremental per-file caching keyed by mtime/size, a longer-lived aggregate cache with only changed files rescanned, or running scheduled refreshes at low CPU/I/O priority while keeping an explicit force refresh.
Activity
omarchybot commentedon Oct 4, 2026
#14049 has merged into
quattroand caches totals for unchanged native Codex rollout files, covering that part of the repeated full-history scan. Pi/OMP history still goes through an rg scan on each collection, and changed native files are still read in full. This broader performance report therefore stays open for the remaining cost; #12336 separately proposes append-only native scanning.Closure scope checked by GPT-6 in Codex and Codex Medium against the reports and landed source. The second opinion agreed; independence is not guaranteed. No runtime tests were rerun for this reconciliation.
omarchybot commentedon Oct 12, 2026
Measured what is left of this on current
quattro(defbf90), on a credential-free Omarchy worker with synthetic history in a temporary home: 491 MB of pi sessions and 383 MB of native Codex rollouts.scan_pi_sessions()(bin/omarchy-agent-usage-codex:255). It runsrgover every pi and omp session root with no date cutoff and no per-file reuse, then decodes each match twice in Python. Its cost grows with the size of the pi/omp history, so the 1.5 GB in this report still gets rescanned on every 15-minute refresh.scan_pi_sessions(), and on the same data the unchanged refresh took the same time (2.46 s, of which 2.41 s was the pi scan). It covers part of this issue, not all of it, so I have not linked it as fixing this one.:395) also runs over all history on each refresh. I did not measure it here.I have not opened a fix. Pi sessions are left out of the per-file cache on purpose (
:589): a forked session repeats its parent's messages, and messages are deduplicated, and sessions assigned to an owner, across the whole tree at once (:294,:337). Per-file totals therefore cannot simply be added up. Making that scan incremental means deciding what to cache per file and rebuilding the deduplication from it. That is a design decision for the maintainer.These timings come from synthetic data with no forks and half of all pi lines matching. They show which part of the collector scales with history, not how many minutes a real machine spends. Codex Medium reviewed the source and the measurements as a second opinion and reached the same conclusions; independence is not guaranteed. This waits on the maintainer: whether #12336 should land, and how the pi/omp scan should become incremental.
LIghtJUNction commentedon Oct 23, 2026
Additional installed-release evidence from Omarchy 4.0.6-1 (2026-10-23):
omarchy.agentswidget is enabled. Its default refresh interval is 900 seconds./usr/bin/python3 /usr/share/omarchy/bin/omarchy-agent-usage-codexprocess was still running about nine minutes after launch, withpsreporting roughly 90–92% CPU. These are process lifetime observations, not a controlled benchmark./proc/PID/ioread showedrchar=21362640903,read_bytes=16835837952, andsyscr=2361044: about 21.36 GB passed through reads and 16.84 GB charged as storage reads. These counters do not establish how much unique session data exists.The installed collector still uses
SCAN_REUSE_SECONDS = 20. Its native-session scan recursively selects JSONL files modified in the last 30 days, then opens each selected file and callsjson.loadson every line. The installed collector SHA256 is821f627f1be5ac691f70885060d0b2b65a14972162f82962e7a5575badaf39dd.I read the notes above about #14049 on current
quattro; this observation concerns the installed 4.0.6-1 artifact and is not a claim that the newer caching patch regressed. It supports getting the applicable caching improvements into the installed release and bounding background collection cost on large histories.Both external-mouse and touchpad stutter were reported during this session, but the laptop was later found to be on battery at 8%. No controlled A/B test isolated the collector from battery/power or graphics behavior. Please treat the CPU/read-volume evidence as confirmed and the pointer-lag relationship as unproven. No private transcripts or raw diagnostic logs are included.
omarchybot commentedon Oct 23, 2026
The 4.0.6-1 observation above is from a collector without #14049's cache. The SHA256 quoted matches
bin/omarchy-agent-usage-codexat thev4.0.6tag exactly, and #14049's merge commit (75250d3) is not an ancestor ofv4.0.6. It first ships inv4.5.0-rc1. So on 4.0.6 every refresh still reopens and parses every native Codex rollout from the last 30 days, which matches the read volume reported.What is left on
quattroafter #14049 is the pi/omprgscan and the OpenCode query, both of which read all history on every refresh. Open #15906 makes both incremental: unchanged pi/omp sessions are not reopened, grown ones resume from a stored offset, and OpenCode rows are reparsed only whentime_updatedmoves. Fork deduplication still runs over the whole tree, from per-message entries kept in the index rather than per-file totals. It is already linked to close this issue. Once both are in, the remaining full read is a native rollout that changed since the last refresh. #12336 proposes resuming those from an offset. My earlier measurement put a changed 100 MB rollout at 0.19 s.Checked: the release tags against the reported hash and #14049, open pull requests touching the collector, and #15906's diff against the scans this issue describes. Nothing was run for this update. Codex Medium reviewed the same as a second opinion and agreed. It added the point about changed native rollouts. Independence is not guaranteed. This waits on the maintainer: merging #15906, and whether #12336 should land.