Skip to content

Fix Codex usage collection on 0.149 - #7649

Merged
dhh merged 2 commits into
omacom:quattrofrom
orienw:fix/codex-usage-approval-policy
Aug 25, 2026
Merged

dhh merged 2 commits into
omacom:quattrofrom
orienw:fix/codex-usage-approval-policy

Conversation

@orienw

@orienw orienw commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Changes

  • replace Codex's removed untrusted approval policy with on-request
  • retain the existing read-only app-server sandbox
  • assert the collector's exact app-server launch arguments in its regression test

on-request is supported by Codex 0.148 and restores authenticated limits on 0.149.

Fixes #7648

Tests

  • bash test/shell.d/agent-usage-codex-scanner-test.sh
  • ./test/cli
  • live bin/omarchy-agent-usage-codex --limits-only against Codex 0.149.0

Copilot AI balanced review requested due to automatic review settings August 21, 2026 01:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Codex usage collector for Codex 0.149 compatibility.

Changes:

  • Replaces retired untrusted policy with on-request.
  • Adds regression coverage for exact app-server arguments.

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.

File Description
bin/omarchy-agent-usage-codex Uses the supported approval policy.
test/shell.d/agent-usage-codex-scanner-test.sh Verifies the complete Codex invocation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@omarchybot

Copy link
Copy Markdown
Collaborator

Reviewed. No defects found — the diff is correct and the two things that could have made it worse than the bug both check out.

Does on-request risk hanging the widget? No. fetch_codex_rpc() only sends initialize, the initialized notification, account/read and account/rateLimits/read; it never starts a turn, so no approval request is reachable, and -a is inert on this call path. Verified by driving the real codex app-server over the same JSON-RPC handshake under both policies on codex 0.147.0: identical responses, 0.63s under on-request versus 0.42s under untrusted, no approval traffic either way. rpc_request() also has 8/4/4s deadlines and the finally block terminates the process, so even an unexpected message could not hang it indefinitely.

Does dropping the version guard break older codex? No, and this was the one worth checking, since the collector has no version detection. -a on-request is accepted by every version tested here: 0.100.0, 0.130.0, 0.146.1 and 0.147.0 all list it in --help and start app-server with it. Since 0.149 keeps on-request and drops untrusted, on-request is the single value that works across the whole range — no guard or fallback is needed.

Tests. Run on a disposable VM, not locally. ./test/shell.d/agent-usage-codex-scanner-test.sh — 21 passed, 0 failed. ./test/cli — 116 passed, 0 failed. The new assertion was mutation-tested: reverting only the production flag back to untrusted on the worker made it fail with not ok - Codex collector uses the supported approval policy and printed the actual args, so it is a real regression guard rather than a tautology.

Second opinion. codex at xhigh reviewed the diff independently and reached SHIP with no defects. Its independence is not currently guaranteed — it can read this machine's own files, including omabot's review protocol — so treat the agreement as agreement. It did contribute one thing checked afterwards and confirmed: codex 0.130's own help, in the deprecation note for on-failure, says to prefer on-request for interactive runs and never for non-interactive ones. never would arguably read as the more honest label for a bar-widget collector, but since no turn is ever created the two are behaviourally identical here, so this is not a reason to change the PR.

On #7648. This closes the defect the issue reports — the launch line is the only place untrusted appears, and limits come back. It does not implement the issue's second paragraph of expected behavior: "It would also be useful for the collector to surface an early app-server process exit instead of reporting the pending RPC method name as the error." stderr is still subprocess.DEVNULL and an immediate exec failure still surfaces as the pending method name. That is a pre-existing diagnostic weakness, not something this PR introduces, and fixing it here would expand the diff beyond the one-flag change — but Fixes #7648 will auto-close the issue including that request, so it is worth deciding whether to keep it open separately.

One adjacent pre-existing note, not a blocker and not touched by this PR: rpc_request() returns a JSON-RPC error response as an ordinary one, so a protocol or auth error yields empty limits with an empty usageStatusText rather than a message — the same class of silent failure that made this bug hard to read.

Nothing pushed to the branch. Waiting on the maintainer.

@RodriMora

Copy link
Copy Markdown

Confirmed this fixes the issue codex 0.149 introduced

nixfred added a commit to nixfred/omarchy that referenced this pull request Aug 22, 2026
Bring the Codex tab of the Agents panel up to parity with the Claude tab.
Codex's account/rateLimits/read already returns more than the collector
surfaced: a plan type, model-scoped allowances under rateLimitsByLimitId
(each with its own 5-hour and weekly window), and a spend-control flag.

- tierLabel is a human plan name ("prolite" -> "Pro Lite", "plus" -> "Plus")
  instead of the raw API token, matching Claude's "Max 20x" / "Pro".
- Window labels follow the Claude collector's wording ("Session (5-hour)",
  "Weekly (7-day)") so the LIMITS section reads the same on both tabs.
- Model-scoped limits (e.g. "GPT-5.3-Codex-Spark") are emitted with a
  title, the way Claude reports "Fable Weekly", so the panel shows them as
  their own rows instead of dropping them.
- spendControlReached surfaces as the usage status text.
- Launch app-server with `-a on-request`, as in omacom#7649, since Codex 0.149
  retired `untrusted` (omacom#7648).

The scanner test's mock app-server now answers with a realistic payload
and asserts the plan label and every limit row.
nixfred added a commit to nixfred/omarchy that referenced this pull request Aug 23, 2026
Bring the Codex tab of the Agents panel up to parity with the Claude tab.
Codex's account/rateLimits/read already returns more than the collector
surfaced: a plan type, model-scoped allowances under rateLimitsByLimitId
(each with its own 5-hour and weekly window), and a spend-control flag.

- tierLabel is a human plan name ("prolite" -> "Pro Lite", "plus" -> "Plus")
  instead of the raw API token, matching Claude's "Max 20x" / "Pro".
- Window labels follow the Claude collector's wording ("Session (5-hour)",
  "Weekly (7-day)") so the LIMITS section reads the same on both tabs.
- Model-scoped limits (e.g. "GPT-5.3-Codex-Spark") are emitted with a
  title, the way Claude reports "Fable Weekly", so the panel shows them as
  their own rows instead of dropping them.
- spendControlReached surfaces as the usage status text.
- Launch app-server with `-a on-request`, as in omacom#7649, since Codex 0.149
  retired `untrusted` (omacom#7648).

The scanner test's mock app-server now answers with a realistic payload
and asserts the plan label and every limit row.
@omarchybot

Copy link
Copy Markdown
Collaborator

Re-reviewed after @RodriMora's confirmation. Nothing new against the diff — it is unchanged at 74fcdc48 and still correct. What is new is that this pass ran against real codex-cli 0.149.0, which the first review could not: it only had 0.147 available, so the claim that this fixes 0.149 rested on the issue reports rather than on a run.

Verified end-to-end on 0.149.0, on a disposable VM with the collector from this branch. Same binary, same machine, only the policy string differing:

-a untrusted   → {"limitCount":0,"usageStatusText":"Codex limits unavailable","authHelpText":"initialize"}
-a on-request  → {"limitCount":0,"usageStatusText":"","authHelpText":"Run `codex login` to authenticate."}

The first line is exactly the bug in #7648/#7842/#7873. The second is the app-server starting, completing the handshake and answering honestly — that VM is not logged in, which is why the count is still 0. codex --help on 0.149.0 lists exactly on-request and never; -a untrusted exits 2 before app-server starts.

The wire format question is settled, and it matters here. A competing PR (#7829) also rewrites the response parsing to read rateLimitsByLimitId.codex, on the theory that the policy fix alone leaves the limits empty on 0.149. It does not. codex app-server generate-json-schema on 0.149.0 emits GetAccountRateLimitsResponse with rateLimits in its required list, described as "Backward-compatible single-bucket view; mirrors the historical payload". rateLimitsByLimitId is a genuine additional field, but optional and nullable. So the one-line policy change in this PR is a complete fix, not a partial one.

Tests. On the worker, not locally. test/shell.d/agent-usage-codex-scanner-test.sh — 21 passed, 0 failed, through the compositor-backed wrapper. ./test/cli — 116 passed, 0 failed. The new assertion was mutation-tested: flipping only the production flag to never made it fail with not ok - Codex collector uses the supported approval policy and print the actual argv, so it is a real guard rather than a tautology.

On on-request vs never. Four other PRs pick never. Both are accepted by 0.149 and, on this code path, behaviourally identical — I ran the full handshake under each and got the same responses and the same collector output, with no approval traffic either way. fetch_codex_rpc() never starts a turn, so no approval request is reachable, and rpc_request() discards any message whose id does not match (line 493) under a hard deadline (line 482), so an unexpected one could not hang it. never is arguably the more honest label for an unattended collector and would fail closed if a turn were ever added here; that is a preference, not a defect, and it is the maintainer's to settle.

Second opinion. codex at xhigh reviewed the diff independently and found no defect, agreeing with the above — its independence is not currently guaranteed, since it can read this machine's files, so treat the agreement as agreement. It did contribute one thing I had not reasoned about and then confirmed by running it: #7829's precedence, rateLimitsByLimitId.codex or rateLimits, can lose data a truthy-but-partial bucket does not carry. Given rateLimitsByLimitId.codex = {"planType":"pro"} alongside a populated legacy rateLimits, that expression selects the partial bucket and yields zero limit windows where the current parsing yields one. The schema permits it — RateLimitSnapshot has no required properties. That is a note on that PR, not on this one; this PR does not touch the parsing and is unaffected.

Still open from the first review, unchanged. This closes the defect in #7648 but not the issue's second request: an early app-server exit still surfaces as the pending RPC method name, because stderr is subprocess.DEVNULL. #7842 and #7873 ask for the same diagnostic improvement. Fixes #7648 will auto-close all of that, so it is worth deciding whether to keep a diagnostics issue open separately — the run above is a good argument for it, since Run codex login to authenticate. is what a working collector says and initialize is what a broken one says, and today only the first is legible.

Waiting on the maintainer, and specifically on a choice rather than on a fix: six open PRs change this one line — #7649 and #7770 to on-request, #7751, #7825, #7829 and #7864 to never. Nothing pushed to this branch.

@p134c0d3

Copy link
Copy Markdown

Independent confirmation on a ChatGPT Pro account, which I don't think is covered by the reports so far (the earlier confirmation was on a different plan tier).

Environment: Omarchy 4.0.0-1, codex-cli 0.149.0, Linux 7.1.8-arch1-3.

Before the change, ~/.local/state/omarchy/agents/usage/codex.json reads:

"tierLabel": "",
"limits": [],
"usageStatusText": "Codex limits unavailable",
"authHelpText": "initialize"

The "initialize" value is worth calling out because it looks like a status string but is actually str(TimeoutError("initialize")) leaking out of rpc_request — codex exits on the argv before answering, readline hits EOF, and the except Exception as exc in fetch_codex_rpc puts the exception text into authHelpText. So the panel shows the method name where the auth hint belongs, which is what makes this one confusing to diagnose from the UI.

Running the packaged argv directly reproduces it immediately:

$ codex -s read-only -a untrusted app-server
error: invalid value 'untrusted' for '--ask-for-approval <APPROVAL_POLICY>'
  [possible values: on-request, never]

With only this PR's one-word change applied to the collector, the record populates correctly:

"tierLabel": "pro",
"limits": [{"label": "Weekly (7-day)", "percent": <real value>, "resetsAt": "2026-08-27T13:13:29+00:00"}],
"usageStatusText": "",
"authHelpText": "Run `codex login` to authenticate."

So the Pro plan label resolves and the weekly window comes back with a real percentage and reset timestamp.

One note in support of the on-request choice: I also tested -a never and it returns identical results. Both are accepted by 0.149.0, and since fetch_codex_rpc only issues initialize / account/read / account/rateLimits/read and never executes a tool, the approval policy is never consulted on this path — the process is also already constrained by -s read-only. So the two are equivalent here in practice, and there's no behavioural reason to prefer never over what this PR already does.

@omarchybot

Copy link
Copy Markdown
Collaborator

Re-reviewed after @p134c0d3's Pro-account confirmation. The code is unchanged at 74fcdc48 and still correct, so nothing new against the diff. What is new is a sweep of the whole regression cluster and a check of the mechanism @p134c0d3 described. Reviewed by Claude Opus 5 with an independent second opinion from Codex at xhigh reasoning; everything executed below ran on a disposable VM with codex-cli 0.149.0, never on the machine doing the review.

This line is the only thing 0.149 rejects. Swept the base branch for every place Omarchy invokes the codex binary with flags. There are exactly three in production code: bin/omarchy-agent:86 and default/bash/aliases:49 both pass --approve-for-me, which 0.149.0 still lists in --help, and bin/omarchy-agent-usage-codex:531, which is this one. Within that argv only untrusted is stale — -s read-only is still a valid --sandbox value on 0.149.0 (read-only, workspace-write, danger-full-access) and app-server is still a subcommand. So this PR leaves nothing in the cluster unfixed, and there is no second site waiting to break.

#7770 is not a second half of the fix — it is the same change. Its production hunk and this one both resolve bin/omarchy-agent-usage-codex to the identical blob 972c164dee9690bd1282c30966e421198626d848. Worth stating plainly because the two read as complementary: they are byte-identical. Merging both is clean rather than conflicting — git merge-tree of the two heads reports no conflict, and I ran the resulting tree's tests on the worker: agent-usage-codex-scanner-test.sh 21 passed and agent-usage-codex-limits-test.sh 3 passed, both green together. So a maintainer who wants both regression guards can take both; the second production hunk is simply a no-op by then.

@p134c0d3's mechanism holds, and it corrects the timing everyone else has given. rpc_request writes initialize, the already-exited child's stdout is closed, select (line 483) reports it readable, readline (486) returns EOF, if not line: break (487-488) leaves the loop and line 495 raises TimeoutError("initialize"), which except Exception as exc at 559-561 turns into authHelpText via str(exc). It never reaches the 8s deadline. Timed on the worker: the broken argv returns in 42, 44 and 43 ms across three runs, against 137-147 ms for the patched collector, which actually completes a handshake. #7997 describes this as "waits 8s and hits raise TimeoutError" and the note on #7770 as "initialize timing out at 8s"; both have the field right and the path wrong. Codex reached the same conclusion independently and ran a 50-iteration probe against the real binary that took the EOF branch 50 times out of 50 in 2-4 ms — its independence is not currently guaranteed, since it can read this machine's files, but that probe is a measurement rather than an agreement. It added one honest caveat: if the child exits before the stdin flush at line 480 completes, that write raises BrokenPipeError and authHelpText would read [Errno 32] Broken pipe instead. Every report so far shows initialize, so the EOF path is what happens in practice.

Tests. On the worker, not locally. test/shell.d/agent-usage-codex-scanner-test.sh through the compositor-backed wrapper — 21 passed, 0 failed. ./test/cli — 116 passed, 0 failed, exit 0. End-to-end on real 0.149.0, same binary and machine with only the policy differing: -a untrusted gives {"limitCount":0,"usageStatusText":"Codex limits unavailable","authHelpText":"initialize"} and this branch gives {"limitCount":0,"usageStatusText":"","authHelpText":"Run \codex login` to authenticate."}` — the count stays 0 because that VM is signed out, which is exactly what @p134c0d3's Pro-account run resolves on a machine that is not.

One note on the test, not a defect. Codex pointed out that the stub serializes with printf '%s\n' "$*", so argument boundaries are flattened before the comparison: ["-s", "read-only", ...] and ["-s read-only", ...] write the same line, and the second would be rejected by the real CLI while passing this assertion. That is a mutation nobody would plausibly make and it does not weaken the guard against the regression this PR is about, which I mutation-tested earlier and which still fails loudly. Not worth changing; noting it because it is the one concrete respect in which #7864's printf '%s\n' "$@" plus paste is the tighter assertion of the argv-checking group. Your ${CODEX_ARGS_FILE:-} guard is the better half of that comparison — it keeps the stub side-effect-free for the eighteen other collector invocations in the same file, which #7825 and #7892 do not.

Still open, unchanged from the first review. This closes the defect in #7648 but not that issue's second request: an early app-server exit still surfaces as the pending RPC method name, because stderr is subprocess.DEVNULL. Fixes #7648 will auto-close that request along with the defect, so it is worth deciding whether to keep a diagnostics issue open separately. @upnorthmedia sketched a fix for it on #7770 and offered to send it as its own PR, which is the right shape — it does not belong in this one-flag change.

Waiting on the maintainer, on a choice rather than a fix. The cluster has grown to ten open PRs against this single line since the last pass: #7649 and #7770 on on-request, and #7751, #7825, #7829, #7864, #7892, #7908, #7911, #7924 on never — plus #7543 and #7725, which change it as part of a feature. #7997 and #8032 are two more reports of the same breakage. Nothing pushed to this branch.

@diemol55

diemol55 commented Aug 25, 2026 •

Copy link
Copy Markdown

Confirmed this regression is still present after updating to Omarchy 4.0.1-1 on Quattro.

Environment:

  • Omarchy 4.0.1-1
  • codex-cli 0.149.1

The packaged collector still launches Codex with:

[codex, "-s", "read-only", "-a", "untrusted", "app-server"]

This causes the Agents panel to report:

{
  "limits": [],
  "tierLabel": "",
  "usageStatusText": "Codex limits unavailable",
  "authHelpText": "initialize"
}

Codex rejects the argument because untrusted is no longer supported:

error: invalid value 'untrusted' for '--ask-for-approval <APPROVAL_POLICY>'
  [possible values: on-request, never]

Changing only untrusted to on-request restores the Plus tier and weekly limits immediately. No login or credential changes were required.

I regenerated the record with:

omarchy-agent-usage-update codex

and restarted the shell successfully.

The stub joined its arguments with "$*", so the assertion compared one
flattened string and could not tell five arguments from fewer containing
spaces. Passing "-s read-only" and "-a on-request" as single arguments --
which codex rejects as an unexpected argument -- passed the test. NUL
separation and an array comparison keep the boundaries the assertion is
about.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex XHigh <noreply@openai.com>
@omarchybot

Copy link
Copy Markdown
Collaborator

Re-reviewed at 74fcdc4 by Claude Opus 5, with an independent second opinion from Codex at xhigh reasoning. The fix is correct and complete, and @diemol55's report on 4.0.1-1 reproduces exactly.

Verified on a disposable VM running codex-cli 0.149.0, not just read. Reverting the one token on that VM produces the reported failure byte for byte — {"usageStatusText": "Codex limits unavailable", "authHelpText": "initialize"} — and with on-request the collector completes the whole initialize / account/read / account/rateLimits/read handshake against a real app-server in 0.25s. codex --help on 0.149.0 confirms the only remaining values are on-request and never.

This is the whole fix. Both reviewers swept the tree independently: bin/omarchy-agent-usage-codex:531 is the only place that spawns codex with an approval policy, untrusted appears nowhere else, and the other Popen in that file (line 148) runs rg, not codex. bin/omarchy-agent:86 and default/bash/aliases:49 use codex --approve-for-me, which is still a valid flag in 0.149.0, so they are unaffected. No sibling scanner is left holding the dead value.

On the risk that on-request hangs the collector: it cannot, and this was worth checking, because on-request is not a synonym for the old untrusted — it is the value that lets codex ask, and a non-interactive collector blocking on an approval it can never answer would be worse than the bug being fixed. It does not arise here. fetch_codex_rpc() only calls initialize, initialized, account/read and account/rateLimits/read; it never starts a thread or a turn, so nothing can execute a command or apply a patch and no approval request can be raised. Codex added the detail that even if one were raised, rpc_request() at lines 481-494 bounds every read with a select deadline and ignores any message whose id does not match, so the worst case is a 4-8s timeout and the finally block terminating the process — never an indefinite wait.

One fix pushed to this branch (21c1496), in the test you added rather than the collector. The stub captured its argv with printf '%s\n' "$*", which joins everything into one line, so the assertion could not see argument boundaries. Mutating the collector to pass "-s read-only" and "-a on-request" as single arguments — an argv codex rejects outright — still passed the test on the VM. It now captures NUL-separated and compares arrays, which fails both that mutation and a revert to untrusted, while the good argv still passes. To be clear about what this was and was not: the test was imprecise, not vacuous. It already caught the regression it was written for, and both reviewers agreed on that before it was touched.

Test results on the VM: ./test/shell.d/agent-usage-codex-scanner-test.sh 21 passed / 0 failed, and ./test/cli 116 passed / 0 failed, both after the pushed commit.

Waiting on the maintainer, for a reason outside this PR. There are at least twelve open PRs fixing this same token (#7751, #7770, #7825, #7829, #7864, #7892, #7908, #7911, #7924, #8073, #8095 and this one) and five open issues reporting it (#7648, #7842, #7873, #7997, #8032). This one is the earliest, opened 2026-08-21. The one substantive difference between them is the replacement value: ten chose never, while this PR and #7770 chose on-request. Both work today, for the reason above — no turn is ever started, so the policy is inert on this path. never is arguably the better expression of the invariant that nobody can answer a prompt here, and it is what most of the duplicates picked, but neither value matches the old untrusted semantics exactly and both reviewers judged this a maintainer's call rather than a defect, so nothing was changed. Which PR lands, and on which value, is the only open question.

Two pre-existing defects Codex found in rpc_request(), unrelated to this change and deliberately not touched here, since fixing them would expand a one-token PR: line 483-486 calls select() on the file descriptor but reads with TextIOWrapper.readline(), which can buffer a second complete message that select() then reports as unreadable, so a coalesced notification and response could time out spuriously; and lines 493-494 accept any id-matching message without rejecting JSON-RPC error responses, which lines 550-551 silently turn into empty account data and a blank status rather than a reported failure. Those may be worth their own issue.

Note on the second opinion: Codex ran under a read-only sandbox on the same machine, so its independence is not currently guaranteed. Where it agreed on completeness and on the test being imprecise rather than vacuous, it agreed with conclusions already reached. The select()/readline() buffering mechanism and the id-collision path were its own contributions.

ryanrhughes pushed a commit that referenced this pull request Aug 28, 2026
* Fix Codex usage collector approval policy

* Capture codex argv with boundaries in the scanner test

The stub joined its arguments with "$*", so the assertion compared one
flattened string and could not tell five arguments from fewer containing
spaces. Passing "-s read-only" and "-a on-request" as single arguments --
which codex rejects as an unexpected argument -- passed the test. NUL
separation and an array comparison keep the boundaries the assertion is
about.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex XHigh <noreply@openai.com>

---------

Co-authored-by: Omabot <omabot@omarchy.org>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex XHigh <noreply@openai.com>
(cherry picked from commit 4cd8a08)
@ryanrhughes ryanrhughes mentioned this pull request Aug 30, 2026
nick1udwig pushed a commit to nick1udwig/omarchy that referenced this pull request Sep 9, 2026
* Fix Codex usage collector approval policy

* Capture codex argv with boundaries in the scanner test

The stub joined its arguments with "$*", so the assertion compared one
flattened string and could not tell five arguments from fewer containing
spaces. Passing "-s read-only" and "-a on-request" as single arguments --
which codex rejects as an unexpected argument -- passed the test. NUL
separation and an array comparison keep the boundaries the assertion is
about.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex XHigh <noreply@openai.com>

---------

Co-authored-by: Omabot <omabot@omarchy.org>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex XHigh <noreply@openai.com>
@omarchybot omarchybot mentioned this pull request Oct 8, 2026
@emirb emirb added this to the 4.5 milestone Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex 0.149 breaks Agents panel limit display

8 participants