Skip to content

Intelligently fallback to available agent when default agent has exhausted usage - #12105

Open
davidsilvasmith wants to merge 1 commit into
omacom:quattrofrom
davidsilvasmith:agent-usage-fallback
Open

davidsilvasmith wants to merge 1 commit into
omacom:quattrofrom
davidsilvasmith:agent-usage-fallback

Conversation

@davidsilvasmith

Copy link
Copy Markdown

Summary

When diagnosing a crashed process from desktop crash notifications (omarchy-agent-crash) or invoking prompts with fallback, Omarchy previously attempted to launch the default agent unconditionally without checking if that agent had exhausted its rate limits, had 0 usage remaining, or had expired credentials.

Changes

  • bin/omarchy-agent: Added --fallback option. When enabled, it checks the default agent's usage in $XDG_STATE_HOME/omarchy/agents/usage/. If the default agent is exhausted (100% limit used), expired, or unavailable, it automatically inspects other installed coding agents and selects the candidate with the most remaining quota headroom.
  • bin/omarchy-agent-crash: Passes --fallback to omarchy-agent so that clicking "Click to diagnose with AI" on a process crash toast will seamlessly fall back to an active agent subscription when the default agent is out of usage.
  • bin/omarchy-agent-prompt: Supports forwarding --fallback.
  • test/shell.d/agent-fallback-test.sh: Added test suite verifying quota retention, 100% limit fallback, expired token fallback, headroom ranking, and crash forwarding.

@llstrk

llstrk commented Sep 27, 2026

Copy link
Copy Markdown

Automated AI review

Community review: Independent automated community review, unaffiliated with the Omarchy team, intended to help prepare PRs for their review.

Verified: --fallback works as described for the PR's own test data. Callers that do not pass the flag behave as on the base. The only caller that passes it is omarchy-agent-crash, the action behind "Click to diagnose with AI".

On a stock Omarchy install, though, Claude and Codex are the only listed agents with usage collectors. The resolver treats almost every other agent as available, can choose an agent the launcher then refuses, and treats several routine collector states as "exhausted". The notes below suggest tightening what counts as "available" before the crash click relies on it.

Almost every listed agent counts as available on a stock install

An agent counts as installed when any of its command names resolves through shutil.which (bin/omarchy-agent:70-78). An installed agent with no usage record scores 50, and any score above 0 is a usable candidate (:90-91, :147-151). Candidates are sorted by score, so ties fall back to the order of the hard-coded AGENTS list (:53-68, :153-155).

On a stock install:

  • install/user/mise.sh writes a lazy mise stub into ~/.local/bin for 12 of the 14 listed agents (omarchy-mise-install:51-57). The stub installs the agent on first run.
  • Of the listed agents, only claude and codex have in-tree usage collectors (omarchy-agent-usage-update:55-62), so the other installed agents normally have no record and score 50.

Sandboxed run with the stock stubs installed and the Claude default at 100% of its session window:

Crash click, default Claude exhausted Base This PR
Stock stubs, no Codex record claude --permission-mode auto codex --approve-for-me (preinstalled stub)
Stock stubs, Codex record "Codex limits unavailable" claude --permission-mode auto cursor-agent --yolo --trust agent (stub)
Positive control: Codex record at 30% claude --permission-mode auto codex --approve-for-me

Impact: a crash-notification click can install an agent the user never chose and start it in its auto-approve mode with the crash prompt. A newly installed agent may still need a sign-in. Agents without records (pi, opencode, omp) may also draw on the same Claude or Codex subscription that is already exhausted.

Suggested change: only count agents that are actually set up, for example by reusing the presence check omarchy-default-agent already has instead of which. Require a usage record that shows headroom before choosing a fallback candidate (today that limits fallback to Claude and Codex).

The resolver can choose an agent the launcher refuses

The resolver accepts alternative command names: omarchy-launch-openclaw for OpenClaw, cursor for Cursor CLI and gemini for Antigravity (bin/omarchy-agent:56, 58, 61). The launcher then checks the agent name itself with omarchy-cmd-missing and exits 1 if it is missing (:177-180). omarchy-launch-openclaw ships with Omarchy in /usr/share/omarchy/bin, which the resolver searches explicitly, so OpenClaw always counts as installed.

Sandboxed run: preinstalled stubs removed, only Claude installed, Claude at 100%

omarchy-agent-crash (this PR)
  -> resolver: "Default agent 'claude' has no available usage; falling back to 'openclaw'"
  -> launcher: "openclaw is not installed. Choose an installed agent with: omarchy default agent <name>"
  -> exit 1, nothing launched          (base: launches Claude)

A second run with Claude plus a cursor command, but no cursor-agent, did the same with cursor-agent.

Impact: notification actions run detached (shell/plugins/notifications/Service.qml:364-368), so from the crash notification the click would appear to do nothing (inferred from source, not observed on a live desktop), where the base opened the default agent.

Suggested change: check the same command name in the resolver that the launcher checks.

Routine collector states are read as "exhausted"

The resolver scores an agent 0 when usageStatusText contains "expired", "sign in" or "unavailable", or when the highest limits[].percent is at least 1.0 (bin/omarchy-agent:97, 100-114). It does not read updatedAt or resetsAt. Scores from the PR's own evaluate_agent for Claude records:

Claude record Meaning in the collector Score
"Sign-in expired", last-known 10% Saved access token lapsed. Claude Code normally refreshes it when it starts (omarchy-agent-usage-claude:804-821) 0 (fallback)
"Waiting for auth", last-known 10% No credentials at all (:809-812) 95 (kept)
"Claude limits unavailable" Limits probe failed, for example on HTTP 429, with no cached limits still in their window (:841-846) 0 (fallback)
Session 8%, weekly 31%, one model-scoped weekly window at 100% Model-scoped windows share the limits array (:669-708, 761) 0 (fallback)
100% in a window that reset hours ago, record two days old Disabled providers are never rewritten (shell/plugins/agents/Main.qml:146-153) 0 (fallback)

Codex writes "Codex limits unavailable" when its RPC probe times out or raises (omarchy-agent-usage-codex:544-561), with the same result. Sandboxed runs confirmed a fallback to Codex for the lapsed-token, probe-failure, model-scoped and stale cases, where the base launched Claude. The percent units themselves match the collectors (fractions, so 1.0 is 100%).

Impact: a Claude default that has not run for a while, or whose limits probe failed without cached limits, gets replaced by another agent. A fully signed-out Claude with usage history or cached limits is kept. Whether Claude Code keeps working on other models when only a model-scoped window is full was not verified here.

Suggested change: treat "limits unavailable", a lapsed access token and records whose window has reset or whose updatedAt is old as unknown, and keep the default for them. Only a current record that positively shows exhaustion should trigger a fallback. The trade-off is that a truly expired login would then also keep the default.

--fallback picks an agent when no default is set

With no default, default_score stays 0 and the highest-scoring installed candidate is printed, the first in list order when scores tie (bin/omarchy-agent:133-155). The notice at :161 is skipped because $agent is empty, and the existing no-default branch (:171-175: the picker with --pick, otherwise "Choose default agent with: omarchy default agent ") is never reached. That contradicts "Omarchy picks no agent for you" (omarchy-default-agent:20-21).

In a sandboxed run with stock stubs and no default, omarchy agent crash <pid>, omarchy agent prompt --fallback "..." and omarchy-agent --pick --fallback all launched Claude silently. The base printed the "Choose default agent" error, or opened the picker with --pick.

Impact: with no default set, these hand-run commands silently launch an agent the user did not choose. The crash notification itself is not affected, because it is only sent when a default exists (omarchy-crash-watch:66-69).

Suggested change: skip the resolver when no default is set, so the existing no-default flow still runs.

Smaller notes

Optional test improvement: two diagnostic mutants pass all 7 cases of test/shell.d/agent-fallback-test.sh. One resolver ignores scores and always prefers cursor-agent. That passes because cursor-agent has the lowest recorded usage, and so the top score, in every fallback case. The other omarchy-agent-crash drops --prompt "$prompt", which passes because case 5 only greps for the launch prefix. A second candidate with a lower percent in case 4, and a check of the full crash launch including the prompt, would catch both. The cases above (no-record candidates, launcher refusal, no default) are also not covered, because every mock agent counts as installed and every winner has a record.

Verified:


Review information

Test scope: Source review of the four changed files, the usage collectors, the stock mise stubs, the launcher and the notification service, plus sandboxed runs with a synthetic home, synthetic usage records, mocked launchers and agents, and the stock mise stubs generated by omarchy-mise-install. No real agent, terminal, desktop session or usage API was used. Model-scoped Claude behaviour and live latency were not observed.

AI process: Opus 5.5 Medium coordination and synthesis, Opus 5.5 Xhigh technical review and final fact check, GPT 6 Sol Xhigh search for related issues, Opus 5.5 Medium editorial check.

Opt out: To stop receiving these reviews, reply to this comment saying so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants