Skip to content

fix(skills): an unreadable skills dir must not abort the run (OPE-216) - #717

Open
devikaverma wants to merge 1 commit into
mainfrom
issue/ope-216-skills-scan-never-aborts-a-run
Open

devikaverma wants to merge 1 commit into
mainfrom
issue/ope-216-skills-scan-never-aborts-a-run

Conversation

@devikaverma

Copy link
Copy Markdown
Collaborator

Two bugs found together while evaluating against benchmark datasets, with OpenWorker running
headless as an unprivileged user. The second made the first hard to find.

What

SkillLoader._discover made its filesystem calls unguarded, and that scan is on the per-turn
path — agent.py:663 calls rescan() every turn to rebuild the live skill menu. pathlib
ignores ENOENT/ENOTDIR/EBADF/ELOOP but propagates EACCES, so an unreadable
<workspace>/.coworker/skills — optional and usually absent — raised straight out of the turn
and ended the run. At startup the directory does not exist, ENOENT is ignored and the session
begins normally, which is why it struck at an arbitrary turn rather than at launch; long runs
died around 100 turns deep, after the work was already done.

The failure was then misreported. _NO_ACCESS listed "permission denied", matched against
str(exc).lower(), and every OS PermissionError stringifies as [Errno 13] Permission denied: <path> — so an unreadable file anywhere in a turn produced "Your account doesn't have
access to <model>", a billing problem that did not exist, sending debugging after a provider
outage that never happened.

  • skills/base.py: _discover degrades to "no skills here" on OSError, and each skill's
    parse is guarded separately so one unreadable or malformed folder costs only that skill
    rather than the rest of the directory.
  • providers/errors.py: drop the "permission denied" marker. Anthropic's genuine marker is
    the underscored type permission_error, already listed and still matching; the OpenAI
    markers are untouched. Unrecognised errors return None so the caller surfaces the raw
    message.
  • Tests: two for the skills scan (unreadable dir, unreadable single folder), one for the error
    classifier.

Testing

  • tests/test_skills.py and tests/test_model_errors.py: 14 passed (3 new). All three new
    tests fail against unmodified main.
  • Skills and provider-error suites: 86 passed, 1 skipped.
  • Full suite on Windows: 2627 passed; the 45 failures are pre-existing and identical on
    unmodified main (toolchain, tool-request, ui-refresh, gallery fixtures).
  • A/B in a container with the trigger manufactured (mkdir /src/.coworker && chmod 000): the
    unfixed build crashed and misreported it, the fixed build returned skills=[] and classified
    the error correctly.

The live skill menu is rebuilt every turn (agent.py calls rescan() while assembling
per-turn context), so every filesystem call in SkillLoader._discover sat on the hot
path unguarded. pathlib ignores ENOENT/ENOTDIR/EBADF/ELOOP but propagates EACCES,
so an unreadable <workspace>/.coworker/skills — an OPTIONAL dir, usually absent —
raised straight out of the turn and terminated the run.

Found while evaluating against benchmark datasets, where the agent runs headless as
an unprivileged user: a root-owned .coworker/ appeared in the workspace mid-session
and long runs died outright, several around 100 turns deep, after the work was
already done. At startup the dir does not exist, ENOENT is ignored and the session
begins normally, which is why it struck at arbitrary points rather than immediately.

_discover now degrades to "no skills here" on OSError, and guards each skill's parse
separately so one unreadable or malformed folder costs only that skill.

Also drop "permission denied" from the provider no-access markers. Every OS
PermissionError stringifies as "[Errno 13] Permission denied: <path>", so that
substring turned any unreadable file into "Your account doesn't have access to
<model>" — a billing failure that did not exist, and hours spent chasing a provider
outage that never happened. Anthropic's genuine marker is the underscored error type
permission_error, which was already listed and still matches.

Three regression tests; all three fail without these changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant