Skip to content

Run only the Integrations checks in its doctor tests - #290

Merged
Silthus merged 2 commits into
mainfrom
test/speed-integrations-doctor
Nov 19, 2026
Merged

Silthus merged 2 commits into
mainfrom
test/speed-integrations-doctor

Conversation

@Silthus

@Silthus Silthus commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Note

🔵 Merge risk: low · two-way door

  • Only the Integrations pack's doctor tests change: their runner no longer spawns other packs' checks.
  • No shipped code, pack, or CLI behavior changes; bun run typecheck and the pack's tests cover the edit.
  • Switch-off: none: live on merge, fix forward.
  • Tier: non-critical

Problem

The test "the Integrations pack > doctor passes each sign-in once its provider reports it, and lists only the ones still missing" timed out at 5 s in CI three times on 2026-11-19 (on #287 and #286). Each red cleared on a rerun.

The doctor() helper ran the real check() with a runner that spawned every command. That includes Base's always-on checks: three Bun scripts and npx --version, which starts Node. Each doctor call ran them, and the test calls doctor twice. When every bun test --parallel worker starts at once, these process starts push the test past 5 s.

Change

The doctor runner now spawns only the commands of the Integrations pack's own checks, taken from the resolved plan. It answers every other command with exit 1 and starts no process. The test reads only Integrations findings already (signInsOf).

The four real Integrations sh -c checks still run against the fake claude and codex scripts, so the tests still prove the pack's check commands.

Proof

Test times from bun test --reporter=junit, on a devbox, file run alone:

Test Before After
doctor passes each sign-in … 0.39 s 0.04 to 0.06 s
doctor accepts a Context7 API key … 0.36 s 0.015 s

A mutation check: changing grep -qx OAuth to grep -qx OAuthX in the pack's context7-codex check fails "doctor passes each sign-in …". So the real check commands still run.

A second mutation check: changing [ -n "$CONTEXT7_API_KEY" ] to [ -n "$CONTEXT7_API_KEYX" ] in the same check fails "doctor accepts a Context7 API key …". So the API-key branch is covered too.

bun run typecheck passes.

Impact

Removes a flaky CI red that blocked PR gates and needed reruns.

🤖 Generated with Claude Code

How did you test this code?

Proof What ran Result Commit
Review 1-lens panel (gpt-6.2-sol), round 1 1 findings: 0 fixed, 0 rejected, 1 other d9294e7
Proof judge an independent model reads the evidence against the diff ✅ passed at 0d51908

Not checked: none recorded
Full receipt (PostHog team): https://artifacts.posthog.com/a/0Sb4lXuuI5Q5NLJqvt4Ued

Silthus and others added 2 commits November 19, 2026 15:46
The doctor tests spawned Base's always-on checks too, three Bun scripts and
npx, on each doctor call. Under bun test --parallel those process starts
pushed the sign-in test past 5 s in CI. The runner now spawns only the
Integrations pack's own check commands against the fake CLIs and fails
every other command without a process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus
Silthus marked this pull request as ready for review November 19, 2026 16:02
@Silthus
Silthus merged commit 9e76276 into main Nov 19, 2026
46 of 47 checks passed
@Silthus
Silthus deleted the test/speed-integrations-doctor branch November 19, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant