Skip to content

ci: shard windows rust tests across 2 parallel jobs via nextest partitioning - #366

Open
Ziinc wants to merge 3 commits into
mainfrom
claude/ci-nextest-partition-windows
Open

ci: shard windows rust tests across 2 parallel jobs via nextest partitioning#366
Ziinc wants to merge 3 commits into
mainfrom
claude/ci-nextest-partition-windows

Conversation

@Ziinc

@Ziinc Ziinc commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • The windows-latest / rust / test job's cargo nextest run step alone reports 937.090s (~15.6 min) of actual test execution out of the job's ~22 minute total — the tests are dominated by jj/git subprocess spawn overhead (individual tests taking 15-49s each), not CPU-bound work.
  • Splits that single matrix entry into two parallel entries using cargo nextest run --partition count:N/2, so each shard runs roughly half the tests concurrently instead of serially in one job.
  • Job names now show the partition, e.g. rust / test / windows-latest (1/2).
  • Reduced timeout-minutes from 45 to 30 per shard to reflect the expected halved runtime, while leaving headroom.

Why this approach

cargo nextest's built-in partitioning (--partition count:N/M) deterministically splits the test binary's test list across shards without needing to hand-maintain a test allowlist per shard, and it composes cleanly with the existing Swatinem/rust-cache step (both shards share the same cache key since they compile identical test binaries — only which tests run differs).

Test plan

  • Confirm both rust / test / windows-latest (1/2) and rust / test / windows-latest (2/2) jobs pass in this PR's CI run.
  • Confirm total test count across both shards' summaries adds up to the full suite (663 as of the baseline run) with no duplicates or gaps.
  • Compare wall time of each shard against the ~16 min baseline for the unsharded run.
    Generated by Claude Code

Ziinc added 3 commits August 28, 2026 05:30
…tioning

The windows-latest rust test step alone takes ~16 of the job's ~22
minutes, dominated by jj/git subprocess spawn overhead rather than
compute. Splitting into two nextest partitions running in parallel
should roughly halve that wall time for the same total CPU cost.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant