[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 - #3559
Conversation
The canary e2e lanes (Local Dev / Local Prod / Local Postgres for nextjs-turbopack and nextjs-webpack) were pinned to 16.3.0-canary.2. Bump the pin to the current canary, 16.3.1-canary.17. Fresh canaries are younger than the repo's 48h minimumReleaseAge gate, which rejects the pinned version at the "Setup canary" install step, so exclude next and its lockstep-published @next/* companion packages from the gate — both are Vercel-published, matching the existing exclusions (@vercel/*, turbo, esbuild). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
🧪 E2E Test Results✅ All tests passed
|
| Passed | Failed | Skipped | Total | |
|---|---|---|---|---|
| ✅ ▲ Vercel Production | 3466 | 0 | 590 | 4056 |
| ✅ 💻 Local Development | 3810 | 0 | 558 | 4368 |
| ✅ 📦 Local Production | 3810 | 0 | 558 | 4368 |
| ✅ 🐘 Local Postgres | 3810 | 0 | 558 | 4368 |
| ✅ 🪟 Windows | 312 | 0 | 0 | 312 |
| ✅ vercel-multi-region | 27 | 0 | 0 | 27 |
| Total | 15235 | 0 | 2264 | 17499 |
Details by Category
✅ ▲ Vercel Production
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-node | 128 | 0 | 28 |
| ✅ astro-quickjs | 128 | 0 | 28 |
| ✅ example-node | 128 | 0 | 28 |
| ✅ example-quickjs | 128 | 0 | 28 |
| ✅ express-node | 128 | 0 | 28 |
| ✅ express-quickjs | 128 | 0 | 28 |
| ✅ fastify-node | 128 | 0 | 28 |
| ✅ fastify-quickjs | 128 | 0 | 28 |
| ✅ hono-node | 128 | 0 | 28 |
| ✅ hono-quickjs | 128 | 0 | 28 |
| ✅ nest-node | 128 | 0 | 28 |
| ✅ nest-quickjs | 128 | 0 | 28 |
| ✅ nextjs-turbopack-node | 153 | 0 | 3 |
| ✅ nextjs-turbopack-quickjs | 153 | 0 | 3 |
| ✅ nextjs-webpack-node | 153 | 0 | 3 |
| ✅ nextjs-webpack-quickjs | 153 | 0 | 3 |
| ✅ nitro-node | 128 | 0 | 28 |
| ✅ nitro-quickjs | 128 | 0 | 28 |
| ✅ nuxt-node | 128 | 0 | 28 |
| ✅ nuxt-quickjs | 128 | 0 | 28 |
| ✅ sveltekit-node | 147 | 0 | 9 |
| ✅ sveltekit-quickjs | 147 | 0 | 9 |
| ✅ tanstack-start-node | 128 | 0 | 28 |
| ✅ tanstack-start-quickjs | 128 | 0 | 28 |
| ✅ vite-node | 128 | 0 | 28 |
| ✅ vite-quickjs | 128 | 0 | 28 |
✅ 💻 Local Development
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 130 | 0 | 26 |
| ✅ astro-stable-quickjs | 130 | 0 | 26 |
| ✅ express-stable-node | 130 | 0 | 26 |
| ✅ express-stable-quickjs | 130 | 0 | 26 |
| ✅ fastify-stable-node | 130 | 0 | 26 |
| ✅ fastify-stable-quickjs | 130 | 0 | 26 |
| ✅ hono-stable-node | 130 | 0 | 26 |
| ✅ hono-stable-quickjs | 130 | 0 | 26 |
| ✅ nest-stable-node | 130 | 0 | 26 |
| ✅ nest-stable-quickjs | 130 | 0 | 26 |
| ✅ nextjs-turbopack-canary-node | 137 | 0 | 19 |
| ✅ nextjs-turbopack-canary-quickjs | 137 | 0 | 19 |
| ✅ nextjs-turbopack-stable-node | 156 | 0 | 0 |
| ✅ nextjs-turbopack-stable-quickjs | 156 | 0 | 0 |
| ✅ nextjs-webpack-canary-node | 137 | 0 | 19 |
| ✅ nextjs-webpack-canary-quickjs | 137 | 0 | 19 |
| ✅ nextjs-webpack-stable-node | 156 | 0 | 0 |
| ✅ nextjs-webpack-stable-quickjs | 156 | 0 | 0 |
| ✅ nitro-stable-node | 130 | 0 | 26 |
| ✅ nitro-stable-quickjs | 130 | 0 | 26 |
| ✅ nuxt-stable-node | 130 | 0 | 26 |
| ✅ nuxt-stable-quickjs | 130 | 0 | 26 |
| ✅ sveltekit-stable-node | 149 | 0 | 7 |
| ✅ sveltekit-stable-quickjs | 149 | 0 | 7 |
| ✅ tanstack-start-node | 130 | 0 | 26 |
| ✅ tanstack-start-quickjs | 130 | 0 | 26 |
| ✅ vite-stable-node | 130 | 0 | 26 |
| ✅ vite-stable-quickjs | 130 | 0 | 26 |
✅ 📦 Local Production
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 130 | 0 | 26 |
| ✅ astro-stable-quickjs | 130 | 0 | 26 |
| ✅ express-stable-node | 130 | 0 | 26 |
| ✅ express-stable-quickjs | 130 | 0 | 26 |
| ✅ fastify-stable-node | 130 | 0 | 26 |
| ✅ fastify-stable-quickjs | 130 | 0 | 26 |
| ✅ hono-stable-node | 130 | 0 | 26 |
| ✅ hono-stable-quickjs | 130 | 0 | 26 |
| ✅ nest-stable-node | 130 | 0 | 26 |
| ✅ nest-stable-quickjs | 130 | 0 | 26 |
| ✅ nextjs-turbopack-canary-node | 137 | 0 | 19 |
| ✅ nextjs-turbopack-canary-quickjs | 137 | 0 | 19 |
| ✅ nextjs-turbopack-stable-node | 156 | 0 | 0 |
| ✅ nextjs-turbopack-stable-quickjs | 156 | 0 | 0 |
| ✅ nextjs-webpack-canary-node | 137 | 0 | 19 |
| ✅ nextjs-webpack-canary-quickjs | 137 | 0 | 19 |
| ✅ nextjs-webpack-stable-node | 156 | 0 | 0 |
| ✅ nextjs-webpack-stable-quickjs | 156 | 0 | 0 |
| ✅ nitro-stable-node | 130 | 0 | 26 |
| ✅ nitro-stable-quickjs | 130 | 0 | 26 |
| ✅ nuxt-stable-node | 130 | 0 | 26 |
| ✅ nuxt-stable-quickjs | 130 | 0 | 26 |
| ✅ sveltekit-stable-node | 149 | 0 | 7 |
| ✅ sveltekit-stable-quickjs | 149 | 0 | 7 |
| ✅ tanstack-start-node | 130 | 0 | 26 |
| ✅ tanstack-start-quickjs | 130 | 0 | 26 |
| ✅ vite-stable-node | 130 | 0 | 26 |
| ✅ vite-stable-quickjs | 130 | 0 | 26 |
✅ 🐘 Local Postgres
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 130 | 0 | 26 |
| ✅ astro-stable-quickjs | 130 | 0 | 26 |
| ✅ express-stable-node | 130 | 0 | 26 |
| ✅ express-stable-quickjs | 130 | 0 | 26 |
| ✅ fastify-stable-node | 130 | 0 | 26 |
| ✅ fastify-stable-quickjs | 130 | 0 | 26 |
| ✅ hono-stable-node | 130 | 0 | 26 |
| ✅ hono-stable-quickjs | 130 | 0 | 26 |
| ✅ nest-stable-node | 130 | 0 | 26 |
| ✅ nest-stable-quickjs | 130 | 0 | 26 |
| ✅ nextjs-turbopack-canary-node | 137 | 0 | 19 |
| ✅ nextjs-turbopack-canary-quickjs | 137 | 0 | 19 |
| ✅ nextjs-turbopack-stable-node | 156 | 0 | 0 |
| ✅ nextjs-turbopack-stable-quickjs | 156 | 0 | 0 |
| ✅ nextjs-webpack-canary-node | 137 | 0 | 19 |
| ✅ nextjs-webpack-canary-quickjs | 137 | 0 | 19 |
| ✅ nextjs-webpack-stable-node | 156 | 0 | 0 |
| ✅ nextjs-webpack-stable-quickjs | 156 | 0 | 0 |
| ✅ nitro-stable-node | 130 | 0 | 26 |
| ✅ nitro-stable-quickjs | 130 | 0 | 26 |
| ✅ nuxt-stable-node | 130 | 0 | 26 |
| ✅ nuxt-stable-quickjs | 130 | 0 | 26 |
| ✅ sveltekit-stable-node | 149 | 0 | 7 |
| ✅ sveltekit-stable-quickjs | 149 | 0 | 7 |
| ✅ tanstack-start-node | 130 | 0 | 26 |
| ✅ tanstack-start-quickjs | 130 | 0 | 26 |
| ✅ vite-stable-node | 130 | 0 | 26 |
| ✅ vite-stable-quickjs | 130 | 0 | 26 |
✅ 🪟 Windows
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack-node | 156 | 0 | 0 |
| ✅ nextjs-turbopack-quickjs | 156 | 0 | 0 |
✅ vercel-multi-region
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack | 27 | 0 | 0 |
📊 Workflow Benchmarkscommit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 226900ms → this run 172717ms (Δ -54183ms, -24%) ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
There was a problem hiding this comment.
Pull request overview
Updates the CI “Next.js canary” E2E lanes to a newer Next.js canary while ensuring pnpm’s minimumReleaseAge gate doesn’t block installs of freshly published canary artifacts.
Changes:
- Bump the canary pin used by the local E2E jobs from
16.3.0-canary.2to16.3.1-canary.17. - Add
nextand@next/*to pnpm’sminimumReleaseAgeExcludeso CI can install new canaries withoutERR_PNPM_NO_MATURE_MATCHING_VERSION.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
pnpm-workspace.yaml |
Excludes next / @next/* from the minimum release-age gate to allow fresh canary installs in CI. |
.github/workflows/tests.yml |
Updates the hardcoded canary pin used in the canary setup step across the local E2E jobs. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Sim WorldSimulated world deterministic testing for races. Traces 🟠 Mint-ordered log — 3 fail of 41 total
Full trace: 🟢 Append-only log — 0 fail of 41 total
Full trace: |
Summary & Motivation
The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for
nextjs-turbopackandnextjs-webpack, × node/quickjs) were pinned to16.3.0-canary.2. This bumps the pin to the current canary,16.3.1-canary.17, in all six spots intests.yml.Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h
minimumReleaseAgegate, so the "Setup canary"pnpm install --no-frozen-lockfilefails withERR_PNPM_NO_MATURE_MATCHING_VERSION(reproduced locally with this exact version). This addsnextand its lockstep-published@next/*companion packages (@next/env,@next/swc-*) tominimumReleaseAgeExclude— both are Vercel-published, matching the trust model of the existing exclusions (@vercel/*,turbo,esbuild).Validation
The full e2e suite was run locally against
next@16.3.1-canary.17on all three worlds (staged tarball workbenches, same as CI'sprepare-workbench-path):world-localdev.test.ts5 passed / 4 skipped (incl. the HMR fuzz test) ·test:e2e137 passed / 19 skippedworld-postgreslocal-build.test.ts13 passed ·test:e2e137 passed / 19 skipped¹world-verceldpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ), node vmtest:e2e134 passed / 22 skipped¹ First pass failed the 3
pages routertests because the local harness ranlocal-build.test.tswithoutCI=true, which deletes theworkflow-sourcemap-warning-fixturepackage the built output still references (the test preserves it only whenCI=true, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue.Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got.
Context on current
mainCIRecent
mainruns are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completedmainrun (with #3530's e2e retry) passed every canary lane and failed only the stable webpack dev lane.🤖 Generated with Claude Code