chore(parity): mine the reference CLI's own e2e test fixtures into declarative parity cases

[pofallon]

The growth rule in docs/PARITY.md / CLAUDE.md says to "periodically mine the reference's own e2e fixtures and real-world configs for scenarios the suite does not have yet" — this issue makes that actionable for the richest single vein: the reference CLI's own test suite.

The idea

@devcontainers/cli ships hundreds of e2e test configs (src/test/** in devcontainers/cli — per-scenario devcontainer.jsons, Dockerfiles, compose files, feature fixtures). Each one encodes a behavior upstream cared enough to pin. Running BOTH CLIs over them differentially asks, for free, "does deacon handle everything upstream's own tests exercise?" — coverage chosen by the reference's authors, not ours, which is exactly the blind-spot antidote.

Method (data-only — this is the suite's whole design)

1. Harvest from the tag matching the oracle pin (v0.87.0, so fixtures and binary agree), not main.
2. Triage the harvest: drop configs exercising out-of-scope surfaces (feature authoring/test/publish), dedupe against existing parity/fixtures/ coverage (many shapes are already covered — the win is the tail), group the rest by area.
3. For each survivor: vendor the fixture into parity/fixtures/fx-upstream-*/ (MIT-licensed; keep upstream attribution in a fixture README line), author a live-differential case — a pure data edit, no new Rust.
4. Run the batch. Every divergence follows the standard triage from docs/PARITY.md: deacon wrong → issue + red case; reference deviates from spec → recorded conformant row; spec-silent deliberate → maintainer ruling before any tolerance.
5. Land in batches by area, each batch's SPEC_STATUS rows in the same commit. Expect the first batch to be the most informative and the cheapest.

Guardrails

• Data and fixtures only; if a fixture needs a runner capability the model lacks, skip it and list it in the report — no machinery growth to chase a fixture.
• Environment-pinned fixtures (uid assumptions, huge images, arch-specific) get the same treatment the suite already documents: first run is a measurement, and scripts/parity/prepull-fixture-images.sh must discover anything image-hungry.
• Volume control: this is a periodic mining pass, not a bulk import — a batch that adds 500 cases nobody triaged is worse than 30 that were each read.

Second vein, same method, separate pass: real-world devcontainer.json files from popular public repos.

[pofallon]

Remainder inventory — the queue for batch 2+

Every config harvested from devcontainers/cli@v0.87.0 src/test/** that is neither in batch 1 (#485) nor dropped as out-of-scope or already-covered. Grouped by the reason it was held back, which is also the thing a later batch has to solve first.

codspace OCI Features + tarballs; the lockfile family needs its own batch — 12

• src/test/container-features/configs/lockfile-dependson/.devcontainer.json
• src/test/container-features/configs/lockfile-frozen-no-lockfile/.devcontainer.json
• src/test/container-features/configs/lockfile-frozen/.devcontainer.json
• src/test/container-features/configs/lockfile-generate-from-empty-file-frozen/.devcontainer/devcontainer.json
• src/test/container-features/configs/lockfile-generate-from-empty-file/.devcontainer/devcontainer.json
• src/test/container-features/configs/lockfile-no-lockfile/.devcontainer.json
• src/test/container-features/configs/lockfile-oci-integrity/.devcontainer.json
• src/test/container-features/configs/lockfile-outdated-command/.devcontainer.json
• src/test/container-features/configs/lockfile-outdated/.devcontainer.json
• src/test/container-features/configs/lockfile-tarball-integrity/.devcontainer.json
• src/test/container-features/configs/lockfile-upgrade-command/.devcontainer.json
• src/test/container-features/configs/lockfile/.devcontainer.json

arch-pinned (linux/arm64) — 4

• src/test/configs/updateUIDarm64-platform-option/.devcontainer.json
• src/test/configs/updateUIDarm64/.devcontainer.json
• src/test/configs/updateUIDarm64v8-platform-option/.devcontainer.json
• src/test/configs/updateUIDarm64v8/.devcontainer.json

private / cross-cloud registries and anonymous-auth probes — 4

• src/test/container-features/configs/registry-compatibility/azure-anonymous/.devcontainer.json
• src/test/container-features/configs/registry-compatibility/azure-registry-scoped/.devcontainer.json
• src/test/container-features/configs/registry-compatibility/github-anonymous/.devcontainer.json
• src/test/container-features/configs/registry-compatibility/github-private/.devcontainer.json

compose project naming collides with an already-adjudicated documented choice (bhv: deacon-namespaced project name) — needs a maintainer ruling before it can land non-red — 3

• src/test/configs/compose-with-name-and-custom-yaml/.devcontainer/devcontainer.json
• src/test/configs/compose-with-name-using-env-var/.devcontainer/devcontainer.json
• src/test/configs/compose-with-name/.devcontainer/devcontainer.json

ghcr.io/codspace/* Features — third-party registry content, unpinnable — 3

• src/test/container-features/configs/feature-dependencies/dependsOn/oci-ab/.devcontainer/devcontainer.json
• src/test/container-features/configs/feature-dependencies/dependsOn/oci-fgh/.devcontainer/devcontainer.json
• src/test/container-features/configs/feature-dependencies/dependsOn/oci-ij/.devcontainer/devcontainer.json

arch-pinned (linux/amd64 build + runArgs --platform) — 2

• src/test/configs/updateUIDamd64-platform-option/.devcontainer.json
• src/test/configs/updateUIDamd64/.devcontainer.json

codspace OCI Features exercising legacyIds renaming — 2

• src/test/container-features/configs/example-legacyIds-2/.devcontainer.json
• src/test/container-features/configs/example-legacyIds/.devcontainer.json

codspace OCI + tarball Features — 2

• src/test/container-features/configs/feature-dependencies/overrideFeatureInstallOrder/image-with-v1-features-with-overrideFeatureInstallOrder/.devcontainer/devcontainer.json
• src/test/container-features/configs/feature-dependencies/overrideFeatureInstallOrder/image-with-v2-features-with-overrideFeatureInstallOrder/.devcontainer/devcontainer.json

resolution failure depends on a live registry 404 — 2

• src/test/container-features/configs/invalid-configs/invalid-v1-features/.devcontainer.json
• src/test/container-features/configs/invalid-configs/invalid-v2-features/.devcontainer.json

bare local id myfeature with no ./ prefix — a v1 shape whose resolution needs a live registry probe to fail the way upstream expects — 1

• src/test/configs/image-with-local-feature/.devcontainer.json

requires a host LOCAL_GID env var the runner does not set — 1

• src/test/configs/updateUIDOnly/.devcontainer.json

ghcr.io/codspace/* Feature — 1

• src/test/container-features/configs/feature-dependencies/dependsOn-and-installsAfter/a/.devcontainer.json

GitHub release tarballs — network-flaky, unpinnable — 1

• src/test/container-features/configs/feature-dependencies/dependsOn/tgz-ab/.devcontainer/devcontainer.json

mixes a local graph with a codspace OCI dependency — 1

• src/test/container-features/configs/feature-dependencies/overrideFeatureInstallOrder/mixed/.devcontainer/devcontainer.json

GitHub release tarball Feature — 1

• src/test/container-features/configs/image-with-v2-tarball/.devcontainer.json

Dropped as already covered — 22

• src/test/configs/compose-Dockerfile-with-features/.devcontainer/devcontainer.json → fx-up-compose-feature / fx-build-compose-feature
• src/test/configs/compose-Dockerfile-with-target/.devcontainer/devcontainer.json → fx-build-compose / fx-tier1-dockerfile-build
• src/test/configs/compose-Dockerfile-without-features/.devcontainer/devcontainer.json → fx-us5-compose-env-precedence
• src/test/configs/compose-image-with-features/.devcontainer/devcontainer.json → fx-up-compose-feature
• src/test/configs/compose-image-with-mounts/.devcontainer/devcontainer.json → fx-upstream-mount-object-anonymous-volume (same object-mount shape)
• src/test/configs/compose-image-without-features-minimal/.devcontainer/devcontainer.json → fx-up-compose-basic
• src/test/configs/compose-image-without-features/.devcontainer/devcontainer.json → fx-up-compose-basic + the legacy-props finding already filed
• src/test/configs/compose-without-name/.devcontainer/devcontainer.json → fx-tier1-compose-array / fx-tier1-compose-postgres
• src/test/configs/dockerfile-with-features/.devcontainer.json → fx-tier1-dockerfile-build / fx-up-dockerfile-feature
• src/test/configs/dockerfile-with-parallel-commands/.devcontainer.json → fx-us5-lifecycle-object
• src/test/configs/dockerfile-without-features/.devcontainer.json → fx-tier1-dockerfile-build
• src/test/configs/example/.devcontainer.json → fx-up-feature-single
• src/test/configs/image-metadata/.devcontainer/devcontainer.json → fx-ruc-image-metadata / fx-up-compose-image-metadata
• src/test/configs/image-with-features/.devcontainer.json → fx-up-feature-single / fx-tier1-node-ts
• src/test/configs/poetry-example/.devcontainer.json → fx-us5-lifecycle-object (object hook whose value is an array)
• src/test/container-features/configs/example-installsAfter/.devcontainer.json → fx-upstream-dependson-installsafter-local (local equivalent)
• src/test/container-features/configs/feature-dependencies/dependsOn/invalid-circular/.devcontainer/devcontainer.json → fx-errorpath-up-feature-cycle
• src/test/container-features/configs/feature-dependencies/dependsOn/local-simple/.devcontainer/devcontainer.json → fx-upstream-dependson-local-with-options (superset)
• src/test/container-features/configs/feature-dependencies/installsAfter/local-simple/.devcontainer/devcontainer.json → fx-upstream-dependson-installsafter-local
• src/test/container-features/configs/feature-dependencies/overrideFeatureInstallOrder/local-simple/.devcontainer/devcontainer.json → fx-upstream-override-order-intermediate (superset)
• src/test/container-features/configs/image-with-v1-features-node-python-local-cache/.devcontainer.json → same v1 bare-id path as fx-upstream-feature-digest-and-v1-ids
• src/test/container-features/configs/lifecycle-hooks-resume-existing-container/.devcontainer/devcontainer.json → fx-up-stale-config-lifecycle + the two batch-1 hook fixtures

Dropped as out of scope — 7

• src/test/configs/disallowed-features/.devcontainer/allowed/devcontainer.json
• src/test/configs/disallowed-features/.devcontainer/disallowed/devcontainer.json
• src/test/configs/set-up-with-config/devcontainer.json
• src/test/container-templates/example-templates-sets/simple/src/alpine/.devcontainer.json
• src/test/container-templates/example-templates-sets/simple/src/cpp/.devcontainer/devcontainer.json
• src/test/container-templates/example-templates-sets/simple/src/mytemplate/.devcontainer/devcontainer.json
• src/test/container-templates/example-templates-sets/simple/src/node-mongo/.devcontainer/devcontainer.json

Counts: 91 harvested = 22 batch-1 + 22 already-covered + 40 environment-hostile + 7 out-of-scope.

[pofallon]

Recognize this on sight: an upstream race in @devcontainers/cli 0.87.0 that makes a parity case red for neither side's reasons

Recorded here because batch 1 hit it, it cost real diagnosis time, and the failure signature
looks nothing like the thing it actually is. Nothing has been filed upstream — whether to
report it is the maintainer's call; this writeup is here to make that easy and to make the
next encounter a five-second recognition instead of an investigation.

The signature

A live-differential up case fails with a chan-exit-code divergence in which the
reference is the side that exited 1, and its stderr tail ends:

Error: ENOENT: no such file or directory, rename
  '/tmp/devcontainercli-runner/updateUID.Dockerfile-0.87.0-1785891315906'
  -> '/tmp/devcontainercli-runner/updateUID.Dockerfile-0.87.0'

The mechanism

When the reference has to remap a container user's uid to the host's, it generates a small
updateUID.Dockerfile and stages it under /tmp/devcontainercli-runner/. The destination
name is scoped by the CLI's version and nothing else —
updateUID.Dockerfile-<version> — and it is produced by writing a timestamp-suffixed temp
file and renaming it onto that fixed path.

So the path is global across every concurrent invocation on the machine. Two reference up
runs that both remap a uid race: whichever renames second finds its own source already
consumed, and dies with ENOENT on the rename.

Our docker-shared driver runs at concurrency 4, which is precisely the condition that
makes it reachable.

Why it is easy to miss locally

Whether a remap happens at all depends on the host uid, so the race is invisible on a dev
box whose uid is 1000:

• mcr.microsoft.com/devcontainers/base:bookworm sets remoteUser: vscode in its
  devcontainer.metadata, and vscode is uid 1000 in that image.
• This project's dev container is also uid 1000 → uid already matches → no remap, no temp
  file, no race.
• A GitHub Actions runner is uid 1001 → every up on such a base remaps → several remaps
  land concurrently → race.

Batch 1 ran green locally across two full docker-shared passes and then produced a ninth
red case in CI for exactly this reason. Per CLAUDE.md's own rule, the first run on a new
substrate is a measurement, not a regression — this is a concrete instance of it.

What to do when you see it

1. Do not chase it as a deacon divergence. Read which side exited non-zero; the
   chan-exit-code verdict now carries a stderr tail of the failing side precisely so this
   is answerable without a re-run (chore(parity): gating lanes upload no evidence on failure, and verdicts omit deacon's stderr #474 / chore(parity): make a failing gating lane diagnosable — evidence artifacts and stderr in exit-code verdicts #478).
2. Ask whether the case CLAIMS anything about uid remapping. If it does not, the base
   image is bringing an interaction the case never wanted: rebase the fixture on an image
   with no remoteUser (debian:bookworm-slim works and is already cached) and the case
   stops touching the remap path entirely. That is what batch 1 did for
   fx-upstream-lifecycle-hooks-inline and fx-upstream-lifecycle-hooks-advanced, whose
   claim is lifecycle-hook collection.
3. If it DOES claim uid remapping — today only fx-upstream-updateuid-remoteuser and
   fx-up-compose-image-metadata do — the case has to stay on the remap path, and the race
   is a real flake risk to weigh. Two such cases have coexisted without incident; more than
   a couple of concurrent remappers is asking for it.

Both lifecycle fixtures now carry a "Do not upgrade this base" section in their
UPSTREAM.md, and both cases' notes carry the ENOENT evidence, so the reasoning travels
with the thing it protects rather than living only here.

[pofallon]

Correction / measurement to the race writeup above. I wrote that "today only
fx-upstream-updateuid-remoteuser and fx-up-compose-image-metadata" claim uid remapping.
That is right about which cases claim it, but wrong about how many can trigger it, and
the second number is the one that governs the race. Measured over the committed case data —
live-differential cases in a Docker resource group with an up operation, whose fixture
declares remoteUser or a remoteUser-bearing base:

18 cases can reach the reference's uid-remap path. Most are pre-existing and several are
error-path cases that fail before they get there, which is presumably why two years of
nightlies never surfaced this. The relevant number for batch 1 is its own contribution:
it now adds exactly one (case-up-upstream-remote-user-uid-remap, the case written for
the claim), down from three before the lifecycle fixtures were rebased.

So the mitigation is not "few remappers exist" — it is "do not add remappers that no case
claims". A future batch should apply the same test: if the case does not measure uid
remapping, do not let its base image opt into it.

[pofallon]

Remainder inventory after batch 2 (PR #559) — the queue for batch 3+

Batch 2 took the lockfile family, which the previous inventory designated as needing its own
batch. Verdict: mostly untenable, exactly as flagged — 3 of 13 configs landed (as 5 cases), 10 deferred, and
the deferrals are content problems rather than effort problems. Two deacon-side defects came
out of it (#556,
#557); #557 carries a spec-silent half
awaiting a ruling.

The finding worth carrying forward: a codspace/tarball dependency is not always fatal.
Where the Feature's IDENTITY is incidental to the claim (does a lockfile get written? is it
read? is it refused?), repinning to a first-party immutable version is a faithful
adaptation. Where the claim IS the artifact — an expected lockfile's exact bytes, a
publisher's dependsOn graph, a tarball's integrity hash — repinning destroys the claim
rather than adapting it, and the config must be deferred. That test is what split this
family 3/10 and is worth applying to the remaining codspace groups below.

lockfile family — RESOLVED: 3 configs landed as 5 cases, 10 deferred

Landed (batch 2): lockfile-no-lockfile, lockfile-frozen-no-lockfile, lockfile-frozen
→ fx-upstream-lockfile-no-lockfile, fx-upstream-lockfile-frozen, five cases.

Still deferred, with the reason each is not a scheduling problem:

• lockfile, lockfile-dependson, lockfile-outdated, lockfile-tarball-integrity —
  the claim IS the third-party artifact. Each asserts the exact generated lockfile for
  specific ghcr.io/codspace/* Features and/or GitHub release tarballs; repinning the
  Features would leave a case that no longer tests what upstream wrote it to test.
• lockfile-outdated-command — resolves latest for six Features live, so its expectations
  move whenever any of them publishes. outdated already carries 22 committed cases.
• lockfile-upgrade-command — upgrade already carries 11, including
  case-upgrade-regenerates-lockfile, which compares the regenerated document across both
  CLIs. Nothing new is reachable here.
• lockfile-oci-integrity — the most landable of the deferrals, and the sharpest
  remaining claim in the family: a lockfile whose integrity hash is deliberately wrong
  for the digest it names must fail the build on both sides. Fully expressible on a
  first-party pinned Feature. Not landed only because batch 2 did not MEASURE it, and an
  unmeasured case is not a case. Recommended first item for batch 3.
• lockfile-generate-from-empty-file, -frozen — need a committed zero-byte file.
  Upstream touches it in before/after hooks precisely because an empty file is awkward
  to keep in a repo; parity/fixtures/** -text makes it a fixture whose bytes no reviewer
  can see. The runner has no "materialise this file empty" step and adding one is machinery
  growth to chase a fixture.
• lockfile-upgrade-feature — was NOT in the previous inventory (found by reading
  lockfile.test.ts, not by globbing configs). upgrade --feature X --target-version N
  rewrites devcontainer.json in place, preserving JSONC comments. A case would need an
  input document and an expected document plus an in-place-rewrite fixture pattern the
  harness does not have. Worth a maintainer decision on whether that pattern is wanted at
  all — it is the only surface in the suite where the CLI edits the user's config.

Still blocked on a maintainer ruling — 3

• compose project naming (compose-with-name, compose-with-name-and-custom-yaml,
  compose-with-name-using-env-var) — collides with the already-adjudicated
  deacon-namespaced project name. Unchanged: needs a ruling before it can land non-red.

Still environment-hostile — 25

Unchanged from the previous inventory, and none re-examined this batch:

• arch-pinned updateUID* (6): updateUIDarm64{,v8}{,-platform-option},
  updateUIDamd64{,-platform-option}
• private / cross-cloud registry auth probes (4): registry-compatibility/{azure-anonymous, azure-registry-scoped,github-anonymous,github-private}
• ghcr.io/codspace/* Feature graphs (7): feature-dependencies/dependsOn/oci-{ab,fgh,ij},
  feature-dependencies/dependsOn-and-installsAfter/a,
  feature-dependencies/overrideFeatureInstallOrder/{image-with-v1-features…, image-with-v2-features…,mixed}
• legacyIds renaming via codspace Features (2): example-legacyIds, example-legacyIds-2
• live-registry-404 resolution failures (2): invalid-configs/invalid-v{1,2}-features
• GitHub release tarball Features (2): feature-dependencies/dependsOn/tgz-ab,
  image-with-v2-tarball
• bare local id myfeature with no ./ prefix (1): image-with-local-feature
• host LOCAL_GID env var (1): updateUIDOnly

The 3/10 test above is worth re-running over the seven codspace Feature-graph configs.
Several of them claim an ORDERING or a dependency-resolution shape rather than a specific
publisher's content, and an ordering claim is exactly the kind that survives a repin — the
suite already has three local-Feature ordering cases (#537) proving the shape is
expressible. That is the most promising remaining vein and a plausible batch 3 alongside
lockfile-oci-integrity.

Running totals

The harvest denominator moves 91 → 92: lockfile-upgrade-feature was missed by the original
config-glob and found by reading lockfile.test.ts, which is a lesson in itself — the
upstream TEST FILE names configs the config-glob does not reach. Worth a pass over
src/test/**/*.test.ts before batch 3 rather than another glob.

92 harvested = 22 batch-1 + 3 batch-2 + 22 already-covered + 7 out-of-scope + 38
remaining, where the remaining 38 are: 10 lockfile deferrals, 3 blocked on a ruling, and
the 25 environment-hostile configs listed above.