Skip to content

ci(refactor-release): cut tag releases over to GoReleaser - #12828

Draft
DariuszPorowski wants to merge 5 commits into
dp/prepare-release-backportsfrom
dp/goreleaser-tag-cutover
Draft

ci(refactor-release): cut tag releases over to GoReleaser#12828
DariuszPorowski wants to merge 5 commits into
dp/prepare-release-backportsfrom
dp/goreleaser-tag-cutover

Conversation

@DariuszPorowski

@DariuszPorowski DariuszPorowski commented Aug 26, 2026

Copy link
Copy Markdown
Member

Summary

Cut tag-triggered releases over to GoReleaser for CLI binaries, checksums, production Go images, and draft GitHub Release staging. Keep Helm, Bicep types, and retained non-Go/test images as gated stages, then promote mutable channel and latest aliases only from verified immutable outputs after every mandatory stage succeeds.

Make the release transaction resumable with source-bound draft locks, query-before-mutate reconciliation, bounded retries, complete-set image adoption, and content verification for existing CLI and Helm artifacts. Main builds now publish edge only.

This stacked PR depends on #12823.

Draft gate: Phase 7 has not yet observed the required full RC-plus-final cycle. Keep this PR in draft until that cycle completes with no unexplained parity differences. The first production GoReleaser release and installation verification remain tracked in #12819.

  • Observe one full RC-plus-final cycle with no unexplained parity differences.

Reason for change

The shadow phase established parity, but the production tag path still uses separate CLI, image, checksum, and release jobs with mutable recovery state. This cutover makes immutable full-version outputs the recovery source of truth and delays mutable aliases and release finalization until the complete publication set is verified.

Fixes #12819

How to test

  • make test-capture-release-image-digests test-release-cutover test-release-oci-artifacts test-release-parity-manifest test-prepare-release
  • make goreleaser-snapshot (built and verified all 22 CLI targets)
  • Validate workflows with actionlint.
  • Validate shell with ShellCheck and shfmt.
  • Run Markdown, 80-column, and whitespace checks.

Docker was unavailable locally, so container registry paths are covered by deterministic Docker fixtures. ORAS publication and the GoReleaser snapshot were exercised with the real tools.

File change summary

File Summary of change
.github/workflows/build-release.yaml Replace legacy tag publication with GoReleaser staging, retained-image and Helm/Bicep gates, digest finalization, alias promotion, and published-state reconciliation.
.github/workflows/__build-{cli,helm-chart,images}.yaml Align reusable build outputs and image behavior with the cutover contract.
.github/scripts/{ensure-draft-release,publish-draft-release,release-assets}.mjs Reconcile source-bound draft releases, lock assets, and final publication through retrying GitHub API operations.
.github/scripts/{release-oci-artifacts,publish-helm-chart,normalize-release-checksums,capture-release-image-digests}.sh Publish and verify immutable OCI/Helm artifacts, normalize split checksums, capture complete image sets, and promote aliases safely.
.github/scripts/*_test.{sh,mjs} Cover reruns, partial/conflicting state, source mismatches, retries, registry ambiguity, checksum normalization, Helm reuse, and workflow contracts.
.github/scripts/{verify-goreleaser-shadow,verify-goreleaser-shadow_test}.sh Remove the superseded shadow-release verifier.
.github/workflows/__publish-release.yaml Remove the superseded legacy GitHub Release publisher.
.goreleaser.yaml Keep GoReleaser releases draft and prevent automatic latest mutation.
build/{artifacts,docker,test}.mk Add cutover publication/test targets and remove single-use legacy targets.
deploy/Chart/README.md Align Helm publication guidance with immutable release staging.
docs/contributing/contributing-releases/README.md Document the GoReleaser-owned tag path, resumability model, and alias timing.
eng/design-notes/tools/2026-09-goreleaser-stack-review/pr-13-goreleaser-tag-cutover.md Review note for this layer: what changed in review, why, the value, and the impact

@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

  • .github/workflows/__publish-release.yaml

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown

Unit Tests

    2 files  ±0    460 suites  ±0   8m 50s ⏱️ +13s
6 573 tests ±0  6 571 ✅ ±0  2 💤 ±0  0 ❌ ±0 
7 874 runs  ±0  7 872 ✅ ±0  2 💤 ±0  0 ❌ ±0 

Results for commit 03b039a. ± Comparison against base commit f72357b.

♻️ This comment has been updated with latest results.

@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from e54a4c5 to a8f7de9 Compare August 28, 2026 16:54
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.77%. Comparing base (f72357b) to head (03b039a).

Additional details and impacted files
@@                      Coverage Diff                      @@
##           dp/prepare-release-backports   #12828   +/-   ##
=============================================================
  Coverage                         59.77%   59.77%           
=============================================================
  Files                               778      778           
  Lines                             45916    45916           
=============================================================
  Hits                              27446    27446           
  Misses                            18470    18470           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown

Functional Tests - upgrade-noncloud

1 tests  ±0   0 ✅ ±0   5m 5s ⏱️ ±0s
1 suites ±0   0 💤 ±0 
1 files   ±0   1 ❌ ±0 

For more details on these failures, see this check.

Results for commit 03b039a. ± Comparison against base commit f72357b.

♻️ This comment has been updated with latest results.

@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from a8f7de9 to 13409c7 Compare August 28, 2026 20:07
@DariuszPorowski DariuszPorowski changed the title ci: cut tag releases over to GoReleaser ci(refactor-release): cut tag releases over to GoReleaser Aug 28, 2026
@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from 13409c7 to cdeb44f Compare September 3, 2026 16:22
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Functional Tests - corerp-noncloud

190 tests   188 ✅  1h 12m 8s ⏱️
  3 suites    2 💤
  1 files      0 ❌

Results for commit 158a1a4.

♻️ This comment has been updated with latest results.

@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from cdeb44f to 158a1a4 Compare September 9, 2026 17:41
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Functional Tests - kubernetes-noncloud

23 tests  ±0   22 ✅  - 1   17m 45s ⏱️ + 9m 3s
 1 suites ±0    0 💤 ±0 
 1 files   ±0    1 ❌ +1 

For more details on these failures, see this check.

Results for commit 03b039a. ± Comparison against base commit f72357b.

♻️ This comment has been updated with latest results.

@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from 158a1a4 to 81376b7 Compare September 9, 2026 20:47
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Functional Tests - daprrp-noncloud

26 tests  ±0   26 ✅ ±0   12m 33s ⏱️ - 1m 21s
 1 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 7ec933b. ± Comparison against base commit c62309f.

♻️ This comment has been updated with latest results.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Mixed registry failures can be misclassified as confirmed absence, allowing an immutable image tag to be overwritten.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Migrates tag releases to a resumable, digest-verified GoReleaser pipeline, while main builds publish only edge.

Changes:

  • Adds immutable release staging, locking, verification, and alias promotion.
  • Adds reconciliation scripts and comprehensive release tests.
  • Removes legacy release and shadow-parity paths; updates documentation.
File summaries
File Description
.cspellignore Allows new terminology.
.github/scripts/capture-release-image-digests.sh Captures and verifies image digests.
.github/scripts/capture-release-image-digests_test.sh Tests digest capture and recovery states.
.github/scripts/ensure-draft-release.mjs Reconciles draft releases.
.github/scripts/ensure-draft-release_test.mjs Tests draft reconciliation.
.github/scripts/image-payload-manifest/main.go Removes obsolete parity helper.
.github/scripts/normalize-release-checksums.sh Normalizes CLI checksums.
.github/scripts/normalize-release-checksums_test.sh Tests checksum normalization.
.github/scripts/prepare-release_test.sh Updates prepared-notes assertions.
.github/scripts/publish-draft-release.mjs Publishes drafts and selects aliases.
.github/scripts/publish-draft-release_test.mjs Tests publication and alias policy.
.github/scripts/publish-helm-chart.sh Reconciles immutable Helm publication.
.github/scripts/publish-helm-chart_test.sh Tests Helm publication recovery.
.github/scripts/release-assets.mjs Manages and verifies release assets.
.github/scripts/release-assets_test.mjs Tests asset reconciliation.
.github/scripts/release-cutover_test.sh Validates cutover workflow contracts.
.github/scripts/release-oci-artifacts.sh Stages and promotes OCI artifacts.
.github/scripts/release-oci-artifacts_test.sh Tests OCI staging and promotion.
.github/scripts/release-parity-manifest.sh Permits release lock assets.
.github/scripts/release-parity-manifest_test.sh Tests lock-asset handling.
.github/scripts/verify-goreleaser-shadow.sh Removes shadow verifier.
.github/scripts/verify-goreleaser-shadow_test.sh Removes shadow-verifier tests.
.github/scripts/verify-goreleaser-snapshot.sh Verifies final release configuration.
.github/workflows/__build-cli.yaml Publishes CLI edge artifacts.
.github/workflows/__build-helm-chart.yaml Uses resumable Helm publishing.
.github/workflows/__build-images.yaml Publishes images only as edge.
.github/workflows/__publish-release.yaml Removes legacy release publisher.
.github/workflows/build-release.yaml Implements the new release transaction.
.goreleaser.yaml Keeps releases draft and avoids latest.
build/artifacts.mk Adds release staging and promotion targets.
build/docker.mk Removes legacy image retagging.
build/test.mk Registers cutover tests.
deploy/Chart/README.md Documents stable latest semantics.
docs/contributing/contributing-releases/README.md Documents the new release workflow.
Review details
  • Files reviewed: 34/34 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

# A single missing read is never trusted: the registry must report the
# tag missing on every attempt before it counts as absent.
case "${error,,}" in
*"manifest unknown"* | *"not found"* | *404*) missing=true ;;
@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch 2 times, most recently from 7ec933b to e5db9b5 Compare September 10, 2026 20:32
@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch 2 times, most recently from 0bd875c to e8540e4 Compare September 10, 2026 20:56
@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from e8540e4 to cbb56e8 Compare September 10, 2026 21:41
@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch 2 times, most recently from 0df90df to fc9fe6f Compare September 10, 2026 21:57
@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from fc9fe6f to e9aba62 Compare September 10, 2026 22:26
@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from e9aba62 to 0f3bdcc Compare September 10, 2026 22:46
Replace the tag-path CLI matrix, production image publisher, checksum
loop, and GitHub Release job with one GoReleaser staging job. Keep Helm,
Bicep types, and retained non-Go/test images as separate gated stages.

Publish full-version images and CLI OCI artifacts first, lock their
digests and source commit on the draft release, and promote channel and
latest aliases only for stable releases after all mandatory stages pass.
Main builds now publish edge only.

Make every stage resumable: reconcile source-bound draft releases,
content-verify existing CLI and Helm artifacts, adopt only complete image
sets with matching OCI source labels, retry transient registry and API
operations, and reject partial or conflicting immutable state.

Remove the superseded shadow comparison and legacy release publisher.

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
capture-release-image-digests.sh rejected any partially published image
set, so an interrupted GoReleaser push left the release stuck: every
rerun failed on the same partial set, and the unclaimed-tag guard refused
to proceed as well. Recovery meant deleting tags from GHCR by hand, which
is exactly the half-published release this migration set out to remove.

A partial set carries no durable lock, so nothing can have resolved those
tags yet and staging may still overwrite them. Report absent, partial,
and complete as distinct states, keep the strict all-or-nothing check for
the post-staging capture that writes the lock, and run the unclaimed-tag
assertion only when nothing exists yet.

Require --state-output whenever --allow-absent is set, so tolerated
absence is always reported to a caller that acts on it.

Name absence confirmation separately from transient failure in the retry
log, since a first release confirms every tag missing five times and read
as a stack of failures. Make the tag-disappeared diagnostic reachable by
propagating the not-found status out of cli_artifact_matches, and stage
CLI artifacts through a subshell function so the directory change stays
contained and the formatter stops fighting trailing padding.

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
@DariuszPorowski
DariuszPorowski force-pushed the dp/goreleaser-tag-cutover branch from 0f3bdcc to 7ddfbef Compare September 11, 2026 04:01
@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown

Functional Tests - corerp-cloud

32 tests  ±0   31 ✅ ±0   19m 24s ⏱️ - 2m 26s
 2 suites ±0    1 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 03b039a. ± Comparison against base commit f72357b.

♻️ This comment has been updated with latest results.

… images

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
@radius-functional-tests

radius-functional-tests Bot commented Sep 11, 2026

Copy link
Copy Markdown

Radius functional test overview

🔍 Go to test action run

Click here to see the test run details
Name Value
Repository radius-project/radius
Commit ref 03b039a
Unique ID funcd5d817f52c
Image tag pr-funcd5d817f52c
  • Dapr: 1.14.4
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcd5d817f52c
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcd5d817f52c
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcd5d817f52c
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcd5d817f52c
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcd5d817f52c
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cut tag builds over to GoReleaser

2 participants