Skip to content

ci: retry the release builds' and compat matrix's external downloads - #4307

Merged
bpamiri merged 3 commits into
developfrom
peter/ci-curl-retries
Oct 4, 2026
Merged

bpamiri merged 3 commits into
developfrom
peter/ci-curl-retries

Conversation

@bpamiri

@bpamiri bpamiri commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

The snapshot build failed when GitHub answered one download with HTTP 429 (run 37235545301, "Building Linux packages"). build-linux-packages.sh fetched the LuCLI jar, and then the SQLite JDBC jar, with a bare curl -fsSL, so one rate-limited response ended the publish. The job's earlier nfpm fetch already retried.

Changes:

  • tools/distribution-drafts/linux-packages/build-linux-packages.sh: the LuCLI jar and SQLite JDBC downloads get --retry 3 --retry-delay 2 --retry-all-errors, like the nfpm fetch. The SHA-256 checks that follow each download are unchanged.

  • .github/workflows/release-candidate.yml: the CommandBox GPG key download gets the same flags. Its -fsSl becomes -fsSL: a lowercase l is --list-only, not "follow redirects".

  • .github/workflows/compat-pr.yml, compat-matrix.yml: the Oracle JDBC driver (ojdbc10.jar) was fetched with a bare wget -q. Oracle gates the matrix, so one rate-limited response turned a whole leg red. Both now use curl with the same retry flags.

Checked and left alone:

  • release.yml's dispatch calls: they're POSTs whose status is checked, and a blind retry could fire a dispatch twice.
  • The tools/ci/* and snapshot.yml requests: they go to the local test server, inside wait loops.
  • The README examples.

Verification:

  • The changed curl commands, run as written against the real URLs, download the LuCLI 0.6.2.2 jar and sqlite-jdbc 3.49.1.0 with SHA-256 matching the pins (tools/lucli.json and the script). The GPG key comes down as a PGP public key block. The new ojdbc10.jar fetch downloads the 4.5 MB jar containing oracle/jdbc/OracleDriver.class.
  • bash -n passes on the script, and all three workflow files parse.

🤖 Generated with Claude Code

bpamiri and others added 2 commits October 4, 2026 14:35
The snapshot build's Linux packaging fetched the LuCLI jar and the SQLite JDBC jar
with a bare curl, so one HTTP 429 from GitHub failed the whole publish. Both now
retry like the job's nfpm fetch (--retry 3 --retry-delay 2 --retry-all-errors), as
does the release-candidate workflow's CommandBox GPG key download, whose -fsSl also
becomes -fsSL (lowercase l is --list-only, not follow redirects).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
compat-pr.yml and compat-matrix.yml fetched ojdbc10.jar with a bare wget, and Oracle
gates the matrix, so one rate-limited response turned a whole leg red. Both now use
curl with the same retry flags as the release builds' downloads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bpamiri bpamiri changed the title ci: retry the release builds' external downloads ci: retry the release builds' and compat matrix's external downloads Oct 4, 2026
@bpamiri

bpamiri commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator Author

rev1-r3 review @ d286ad9: APPROVE

  • Retries. --retry 3 --retry-delay 2 --retry-all-errors on the lucli.jar and sqlite-jdbc.jar downloads in build-linux-packages.sh and on the CommandBox GPG key fetch in release-candidate.yml, matching the job's existing nfpm fetch. --retry-all-errors is what makes a 429 (or any HTTP error under -f) retryable. The jars are still checked against their pinned SHA-256s afterwards, so a bad retry can't get through.
  • Typo fix. -fsSl → -fsSL fixes a real bug: lower-case -l is --list-only, not follow-redirects.
  • Left alone, correctly. The release.yml dispatch POSTs (a retry could double-dispatch), the localhost probes and the README examples.

Nit, no change needed: the GPG key goes to stdout and is piped into gpg --dearmor. If a transfer fails after writing some bytes, curl's retry re-fetches into the same pipe, which can't be rewound, so gpg would see a mangled key. For a ~2 KB key that's very unlikely (a 429 arrives before any body), and it fails loudly in gpg rather than silently. curl … -o "$tmp" && sudo gpg --dearmor -o … "$tmp" would rule it out completely.

@bpamiri

bpamiri commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator Author

rev1-r3 review @ c92eaca: code APPROVE (final sign-off after matrix run 37236762366)

Delta d286ad92e..c92eacae5: the ojdbc10.jar fetch in compat-matrix.yml (the "Download ojdbc10 for Adobe engines" step) and compat-pr.yml changes from a bare wget -q to curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors -o …. -L keeps wget's redirect-following and -f keeps a non-zero exit on an HTTP error, so the only behaviour change is the retry. Since this edits compat-matrix.yml itself, the matrix on this head is the test of the change.

Remaining gate: on run 37236762366, the "Download ojdbc10 for Adobe engines" step succeeds on every Adobe job (the step only runs for adobe* engines, on every database), the Oracle legs pass, and every leg is green with tests > 0.

With --retry-all-errors on a piped download, a transfer retried mid-body could hand
gpg a partial or duplicated stream. The key now goes to $RUNNER_TEMP first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bpamiri

bpamiri commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator Author

rev1-r3 review @ 3648649: code APPROVE (final sign-off after matrix run 37236762366 on c92eacae5)

Delta c92eacae5..36486497c touches only .github/workflows/release-candidate.yml. The CommandBox key is now downloaded to $RUNNER_TEMP/ortussolutions.gpg with the same retry flags, and sudo gpg --dearmor -o /usr/share/keyrings/ortussolutions.gpg reads that file. A retried transfer can no longer hand gpg a partial or duplicated stream. No compat workflow changed, so matrix run 37236762366 on c92eacae5 still covers the ojdbc10.jar part.

Nit, no change needed: gpg --dearmor -o <existing file> prompts before overwriting. That's fine on a fresh runner, and the previous piped form behaved the same, but adding --batch --yes (as the apt/yum install docs do) would make it safe to re-run on a reused runner.

@bpamiri

bpamiri commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator Author

rev1-r3 review @ 3648649: APPROVE (final)

Matrix run 37236762366 ran on c92eacae5, the commit that changed compat-matrix.yml. The only commit after it (36486497c) touches release-candidate.yml alone, so this run is the test of the workflow change. Results:

  • The new curl download of ojdbc10.jar worked everywhere. The "Download ojdbc10 for Adobe engines" step succeeded on all twelve Adobe jobs (Adobe 2023 and 2025 × SQLite, MySQL, PostgreSQL, SQL Server, CockroachDB, Oracle).
  • All five Oracle legs pass (Lucee 6 and 7, Adobe 2023 and 2025, BoxLang): 6756 tests each, 0 failures, 0 errors.
  • All 32 engine × database legs: 0 failures, 0 errors, the smallest 6756 tests. 37 jobs succeeded, plus the skipped notifier.

With the code reviews at d286ad92e (the jar and key retries, -fsSl → -fsSL) and 36486497c (the key goes to a temp file before gpg --dearmor), this is good to merge.

@bpamiri
bpamiri merged commit e5060c9 into develop Oct 4, 2026
16 checks passed
@bpamiri
bpamiri deleted the peter/ci-curl-retries branch October 4, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant