Skip to content

ci: stop Renovate from bumping the CI Python version - #7721

Closed
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:ci/renovate-python-cap
Closed

ci: stop Renovate from bumping the CI Python version#7721
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:ci/renovate-python-cap

Conversation

@aglinxinyuan

@aglinxinyuan aglinxinyuan commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

renovate.json5 already freezes numpy / pandas / pyarrow as Arrow IPC data-path pins, but not the interpreter their wheels are selected for. Renovate raised actions/setup-python's python-version: to 3.14 in #7715, and build / amber-integration failed on both runners at Install Python dependencies:

Before:  Renovate bumps python-version -> 3.14 -> no numpy cp314 wheel
         -> uv source-builds numpy -> 11 compile errors -> job fails
After:   python-version frozen with the numpy/pandas pins
         -> prebuilt wheel resolves -> job installs as before

The interpreter is downstream of the pins the rule above deliberately holds back:

Pin in amber/requirements.txt Wheels published cp314?
numpy==2.1.0 cp310–cp313 no
pandas==2.2.3 cp310–cp313 no
pyarrow==23.0.1 cp310–cp314 yes

So this adds one packageRules entry disabling the python dep under the github-actions manager, directly below the pins it is coupled to, with a comment recording why. The selector is matchDepNames, not matchPackageNames: Renovate names this dep python but resolves it from actions/python-versions, and matchPackageNames tests the latter. python is what #7715's own body table reports in its Package column.

Freezing rather than capping at <3.14 was deliberate: matching is by name, so the rule can be verified offline, whereas an allowedVersions range is compared against actions/python-versions strings like 3.14.7-31064857500 and a wrong range fails silently in either direction.

Three sites stop moving on their own — build.yml:401 (amber-integration, 3.11), build.yml:127 (frontend, 3.12), benchmarks.yml:217 (Bench, 3.12). The pyamber matrix at build.yml:1041 is ["3.11", "3.12", "3.13"], a plain list Renovate never read, so it has to be widened in the same manual step as the pins — the comment says so.

Not addressed here: build / frontend (macos-latest) is also red on #7715, but that reproduces on clean main and is the Vitest CI stall already tracked by #7623 (details below).

Any related issues, documentation, discussions?

Closes #7720

How was this PR tested?

Validated with Renovate's own validator, in repo-config mode (auto-discovery, run from the repo root):

npx --yes --package renovate renovate-config-validator
 INFO: Validating .github/renovate.json5
 INFO: Config validated successfully against 1 file(s)

Negative control, to confirm the validator actually inspects packageRules rather than rubber-stamping the file — temporarily renaming the matcher and adding a junk key:

ERROR: Found errors in configuration
       "errors": [
         { "message": "Invalid configuration option: packageRules[19].bogusOption" },
         { "message": "Invalid configuration option: packageRules[19].matchDepNamesTYPO" }
       ]

Wheel availability confirmed against PyPI, which is the fact the rule encodes:

curl -s https://pypi.org/pypi/numpy/2.1.0/json  | grep -o 'cp3[0-9][0-9]' | sort -u  # cp310 cp311 cp312 cp313
curl -s https://pypi.org/pypi/pandas/2.2.3/json | grep -o 'cp3[0-9][0-9]' | sort -u  # cp310 cp311 cp312 cp313

Matching the failing CI log from #7715, which source-builds numpy for cp314 and stops on the 3.14 headers:

Building numpy==2.1.0
  × Failed to build `numpy==2.1.0`
  ╰─▶ Call to `mesonpy.build_wheel` failed (exit status: 1)
      ../numpy/_core/src/umath/string_fastsearch.h:132:5: error: no type named
      'ptrdiff_t' in namespace 'std'
      11 errors generated.
      numpy/_core/_multiarray_umath.cpython-314-darwin.so.p/...

Separately, build / frontend (macos-latest) was checked and ruled out as unrelated: it fails the same way on a push to main at 7a2c8d03b5, which carries none of #7715's changes — same step, same Test timed out in 5000ms, ~332s run, a different randomly-hit spec each time. That is #7623, not this.

The rule itself only takes effect once Renovate next regenerates the github-actions branch, so #7715 needs a rebase after this merges to pick it up.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-opus-5)

Copilot AI lite review requested due to automatic review settings August 17, 2026 05:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added the ci changes related to CI label Aug 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @Yicong-Huang
    You can notify them by mentioning @Yicong-Huang in a comment.

The config already freezes numpy/pandas/pyarrow as Arrow IPC data-path
pins, but not the interpreter their wheels are selected for. Renovate
raised actions/setup-python to 3.14, and since numpy 2.1.0 and pandas
2.2.3 publish cp310-cp313 only, uv fell back to a source build of numpy
that fails to compile against the 3.14 headers.

Freeze the python uses-with dep so the interpreter moves by hand along
with those pins, and with the pyamber matrix Renovate cannot read.

Closes apache#7720
@aglinxinyuan
aglinxinyuan force-pushed the ci/renovate-python-cap branch from e229e06 to c614855 Compare August 17, 2026 05:34
@codecov-commenter

codecov-commenter commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.93%. Comparing base (1011ff7) to head (c614855).

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #7721   +/-   ##
=========================================
  Coverage     90.93%   90.93%           
  Complexity     4452     4452           
=========================================
  Files          1175     1175           
  Lines         47140    47140           
  Branches       5284     5284           
=========================================
  Hits          42869    42869           
  Misses         2581     2581           
  Partials       1690     1690           
Flag Coverage Δ *Carryforward flag
access-control-service 81.00% <ø> (ø)
agent-service 98.62% <ø> (ø)
amber 87.39% <ø> (ø) Carriedforward from 1011ff7
computing-unit-managing-service 73.67% <ø> (ø)
config-service 86.73% <ø> (ø)
file-service 68.90% <ø> (ø)
frontend 92.44% <ø> (ø) Carriedforward from 1011ff7
notebook-migration-service 83.74% <ø> (ø)
pyamber 97.57% <ø> (ø)
workflow-compiling-service 77.19% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 5 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 1011ff7 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 373 0.228 25,442/33,809/33,809 us 🔴 +10.2% / 🔴 +109.1%
🔴 bs=100 sw=10 sl=64 792 0.484 120,687/175,478/175,478 us 🔴 +12.0% / 🔴 +66.1%
bs=1000 sw=10 sl=64 949 0.579 1,053,232/1,093,556/1,093,556 us ⚪ within ±5% / 🔴 -10.2%
Baseline details

Latest main 1011ff7 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 373 tuples/sec 406 tuples/sec 784.16 tuples/sec -8.1% -52.4%
bs=10 sw=10 sl=64 MB/s 0.228 MB/s 0.248 MB/s 0.479 MB/s -8.1% -52.4%
bs=10 sw=10 sl=64 p50 25,442 us 23,091 us 12,626 us +10.2% +101.5%
bs=10 sw=10 sl=64 p95 33,809 us 35,776 us 16,169 us -5.5% +109.1%
bs=10 sw=10 sl=64 p99 33,809 us 35,776 us 18,986 us -5.5% +78.1%
bs=100 sw=10 sl=64 throughput 792 tuples/sec 825 tuples/sec 1,023 tuples/sec -4.0% -22.6%
bs=100 sw=10 sl=64 MB/s 0.484 MB/s 0.504 MB/s 0.625 MB/s -4.0% -22.5%
bs=100 sw=10 sl=64 p50 120,687 us 117,719 us 99,185 us +2.5% +21.7%
bs=100 sw=10 sl=64 p95 175,478 us 156,683 us 105,616 us +12.0% +66.1%
bs=100 sw=10 sl=64 p99 175,478 us 156,683 us 113,681 us +12.0% +54.4%
bs=1000 sw=10 sl=64 throughput 949 tuples/sec 949 tuples/sec 1,057 tuples/sec 0.0% -10.2%
bs=1000 sw=10 sl=64 MB/s 0.579 MB/s 0.579 MB/s 0.645 MB/s 0.0% -10.2%
bs=1000 sw=10 sl=64 p50 1,053,232 us 1,055,109 us 965,435 us -0.2% +9.1%
bs=1000 sw=10 sl=64 p95 1,093,556 us 1,103,290 us 1,012,135 us -0.9% +8.0%
bs=1000 sw=10 sl=64 p99 1,093,556 us 1,103,290 us 1,042,088 us -0.9% +4.9%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,535.74,200,128000,373,0.228,25441.68,33809.13,33809.13
1,100,10,64,20,2524.10,2000,1280000,792,0.484,120687.06,175478.49,175478.49
2,1000,10,64,20,21065.68,20000,12800000,949,0.579,1053231.64,1093556.19,1093556.19

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci changes related to CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Renovate bumps the CI Python past the numpy and pandas wheels it freezes

3 participants