gh-152384: pixi-packages: use `flags` to define variants by lucascolley · Pull Request #152385 · python/cpython

lucascolley · 2026-06-27T10:56:40Z

cc @wolfv @baszalmstra @jaimergp I'm very excited about this PR!

Issue: pixi-packages: simplify to a single pixi.toml #152384
Issue: bump pixi-packages to 3.16 #152313

lucascolley · 2026-06-27T10:57:35Z

 - `freethreading`
 - `asan`: ASan-instrumented build
- `tsan-freethreading`: TSan-instrumented free-threading build
+- `tsan_freethreading`: TSan-instrumented free-threading build


I think it makes sense to remove the - from this variant name, given that it is invalid in the flags field: https://conda.org/learn/ceps/cep-0045#repodata-record-syntax

seems fine to me, I think you and @crusaderky are just about the only users of these builds so far for what its worth

lucascolley · 2026-06-27T10:59:04Z

    run_exports:
      noarch:
        - python
      weak:
        - python_abi ${{ version }}.* *_${{ abi_tag }}


we could also define flags on python_abi and perhaps not need to rely on abi_tag as much? But I suppose we still want to keep abi_tag around at least while conda-forge does?

EDIT: can probably leave this for a follow-up and open an issue

lucascolley · 2026-06-27T11:01:37Z

@StanFromIreland this goes some way towards making the version update situation better by reducing duplication in the variant definition

lucascolley · 2026-06-27T11:12:25Z

+openssl:
+  - '3.5'


ref. prefix-dev/pixi#6460 (comment) @h-vetinari, I wonder whether there is a better way to do this by pulling in (some of?) https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml

obviously you could download the current pinning file, yaml-load it and extract the openssl pin, but that's perhaps more complexity and dynamism than justified - an argument could be made that openssl is special enough to handle it separately... and although pins have a habit of going stale, staying on an LTS version probably makes sense here.

conda-forge will probably migrate to openssl 4 in the fall, but since AFAICT python has no other dependencies that are themselves openssl-dependent, this should not cause any conflicts (except in larger environments with the most recent builds, but that's not really relevant for the ASAN stuff).

obviously you could download the current pinning file, yaml-load it and extract the openssl pin, but that's perhaps more complexity and dynamism than justified

an idea was floated at some point that variant configs themselves could be packaged as conda packages and Pixi could consume them that way. Not sure if anyone has thought about that recently.

I mean, conda-forge-pinning is exactly that package. You'd still have to go parse the yaml though. 🤷

ngoldbaum · 2026-07-09T18:02:19Z

The CI failure looks unrelated but before this PR can be merged it's going to need a NEWS entry. Can you add one?

hugovk · 2026-07-10T11:39:45Z

I've updated this branch from main, which should pull in a skip for the failing unrelated macOS test.

lucascolley · 2026-07-10T13:40:04Z

news added

lucascolley · 2026-07-13T12:23:17Z

@ngoldbaum @kumaraditya303 any remaining concerns from your side? I think we can merge and patch up any more changes in a follow-up, this seems to be an improvement on the situation in main at least :)

Co-authored-by: Isuru Fernando <isuruf@gmail.com>

lucascolley · 2026-07-16T09:46:08Z

@ngoldbaum @hugovk I believe this is ready for a final look!

ngoldbaum

Other than the issue with the error message, LGTM!

Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>

kumaraditya303 · 2026-07-17T06:31:25Z

This looks good, I'll merge it now to unblock other stuff.

hugovk · 2026-07-17T06:44:49Z

How far do we want to backport this? Or just main for now?

I generally prefer to have less diverse config across branches, as it makes it harder to maintain.

rgommers · 2026-07-17T06:58:20Z

Either way works I think, it shouldn't break anything to backport. Both because it's still very new with only a few users, and because the main way of using it is to point at an exact commit hash in a downstream pixi.toml; in SciPy CI we're pointing at commits on main.

lucascolley · 2026-07-17T08:20:41Z

a backport to 3.15 would be nice, however https://github.com/python/cpython/blob/main/Tools/pixi-packages/variants.yaml#L1 will need a tweak

lucascolley · 2026-08-02T16:40:26Z

a backport to 3.15 would be nice, however main/Tools/pixi-packages/variants.yaml#L1 will need a tweak

hey @hugovk, any chance of a backport?

miss-islington-app · 2026-08-02T17:49:06Z

Thanks @lucascolley for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

bedevere-app · 2026-08-02T17:49:17Z

GH-155091 is a backport of this pull request to the 3.15 branch.

hugovk · 2026-08-02T17:51:41Z

Sure, how does #155091 look?

…52385) (#155091) * gh-152384: pixi-packages: use `flags` to define variants (GH-152385) (cherry picked from commit 2931f20) Co-authored-by: Lucas Colley <lucas.colley8@gmail.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Isuru Fernando <isuruf@gmail.com> Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com> * version: ["3.15"] --------- Co-authored-by: Lucas Colley <lucas.colley8@gmail.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Isuru Fernando <isuruf@gmail.com> Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>

bedevere-app Bot mentioned this pull request Jun 27, 2026

pixi-packages: simplify to a single pixi.toml #152384

Closed

bedevere-app Bot added the awaiting review label Jun 27, 2026

lucascolley commented Jun 27, 2026

View reviewed changes

lucascolley force-pushed the pixi-flags branch from 714d63f to 34cbfc3 Compare June 27, 2026 11:10

lucascolley commented Jun 27, 2026

View reviewed changes

pythongh-152384: pixi-packages: use flags to define variants

8b5b0c2

lucascolley force-pushed the pixi-flags branch from 34cbfc3 to 8b5b0c2 Compare June 27, 2026 11:16

lucascolley commented Jun 27, 2026

View reviewed changes

Comment thread Tools/pixi-packages/README.md Outdated

This was referenced Jun 27, 2026

ENH: pixi-packages: use skip-pyc-compilation for tsan-freethreading numpy/numpy#31558

Draft

ENH: pixi-packages: use flags numpy/numpy#31805

Merged

This was referenced Jul 4, 2026

BLD/DEV: ASAN at your fingertips! scipy/scipy#24066

Open

GH-49321: [C++][Python] Add ASAN / UBSAN pixi builds for Arrow and PyArrow apache/arrow#49849

Open

ngoldbaum added awaiting changes and removed awaiting review labels Jul 9, 2026

Merge branch 'main' into pixi-flags

81c8972

lucascolley added 2 commits July 10, 2026 14:31

tweaks

b226f35

add news

5cba45c

lucascolley requested a review from ngoldbaum July 10, 2026 13:39

ngoldbaum added awaiting core review and removed awaiting changes labels Jul 10, 2026

lucascolley commented Jul 12, 2026

View reviewed changes

Comment thread Tools/pixi-packages/variants.yaml

pin to clang 21

f152ba7

isuruf reviewed Jul 13, 2026

View reviewed changes

Comment thread Tools/pixi-packages/variants.yaml Outdated

remove abi_tag from variant

18c12e6

Co-authored-by: Isuru Fernando <isuruf@gmail.com>

isuruf reviewed Jul 13, 2026

View reviewed changes

Comment thread Tools/pixi-packages/recipe.yaml Outdated

lucascolley requested review from AA-Turner, JacobCoffee, ezio-melotti, hugovk, itamaro and webknjaz as code owners July 16, 2026 09:45

ngoldbaum reviewed Jul 16, 2026

View reviewed changes

Comment thread Tools/pixi-packages/build.sh Outdated

ngoldbaum approved these changes Jul 16, 2026

View reviewed changes

Update Tools/pixi-packages/build.sh

d72075b

Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>

isuruf reviewed Jul 16, 2026

View reviewed changes

Comment thread Tools/pixi-packages/build.sh Outdated

Update Tools/pixi-packages/build.sh

de788e8

kumaraditya303 approved these changes Jul 17, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels Jul 17, 2026

kumaraditya303 merged commit 2931f20 into python:main Jul 17, 2026
55 checks passed

bedevere-app Bot removed the awaiting merge label Jul 17, 2026

lucascolley deleted the pixi-flags branch July 17, 2026 08:19

hugovk added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Aug 2, 2026

bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Aug 2, 2026

hugovk mentioned this pull request Aug 2, 2026

[3.15] gh-152384: pixi-packages: use flags to define variants (GH-152385) #155091

Merged

		openssl:
		- '3.5'

Uh oh!

Uh oh!

Conversation

lucascolley commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucascolley Jun 27, 2026

Choose a reason for hiding this comment

Uh oh!

ngoldbaum Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

lucascolley Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucascolley commented Jun 27, 2026

Uh oh!

lucascolley Jun 27, 2026

Choose a reason for hiding this comment

Uh oh!

h-vetinari Jun 28, 2026

Choose a reason for hiding this comment

Uh oh!

lucascolley Jun 28, 2026

Choose a reason for hiding this comment

Uh oh!

h-vetinari Jun 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ngoldbaum commented Jul 9, 2026

Uh oh!

hugovk commented Jul 10, 2026

Uh oh!

lucascolley commented Jul 10, 2026

Uh oh!

Uh oh!

lucascolley commented Jul 13, 2026

Uh oh!

Uh oh!

Uh oh!

lucascolley commented Jul 16, 2026

Uh oh!

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kumaraditya303 commented Jul 17, 2026

Uh oh!

Uh oh!

hugovk commented Jul 17, 2026

Uh oh!

rgommers commented Jul 17, 2026

Uh oh!

lucascolley commented Jul 17, 2026

Uh oh!

lucascolley commented Aug 2, 2026

Uh oh!

miss-islington-app Bot commented Aug 2, 2026

Uh oh!

bedevere-app Bot commented Aug 2, 2026

Uh oh!

hugovk commented Aug 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

lucascolley commented Jun 27, 2026 •

edited

Loading

lucascolley Jun 27, 2026 •

edited

Loading