Missing target feature ABI checks for nvptx

@kulst

The Target::abi_required_features and Target::check_consistency ABI-checking logic currently has no handling for NVPTX.

For context: on other targets this logic ensures things like

valid values for llvm_abiname/llvm_floatabi
consistent values for cfg_abi relative to llvm_abiname
preventing target features from being enabled/disabled in a way that would alter the ABI

This is needed to prevent issues like #116344.

To resolve this, we need to figure out if there are target features that would alter the ABI, and pin those down in abi_required_features to avoid ABI issues. This target is a bit special since one has to pick a consistent -Ctarget-cpu value across all compilation units that are linked with each other. There is work on the way to enforce this. However, even with that in place, we need to worry about people using -Ctarget-feature in a way that would produce incorrect code.

From prior discussion in #131799 and #141468, it seems like the general constraint is that

A binary will end up with one exact ptx and sm version.

That would seem to indicate that we have to mark all of these target features as "forbidden"? Then only -Ctarget-cpu would remain usable and will be locked down across the crate graph. However I don't actually know the target so this may make no sense.
Cc @kulst @ZuseZ4

Ah this was also discussed in #141468

After looking into It seems like the most reasonable thing to do is add an unstable compiler flag like -Zptx-isa-version= that is a target modifier. This flag should get a tracking issue of it's own and in general follow the same track as the already added target flag -Zfixed-x18. The big drawback is of course that this will be yet another target specific compiler flag, on the other hand it will be unstable so it can be moved in case special infrastructure for target specific flags that can be used as target modifier are created.

So sounds like configuring the PTX version is still needed separately from -Ctarget-cpu? Bummer.

One thing we should figure out is how to #[cfg(…)] the ptx.. and sm_.. features then. Currently target features activated by target modifiers are marked Forbidden and cannot be used in #[cfg(…)].

FWIW we now have a concept of "target features that you cannot set but that you can query via cfg" that we could use here.

@kulst

@kulst with #150732 having landed, what remaining checks are we missing? Currently one can still arbitrarily mix "sm" and "ptx" target features in a single crate graph, though we force them all to have the same target-cpu (if I understood correctly). However the sm and ptx features also all imply the lower version. So I would assume it is fine to mix "sm_87" in some creates and "sm_120" in another, the result then just needs "sm_120" to really run? And same for e.g. mixing "ptx74" and "ptx86"?

Looking at the LLVM backend, I can see hasF32x2Instructions() affecting whether some vector registers are available, which could affect the ABI. That should usually be handled by FEATURES_FOR_CORRECT_FIXED_LENGTH_VECTOR_ABI but NVPTX does not have anything there, just a comment saying "no vector ABI". So vectors are never passed in registers?

@kulst with #150732 having landed, what remaining checks are we missing?

My plan is to file an MCP in the next few weeks to make the PTX version another target modifier.
When this has landed, my plan would be to deprecate the corresponding Rust target features (making them Forbidden).
I want -Ctarget-cpu (or in the future -Ttarget-cpu) be the single source of truth for the SM version and the PTX target modifier the single source of truth for the PTX version.

This would reflect the requirements of the final PTX artifact and would also be more in line with LLVM's handling of the PTX and SM version. For example, the way in which LLVM handles the SM version differs depending on whether it is specified via -mcpu or -mattr. This inconsistency can lead to miscompiles.

It would be great to have #152038 landed before. This would preserve the ability to use the PTX and SM versions in conditional compilation after making the Rust target features Forbidden.¹

Technically, we could also introduce a CfgStableToggleForbidden and a CfgUnstableToggleForbidden variant in https://github.com/rust-lang/rust/blob/b070f45ad92ed45b20e57d9483a21657d0d00715/compiler/rustc_target/src/target_features.rs#L54-L85
While this would allow #[cfg(target_feature = "ptxyy")] and #[cfg(target_feature = "sm_xx")], it would feel odd to me since these features cannot be enabled using the identically named codegen flag. ↩

So -Ctarget-cpu can set both the SM and PTX version?
Or are you imagining a -Zptx-isa-version as has been mentioned elsewhere?

While this would allow #[cfg(target_feature = "ptxyy")] and #[cfg(target_feature = "sm_xx")]

I don't think we should use cfg(target_feature) for something that we don't actually consider a target feature.
In #160530 I am renaming Forbidden to InternalOnly to better reflect what these are -- we track them as target features internally in rustc but to the user we don't present them as such, so they don't interact with -Ctarget-feature or #[cfg(target_feature)].
In #160593 I am adding the ability for internal target features to be target modifiers so that we are sure that they are consistent across the crate graph.

Okay so we want to ensure PTX and SM are the same across the crate graph. I'm not sure why we also have implications then as that seems to me to indicate some sort of compatibility... if we have a cfg for -Zptx-isa-version then that would not have implications, you could only check the exact value. And target-cpu is also not something one can check in cfg.

What are motivating examples for having cfg to check the PTX or SM version?

are you imagining a -Zptx-isa-version as has been mentioned elsewhere?

Exactly, I am talking about a -Zptx-isa-version.
So you could set the sm version via -Ctarget-cpu=sm_100 and the ptx version via -Zptx-isa-version=ptx88 and not via target features.

What are motivating examples for having cfg to check the PTX or SM version?

The availability of instructions depends on the SM version and the PTX version. So this is valuable, if one wants to write code optimized for example for a specific SM version or greater than a specific SM and or PTX version.

Do they need to check "does the current version equal X" or also "is the current version at least / at most X"?

Compute capabilities form a compatibility DAG. For example:

sm_90   ⊆  sm_100   ⊆  sm_103   ⊆  sm_110   ⊆  sm_120   ⊆  sm_121
  |∩         |∩          |∩          |∩          |∩          |∩
sm_90a     sm_100f  ⊆  sm_103f     sm_110f     sm_120f  ⊆  sm_121f
             |∩          |∩          |∩          |∩          |∩
           sm_100a     sm_103a     sm_110a     sm_120a     sm_121a

Therefore, a hypothetical #[cfg("sm_110f")] could indicate that the compute capability selected by -Ctarget-cpu is a superset of sm_110f. #[cfg(not("sm_110f"))] would indicate that it is not. Something like this might be helpful.

In terms of PTX versions it is less clear to me if this is correct. Recent PTX versions appear to be mostly monotonic (... ⊆ ptx76 ⊆ ptx77 ⊆ ...) in terms of available instructions and features, but I could not find a documented guarantee that every newer PTX version is a superset of every older one.

It is odd to me that the capabilities form a DAG and yet they are all mutually incompatible with each other... how can that be?^^
If I use sm100 in some part and sm120 in the rest, why isn't the entire thing then in sm120 seeing as sm100 is a subset of sm120?

It is odd to me that the capabilities form a DAG and yet they are all mutually incompatible with each other... how can that be?^^

Probably we are overly cautious at the moment. If crate A depends on B, maybe we could allow A to be compiled with a SM version that is ⊇to the SM version B was compiled with. I need to think it through but adhoc I do not see a problem with this approach.

That would be hard to implement.
But having each crate at some arbitrary SM version, and then saying the biggest SM version is actually required, would work... except that assumes a lattice and this DAG does not seem to form a lattice. So maybe this won't work.

An alternative to a new -Z target modifier flag might be to add a target modifier version of "target-features".
The plan is to make it so -T flags are target modifiers while -C/-Z flags are not. Some flags can exist in both, like target-cpu. We could also allow target-feature in both, and then require that some target features, like all the nvptx ones, only ever appear in -T.

FWIW this is now the only tier 2 target where we haven't resolved the target feature ABI situation yet. :)

But progress here is blocked on #160167 I think which redoes target modifiers.

Navigation Menu

Repository navigation

Missing target feature ABI checks for nvptx #157532

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Missing target feature ABI checks for nvptx #157532

Description

Activity

RalfJung commented on Jun 6, 2026

RalfJung commented on Aug 6, 2026

RalfJung commented on Aug 6, 2026

kulst commented on Aug 6, 2026

Footnotes

RalfJung commented on Aug 6, 2026

RalfJung commented on Aug 6, 2026

kulst commented on Aug 6, 2026

RalfJung commented on Aug 6, 2026

kulst commented on Aug 6, 2026

RalfJung commented on Aug 6, 2026

kulst commented on Aug 6, 2026

RalfJung commented on Aug 6, 2026

RalfJung commented on Aug 7, 2026

RalfJung commented on Sep 3, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions