Skip to content

Fix GroupNorm NHWC one-pass backward execution - #2018

Open
Aidyn-A wants to merge 4 commits into
NVIDIA:masterfrom
Aidyn-A:fix_group_norm_on_thor
Open

Fix GroupNorm NHWC one-pass backward execution#2018
Aidyn-A wants to merge 4 commits into
NVIDIA:masterfrom
Aidyn-A:fix_group_norm_on_thor

Conversation

@Aidyn-A

@Aidyn-A Aidyn-A commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Disclaimer: This PR was vibe-coded with Codex GPT v5.5 model.

Description

Fix GroupNorm NHWC one-pass backward execution on GPUs where the number of activation tiles exceeds the maximum number of cooperatively resident blocks (like Thor).

For example, the failing configuration requires 128 blocks per activation slice, while the target GPU supports only 120 resident blocks. This previously triggered:

Assertion `blocks_per_slice <= max_blocks_per_grid' failed.

Solution

The cooperative grid is capped at the device’s residency limit. When additional activation tiles remain, resident blocks process them using a grid-stride loop.

Overflow tiles participate in the same reductions and are completed within the same kernel launch, so the backward implementation remains strictly one-pass. There is no fallback to the
two-pass implementation.

Configurations whose grids already fit retain their existing processing path.

Performance impact

For affected configurations, some blocks process an additional activation tile and reload that tile’s input after the global reduction. In the failing case 128 → 120 case, this applies to 8 of
128 tiles, adding approximately 6.25% to the x/dy read traffic and introducing minor work imbalance.

Unaffected configurations only incur inexpensive loop-condition checks.

Validation

Tested locally on Thor machine.

cc @crcrpar

@crcrpar crcrpar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excuse me for the delay, I think this targets data center grade devices so a guard on the device prop could be better

@Aidyn-A

Aidyn-A commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

Whaaat did the bot just do? 🫨

@Aidyn-A
Aidyn-A force-pushed the fix_group_norm_on_thor branch from 9072875 to 661aee8 Compare August 4, 2026 05:23

@crcrpar crcrpar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently most of the changes are identical to #2022. I'm speculating merging this per-commit hook would let them go vanish

@Aidyn-A
Aidyn-A force-pushed the fix_group_norm_on_thor branch from 1b4bb39 to 661aee8 Compare August 5, 2026 05:49
@Aidyn-A
Aidyn-A requested a review from crcrpar August 7, 2026 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants