Skip to content

DSv3 improvement by keeping the unreduced gradient tag under ZeRO-1 + explicit sharding, including through the MoE shard_map - #5056

Draft
NuojCheng wants to merge 2 commits into
mainfrom
chengnuojin-ds-ga
Draft

DSv3 improvement by keeping the unreduced gradient tag under ZeRO-1 + explicit sharding, including through the MoE shard_map#5056
NuojCheng wants to merge 2 commits into
mainfrom
chengnuojin-ds-ga

fix(ga): keep the unreduced tag through the MoE shard_map

a910227
Select commit
Loading
Failed to load commit list.
Google CLA / cla/google succeeded Sep 23, 2026 in 8s

✅ All contributors are covered under a CLA with Google

See https://cla.developers.google.com/ for more info about Google's Contributor License Agreement (CLA).

ℹ️ Googlers: Go here to view more details and manage scans for this pull request.

Details

The following contributors were found for this pull request:

✅ a910227 Author: @NuojCheng <che******in​@google.com>

(Only the first commit for a unique contributor is listed.)

View more details on Google CLA