Skip to content

[CELEBORN-2423] Fix rack-aware replica storage selection - #3803

Open
Kalvin2077 wants to merge 2 commits into
apache:mainfrom
Kalvin2077:fix/CELEBORN-2423
Open

[CELEBORN-2423] Fix rack-aware replica storage selection#3803
Kalvin2077 wants to merge 2 commits into
apache:mainfrom
Kalvin2077:fix/CELEBORN-2423

Conversation

@Kalvin2077

@Kalvin2077 Kalvin2077 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This follow-up to PR #3781:

  • Validates that rack-aware replica candidates support the requested storage type.
  • Builds replica StorageInfo from the selected replica worker.
  • Adds regression tests for diskless replicas and different mount points.

Why are the changes needed?

During best-effort fallback, the allocator could select a diskless worker for a local-disk replica or incorrectly reuse the primary worker's mount point.

Does this PR resolve a correctness bug?

  • Yes

Does this PR introduce any user-facing change?

  • Yes

How was this patch tested?

build/mvn -pl master -am -Dtest=SlotsAllocatorRackAwareSuiteJ test

@Kalvin2077 Kalvin2077 changed the title [CELEBORN-2423] Refactor slot assignment strategy and fix storage selection [CELEBORN-2423] Fix rack-aware replica storage selection Aug 13, 2026
@Kalvin2077
Kalvin2077 marked this pull request as ready for review August 21, 2026 08:10
@github-actions github-actions Bot added the correctness Correctness bugfix label Aug 21, 2026
@Kalvin2077

Copy link
Copy Markdown
Contributor Author

@zaynt4606 @SteNicholas
PTAL, thx.

@yew1eb

yew1eb commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

@Kalvin2077 LGTM overall. One nit on test coverage: offerSlotsRackAwareFallbackRequiresReplicaDisk only covers the case where no eligible replica exists. Could we add a case where the replica rack has both a diskless and a local-disk worker, and assert the allocator skips the diskless on and still allocates? That's the path the new canAssign predicate actually guards — currently it would regress unnoticed.

@Kalvin2077

Copy link
Copy Markdown
Contributor Author

@Kalvin2077 LGTM overall. One nit on test coverage: offerSlotsRackAwareFallbackRequiresReplicaDisk only covers the case where no eligible replica exists. Could we add a case where the replica rack has both a diskless and a local-disk worker, and assert the allocator skips the diskless on and still allocates? That's the path the new canAssign predicate actually guards — currently it would regress unnoticed.

@yew1eb
Thx.

I've added offerSlotsRackAwareFallbackSkipsDisklessReplica to cover this path.

It uses two partitions so the diskless candidate is scanned regardless of the randomized initial index, and verifies both replicas are allocated to the eligible disk-backed worker. Removing the canAssign guard causes the test to fail.

PTAL.

AI-Contributed/Feature: 0/0
AI-Contributed/UT: 0/26
@codecov

codecov Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.59%. Comparing base (90954f7) to head (55ecb31).
⚠️ Report is 24 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3803      +/-   ##
============================================
- Coverage     58.74%   58.59%   -0.14%     
  Complexity      231      231              
============================================
  Files           399      399              
  Lines         28058    28065       +7     
  Branches       2740     2745       +5     
============================================
- Hits          16479    16443      -36     
- Misses        10380    10418      +38     
- Partials       1199     1204       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants