Repository navigation
Move to SelectionCore follow-ups #15053
Description
Track the remaining router work from #14570 and #14310. This is a proposed work order, based on impact and what later changes depend on. Being worked on internally for now
| Rank | Cleanup | What to change and why |
|---|---|---|
| 1 | Routing configuration | Define defaults, validation and override rules in one place. Show which settings took effect. This affects every routing path and has already caused bugs. |
| 2 | Session-affinity rules | The session table is shared. Unify what happens when a worker disappears, requests compete to move a session, or the table fills up. |
| 3 | Worker membership and load accounting | Keep worker updates consistent. Preserve active-request counts when discovery arrives late, without making undiscovered workers available for routing. |
| 4 | KV-event recovery | Share the logic for detecting missing events, replaying them and rebuilding the index. Add recovery directly from workers for standalone indexing as an explicit behavior change. |
| 5 | Shared logic still in the frontend or EPP | Assess which LoRA metadata, external-cache clients and load calculations belong in the shared core. Keep host-specific information, such as whether the transport can reach a worker. |
| 6 | Prefill/decode coordination | Share the remaining coordination and retry rules. This needs a separate design. Adding disaggregated serving to standalone EPP is additional feature work. |
| 7 | Core packaging and API cleanup | Remove duplicate indexer handling and unnecessary wrappers. Separate the reusable core from standalone-server dependencies. Small cleanups can run alongside the other work. |
Start with configuration. Fix the smaller endpoint-check and background-task shutdown issues in parallel.
Track shared HTTP indexing and EPP soft affinity as separate features.
Activity
refactor(epp): share renderer HTTP handling