Skip to content

fix(gms): verify persistent KV installation at startup - #15036

Draft
hutm wants to merge 1 commit into
hutm/gms-common-16-v1-persistent-poolfrom
hutm/gms-hardening-01-startup-verification
Draft

hutm wants to merge 1 commit into
hutm/gms-common-16-v1-persistent-poolfrom
hutm/gms-hardening-01-startup-verification

Conversation

@hutm

@hutm hutm commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Verify at worker startup that the requested persistent-KV integration was actually installed for vLLM or SGLang. Incomplete or missing hooks now fail closed instead of silently serving with ordinary process-owned KV.

Why this is needed

Configuration alone is not evidence that cache allocation and recovery hooks are active. A silent partial install would make the endpoint look healthy while providing no crash-surviving KV guarantee.

Validation

  • The cumulative publication tip is byte-identical to the locally validated implementation.
  • Cumulative focused validation: 994 passed, 1 skipped; the later vLLM-focused rerun passed 244 tests.
  • Local TP2 crash tests passed exact-output recovery for vLLM and SGLang; final performance qualification remains tracked in DEP: Production hardening for persistent GMS KV failover #15035.
  • This draft checkpoint still requires normal CI and reviewer validation before merge.

Stack

This PR is intentionally draft and stacked on its immediate predecessor.

Make engine startup validate that persistent KV hooks were actually installed. Fail closed when the requested integration is missing or incomplete instead of starting with non-persistent cache semantics.

Signed-off-by: mkhadkevich <mkhadkevich@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 18, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant