Skip to content

shim: don't tear down shared pod UVM on Hyper-V container restart - #2846

Open
Dawei Wei (rzlink) wants to merge 1 commit into
microsoft:mainfrom
rzlink:fix-hyperv-shared-uvm-restart
Open

shim: don't tear down shared pod UVM on Hyper-V container restart#2846
Dawei Wei (rzlink) wants to merge 1 commit into
microsoft:mainfrom
rzlink:fix-hyperv-shared-uvm-restart

Conversation

@rzlink

Copy link
Copy Markdown
Contributor

Description

Fix in-place container restarts for Hyper-V-isolated Windows pods.

Two issues could combine to destroy the shared pod UVM:

  1. When SignalProcess returned hrNotFound while WaitForProcess remained pending, process shutdown could block indefinitely. Force-complete the pending wait and tolerate its possible late response.
  2. The SIGKILL watchdog could directly close the UVM from a workload task that did not own it. Gate the watchdog on ownsHost and use closeHost for teardown.

This prevents a workload container restart from terminating the sandbox and sibling containers sharing the UVM.

Testing

  • go test ./internal/gcs

@rzlink
Dawei Wei (rzlink) requested a review from a team as a code owner July 30, 2026 18:14
@rzlink
Dawei Wei (rzlink) force-pushed the fix-hyperv-shared-uvm-restart branch from 65fd990 to 808e988 Compare July 31, 2026 23:59
@helsaawy

Copy link
Copy Markdown
Contributor

Dawei Wei (@rzlink)
what is the context for restarting containers? that is not exactly a supported OCI/containerd scenario?
also, are you seeing that exec signal/kill takes longer than 30seconds?

@rzlink

Copy link
Copy Markdown
Contributor Author

Dawei Wei (@rzlink) what is the context for restarting containers? that is not exactly a supported OCI/containerd scenario? also, are you seeing that exec signal/kill takes longer than 30seconds?

Thanks. This is for Kubernetes RestartAllContainersOnContainerExits, tracked in kubernetes/kubernetes#140187.
Kubelet implements the in-place restart using standard CRI stop/remove/create/start operations while preserving the pod sandbox. The process-isolated Windows tests pass; only Hyper-V isolation fails.
The signal call itself does not take 30 seconds. It returns  hrNotFound  because the process is already gone, but the outstanding WaitForProcess remains pending. Consequently,  e.Wait()  never completes and the existing 30-second watchdog fires.
For a workload task, ht.host is the shared pod UVM and ht.ownsHost is false. The watchdog currently closes that UVM, terminating the sandbox and sibling containers. This change completes the stale wait and prevents non-owning workload tasks from closing the shared UVM.

On Hyper-V-isolated Windows pods (runhcs-wcow-hypervisor), in-place container
restarts (ContainerRestartRules / RestartAllContainersOnContainerExits) fail:
when a workload container exits it is not restarted in place; the pod ends up
Failed. Two hcsshim/GCS issues combine to destroy the shared pod UVM.

1. Under RestartAll churn the guest GCS delivers a container's WaitForProcess
   exit reply late (or drops it); a crossing SIGKILL then returns hrNotFound
   while the wait is still pending. Process.Signal only logged "ignoring
   missing process", so the wait never completed and container Stop blocked
   forever. Force-complete the pending wait on hrNotFound so Wait()/Stop can't
   hang, and make an unmatched late response in recvLoop non-fatal (log and
   continue) rather than tearing the bridge down.

2. The KillExec 30s SIGKILL-init watchdog closed ht.host directly with no
   ownsHost guard, so a non-owning workload container's stuck stop tore down
   the shared pod UVM, killing the sandbox and its siblings. Gate the watchdog
   on ht.ownsHost and route teardown through closeHost so only the UVM owner
   may close it.

Process-isolated pods are unaffected (host HCS exit path, no shared UVM).

Adds unit tests for bridge.forceComplete and the non-fatal unknown-rpc path.

Signed-off-by: Dawei Wei <wei.dawei.cn@gmail.com>
@rzlink
Dawei Wei (rzlink) force-pushed the fix-hyperv-shared-uvm-restart branch from 808e988 to 0125160 Compare August 3, 2026 22:27
// timer and wait on either the timer expiring or the process exiting
// cleanly. If the timer expires first we forcibly close the UVM as we
// assume the guest is misbehaving for some reason.
if signal == 0x9 && eid == "" && ht.host != nil && ht.ownsHost {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the forceComplete call should unblock execExited and render this change unneeded, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants