Skip to content

gcs-sidecar, resource_wcow: Don't use UVM exec to create sandbox mount sources - #2806

Open
Tingmao Wang (micromaomao) wants to merge 3 commits into
microsoft:mainfrom
micromaomao:wcow-mount-mkdir
Open

gcs-sidecar, resource_wcow: Don't use UVM exec to create sandbox mount sources#2806
Tingmao Wang (micromaomao) wants to merge 3 commits into
microsoft:mainfrom
micromaomao:wcow-mount-mkdir

Conversation

@micromaomao

@micromaomao Tingmao Wang (micromaomao) commented Jul 3, 2026

Copy link
Copy Markdown
Member

gcs-sidecar, resource_wcow: Don't use UVM exec to create sandbox mount sources

In confidential WCOW this will not work due to the policy not allowing exec, and
we don't want to add an exception for this. Instead, we make the gcs-sidecar
handle this job, and make the host skip this for confidential containers.

To test, use a container json that has a mount:

{
  "metadata": {
    "name": "wcow_emptydir_info_cwcow_primary"
  },
  "image": {
    "image": "mcr.microsoft.com/windows/servercore@sha256:f5f92ece3c213cec075d4abbcf6263eb69c1b94daf17b777da239b909d2862a6"
  },
  "command": [
    "ping",
    "-t",
    "127.0.0.1"
  ],
  "mounts": [
    {
      "container_path": "/volumemountscratch",
      "host_path": "sandbox:///tmp/atlas/emptydir/vol1",
      "propagation": 2
    }
  ]
}

along with a policy that denies arbitrary execs:

package policy

import future.keywords.every
import future.keywords.in

api_version := "0.12.0"
framework_version := "0.5.0"

transparency_trust_lists := [
]

fragments := [
]

containers := [
  {
    "allow_stdio_access": true,
    "command": [
      "ping",
      "-t",
      "127.0.0.1"
    ],
    "env_rules": [
      {
        "pattern": ".+",
        "required": false,
        "strategy": "re2"
      }
    ],
    "exec_processes": [
      {
        "command": [
          "cmd"
        ],
        "signals": []
      }
    ],
    "id": "mcr.microsoft.com/windows/servercore@sha256:f5f92ece3c213cec075d4abbcf6263eb69c1b94daf17b777da239b909d2862a6",
    "layers": [
      "1aba71bb6a4b4c6df1dee7c37607c8e1da4eb6b93de103b6509daf8efd663a7d",
      "8f05ecc2618a5410c5f5360d1bfe5f4b413dd18e3d7ba0bbc724d77e8d6e4628"
    ],
    "mounted_cim": [
      "0b1bdf3036da10b119eb0f656d67f59a87e54bdaa932c3f9dbd1ace1a1738743"
    ],
    "mounts": [
      {
        "destination": "/volumemountscratch",
        "options": [
            "rbind",
            "rshared",
            "rw"
        ],
        "source": "sandbox:///tmp/atlas/emptydir/.+",
        "type": "bind"
      },
    ],
    "name": "mcr.microsoft.com/windows/servercore@sha256:f5f92ece3c213cec075d4abbcf6263eb69c1b94daf17b777da239b909d2862a6",
    "signals": [],
    "user": "",
    "working_dir": "C:\\"
  }
]

external_processes := [
  {
    "command": [
      "cmd"
    ],
    "env_rules": [
      {
        "pattern": ".+",
        "required": false,
        "strategy": "re2"
      }
    ],
    "working_dir": "C:\\",
    "allow_stdio_access": true
  }
]

allow_properties_access := true
allow_dump_stacks := true
allow_runtime_logging := true
allow_environment_variable_dropping := true

mount_device := data.framework.mount_device
mount_overlay := data.framework.mount_overlay
create_container := data.framework.create_container
mount_cims := data.framework.mount_cims
unmount_device := data.framework.unmount_device
unmount_overlay := data.framework.unmount_overlay
# exec_in_container := {"allowed": true, "env_list": null}
exec_in_container := data.framework.exec_in_container
# exec_external := {"allowed": true, "env_list": null, "allow_stdio_access": true}
exec_external := data.framework.exec_external
shutdown_container := data.framework.shutdown_container
signal_container_process := data.framework.signal_container_process
plan9_mount := data.framework.plan9_mount
plan9_unmount := data.framework.plan9_unmount
get_properties := data.framework.get_properties
dump_stacks := data.framework.dump_stacks
runtime_logging := data.framework.runtime_logging
load_fragment := data.framework.load_fragment
scratch_mount := data.framework.scratch_mount
scratch_unmount := data.framework.scratch_unmount
registry_changes := data.framework.registry_changes
log_provider := data.framework.log_provider
load_transparency_trust_list := data.framework.load_transparency_trust_list

reason := {"errors": data.framework.errors, "candidate_containers": data.framework.candidate_containers}

Reported-by: Adrian Cotolan ManalahBT
Assisted-by: GitHub-Copilot copilot-review
Signed-off-by: Tingmao Wang tingmaowang@microsoft.com

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates WCOW sandbox-mount handling to avoid using ExecInUVM to create sandbox mount source directories when running under a confidential policy (where arbitrary exec is denied). Instead, it shifts the directory-creation responsibility into the Windows gcs-sidecar during CreateContainer handling for confidential WCOW.

Changes:

  • Skip ExecInUVM(cmd /c mkdir ...) sandbox mount source directory creation when the UVM has a confidential policy enabled.
  • Add gcs-sidecar logic to create mapped directory source paths before forwarding the CreateContainer request to inbox GCS.
  • Add a helper function in gcs-sidecar to create mapped directory source directories if missing.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
internal/hcsoci/resources_wcow.go Avoids ExecInUVM directory creation for sandbox:// mounts when confidential policy is enabled.
internal/gcs-sidecar/handlers.go Creates mapped-directory source dirs inside the sidecar before forwarding confidential WCOW CreateContainer to inbox GCS.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/gcs-sidecar/handlers.go Outdated
Comment thread internal/hcsoci/resources_wcow.go Outdated
Comment thread internal/gcs-sidecar/handlers.go Outdated
Comment on lines +139 to +141
if err := createMappedDirectorySourceDirs(ctx, container.MappedDirectories); err != nil {
return fmt.Errorf("failed to create mapped directory source directories: %w", err)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tingmao Wang (@micromaomao) , lets add a few tests

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a test, assuming you don't mind side effect of creating C:\SandboxMounts in the runner

…t sources

In confidential WCOW this will not work due to the policy not allowing exec, and
we don't want to add an exception for this.  Instead, we make the gcs-sidecar
handle this job, and make the host skip this for confidential containers.

To test, use a container json that has a mount:

	{
	  "metadata": {
	    "name": "wcow_emptydir_info_cwcow_primary"
	  },
	  "image": {
	    "image": "mcr.microsoft.com/windows/servercore@sha256:f5f92ece3c213cec075d4abbcf6263eb69c1b94daf17b777da239b909d2862a6"
	  },
	  "command": [
	    "ping",
	    "-t",
	    "127.0.0.1"
	  ],
	  "mounts": [
	    {
	      "container_path": "/volumemountscratch",
	      "host_path": "sandbox:///tmp/atlas/emptydir/vol1",
	      "propagation": 2
	    }
	  ]
	}

along with a policy that denies arbitrary execs:

	package policy

	import future.keywords.every
	import future.keywords.in

	api_version := "0.12.0"
	framework_version := "0.5.0"

	transparency_trust_lists := [
	]

	fragments := [
	]

	containers := [
	  {
	    "allow_stdio_access": true,
	    "command": [
	      "ping",
	      "-t",
	      "127.0.0.1"
	    ],
	    "env_rules": [
	      {
	        "pattern": ".+",
	        "required": false,
	        "strategy": "re2"
	      }
	    ],
	    "exec_processes": [
	      {
	        "command": [
	          "cmd"
	        ],
	        "signals": []
	      }
	    ],
	    "id": "mcr.microsoft.com/windows/servercore@sha256:f5f92ece3c213cec075d4abbcf6263eb69c1b94daf17b777da239b909d2862a6",
	    "layers": [
	      "1aba71bb6a4b4c6df1dee7c37607c8e1da4eb6b93de103b6509daf8efd663a7d",
	      "8f05ecc2618a5410c5f5360d1bfe5f4b413dd18e3d7ba0bbc724d77e8d6e4628"
	    ],
	    "mounted_cim": [
	      "0b1bdf3036da10b119eb0f656d67f59a87e54bdaa932c3f9dbd1ace1a1738743"
	    ],
	    "mounts": [
	      {
	        "destination": "/volumemountscratch",
	        "options": [
	            "rbind",
	            "rshared",
	            "rw"
	        ],
	        "source": "sandbox:///tmp/atlas/emptydir/.+",
	        "type": "bind"
	      },
	    ],
	    "name": "mcr.microsoft.com/windows/servercore@sha256:f5f92ece3c213cec075d4abbcf6263eb69c1b94daf17b777da239b909d2862a6",
	    "signals": [],
	    "user": "",
	    "working_dir": "C:\\"
	  }
	]

	external_processes := [
	  {
	    "command": [
	      "cmd"
	    ],
	    "env_rules": [
	      {
	        "pattern": ".+",
	        "required": false,
	        "strategy": "re2"
	      }
	    ],
	    "working_dir": "C:\\",
	    "allow_stdio_access": true
	  }
	]

	allow_properties_access := true
	allow_dump_stacks := true
	allow_runtime_logging := true
	allow_environment_variable_dropping := true

	mount_device := data.framework.mount_device
	mount_overlay := data.framework.mount_overlay
	create_container := data.framework.create_container
	mount_cims := data.framework.mount_cims
	unmount_device := data.framework.unmount_device
	unmount_overlay := data.framework.unmount_overlay
	# exec_in_container := {"allowed": true, "env_list": null}
	exec_in_container := data.framework.exec_in_container
	# exec_external := {"allowed": true, "env_list": null, "allow_stdio_access": true}
	exec_external := data.framework.exec_external
	shutdown_container := data.framework.shutdown_container
	signal_container_process := data.framework.signal_container_process
	plan9_mount := data.framework.plan9_mount
	plan9_unmount := data.framework.plan9_unmount
	get_properties := data.framework.get_properties
	dump_stacks := data.framework.dump_stacks
	runtime_logging := data.framework.runtime_logging
	load_fragment := data.framework.load_fragment
	scratch_mount := data.framework.scratch_mount
	scratch_unmount := data.framework.scratch_unmount
	registry_changes := data.framework.registry_changes
	log_provider := data.framework.log_provider
	load_transparency_trust_list := data.framework.load_transparency_trust_list

	reason := {"errors": data.framework.errors, "candidate_containers": data.framework.candidate_containers}

Reported-by: Adrian Cotolan @ManalahBT
Assisted-by: GitHub-Copilot copilot-review
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Assisted-by: GitHub-Copilot copilot-review
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

internal/gcs-sidecar/handlers.go:287

  • createSandboxMountSourceDirs uses os.Stat to avoid MkdirAll failing on a source that is actually a file, but it currently treats any existing path as success and skips creation without verifying it's a directory. If a file exists at the mount source path, this will silently proceed and the later mapped-directory mount will fail with a less clear error.

Consider checking fi.IsDir() on the Stat() success path and returning an explicit error when the path exists but is not a directory.

			// do this stat rather than call MkdirAll unconditionally,
			// since the latter will fail with a source file (not dir)
			if _, err := os.Stat(source); err == nil {
				log.G(ctx).WithField("source", source).Debug("source of mapped directory mount exists, not creating directories")
				continue

internal/gcs-sidecar/handlers.go:146

  • This change introduces new behavior in the sidecar (creating sandbox mount source directories) but there is no accompanying unit test coverage. Since this package already has handler tests, adding focused tests for createSandboxMountSourceDirs would help catch regressions (e.g., creates missing dirs under C:\SandboxMounts, skips non-sandbox HostPath values, errors when HostPath exists as a file).

This issue also appears on line 283 of the same file.

		if err := createSandboxMountSourceDirs(ctx, container.MappedDirectories); err != nil {

Assisted-by: GitHub-Copilot copilot-review
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants