Skip to content

feat(sandbox): adopt existing bare Sandbox CRs into Gateway management #2187

Description

@Gkrumbach07

Problem Statement

Platforms deploying agents as bare Sandbox CRs, using the agent-sandbox controller directly without an OpenShell Gateway, have no path to enable OpenShell security on those agents without deleting and recreating the CR.

The Sandbox CR is often the source of truth for a running agent's configuration (image, env vars, ports, PVCs, annotations). Deleting it loses that state, PVCs are garbage-collected via owner references, and post-deploy config changes are lost.

Proposed Design

Technical Context

OpenShell enriches Sandbox CRs at creation time, injecting the supervisor, capabilities, credentials, and policy enforcement into spec.podTemplate. A bare Sandbox CR created outside the Gateway runs the agent image directly without any of this.

The Gateway watcher already sees bare CRs (no label filter), but has no mechanism to retroactively apply enrichment to them. CreateSandbox returns AlreadyExists if the name is taken. The only current path is delete and recreate, which loses the CR and its owned resources.

Proposed Design

Add an AdoptSandbox RPC to the compute driver and public API.

Behavior:

  1. GET existing Sandbox CR (fail with NotFound if absent)
  2. Read current spec.podTemplate — preserve user's image, env vars, ports, volumes
  3. Apply enrichment via sandbox_to_k8s_spec() — same code path as Create, merging with existing spec rather than overwriting
  4. PATCH CR with enriched spec.podTemplate + openshell.ai/managed-by and openshell.ai/sandbox-id labels
  5. PATCH operatingModeSuspended (controller deletes old pod)
  6. PATCH operatingModeRunning (controller creates new pod from enriched spec)
  7. Register sandbox in Gateway's internal store

The CR is never deleted. Same UID, same PVCs, same user annotations. The pod restarts with the supervisor.

Merge semantics: Preserve user's image, env, ports, volumes. Append OpenShell env vars and volumes. Override container command to supervisor entrypoint (documented breaking change). Inject supervisor init container.

Proto: New rpc AdoptSandbox(AdoptSandboxRequest) returns (SandboxResponse) in both openshell.proto and compute_driver.proto.

RBAC: Add patch verb to the K8s Role for sandboxes.agents.x-k8s.io.

Alternatives Considered

  1. Delete and recreate: loses the CR (source of truth). PVCs garbage-collected. Not acceptable for stateful workloads.
  2. Platform patches CR directly: reverse-engineers sandbox_to_k8s_spec() output. Fragile, breaks on upstream changes.
  3. Mutating admission webhook: enriches CRs on label change. More Kubernetes-native but more complex. Could be future evolution (Kubernetes Operator #1719).

Agent Investigation

AdoptSandbox RPC — Feasibility Summary

Complexity: Medium-High
Confidence: High — all code paths read end-to-end
Estimated files to change: 8-12
Issue type: feat

Why it's feasible

The enrichment logic is well-encapsulated in three functions that already accept &mut serde_json::Value pod templates:

  • apply_supervisor_sideload() (driver.rs:1124-1228) — supervisor binary, runAsUser:0, capabilities
  • apply_workspace_persistence() (driver.rs:1245-1334) — PVC init container, volume mount
  • sandbox_template_to_k8s_with_gpu_requirements() (driver.rs:1507-1788) — env vars, TLS mounts, SA token projection

The watcher already sees bare CRs (uses ListParams::default(), no label filter). The store-first-then-driver pattern from CreateSandbox is directly reusable.

Why it's not trivial

  1. No patch operations exist in the K8s driver today — only create, get, list, delete. Need to add kube::api::PatchParams / Api::patch() or server-side apply.
  2. Enrichment builds from scratch, not mergessandbox_to_k8s_spec() constructs the full containers array rather than merging into an existing one. Refactoring needed.
  3. StopSandbox is unimplemented — the K8s driver returns Status::unimplemented. Adopt needs suspend/resume for pod restart. This is a prerequisite.
  4. 29 enrichment items must be injected (supervisor sidecar, 6 env vars, mTLS volume, SA token projection, capabilities, workspace PVC, init containers, labels, etc.).
  5. Conflict resolution is a design decision — bare CRs may already have containers named "agent", existing runAsUser, or PVCs at /sandbox. No existing pattern for this.

Critical design decisions needing human input

  1. Conflict resolution: Overwrite (simple, may break workload) vs merge-with-precedence (complex, safer) vs fail-on-conflict (safest, most restrictive)
  2. Agent container identification: Bare CRs won't name their container "agent" — require it, target containers[0], or accept name in request?
  3. Pod restart: Does the agent-sandbox controller recreate pods on podTemplate change, or must we delete the pod manually? Needs controller behavior verification.
  4. Idempotency: Re-adopt for supervisor upgrades, or reject already-adopted CRs?
  5. RBAC: Gateway ServiceAccount needs patch verb added — Helm chart update required.

Recommended phasing

  1. Refactor enrichment functions to accept existing podTemplate (independently testable)
  2. Add K8s Patch support to the driver
  3. Implement StopSandbox (prerequisite)
  4. Add proto definitions, driver RPC, gateway handler, store integration
  5. Edge cases, RBAC, Helm chart, docs, e2e tests

Checklist

  • I've reviewed existing issues and the architecture docs
  • This is a design proposal, not a "please build this" request

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions