Skip to content

feat(demos/agent-gateway-shared-vpc): add Agent Gateway egress governance with Shared VPC (CUJ2 Extension) - #218

Draft
augustoscit wants to merge 6 commits into
GoogleCloudPlatform:mainfrom
augustoscit:feat/cuj2-agent-gateway-shared-vpc
Draft

augustoscit wants to merge 6 commits into
GoogleCloudPlatform:mainfrom
augustoscit:feat/cuj2-agent-gateway-shared-vpc

Conversation

@augustoscit

Copy link
Copy Markdown

Adds the click-to-deploy Shared VPC egress governance demo (CUJ2 Extension) for the Gemini Enterprise Agent Platform.

This demo demonstrates how to implement an enterprise-grade, multi-project outbound network governance architecture for AI agents deployed on Vertex AI Agent Engine (Reasoning Engine). By leveraging Google Cloud Shared VPC, it enforces strict separation of duties between a centralized Networking Host Project (VPC, SWP, Cloud NAT with static IP, Private DNS) and a Governance Service Project (Agent Gateway, Model Armor guardrails, Agent Registry, and Agent Runtime), while routing internal Google APIs over Private Google Access (PGA).

What this does

Deploys a complete end-to-end multi-project outbound network governance solution via Terraform:

  • Decoupled Two-Project Enterprise Architecture (Shared VPC):

    • Networking Host Project: Centralizes core network infrastructure, including Shared VPC host enablement (google_compute_shared_vpc_host_project), subnets (workload, PSC-I /26, SWP proxy-only /24), Secure Web Proxy in NEXT_HOP_ROUTING_MODE, GatewaySecurityPolicy, Cloud Router, Cloud NAT (MANUAL_ONLY with reserved static external IP), and Private Cloud DNS zone for googleapis.com. (Private Google Access).
    • Governance Service Project: Hosts business and AI logic, including Shared VPC service attachment (google_compute_shared_vpc_service_project), cross-project PSC-I Network Attachment, Agent Gateway in AGENT_TO_ANYWHERE mode, native CENTRALIZED AgentConnectivityTemplate, Model Armor guardrail templates, Cloud Run FastMCP service, and Vertex AI Reasoning Engine.
    • Cross-Project IAM Automation: Automatically provisions roles/compute.networkUser and roles/dns.peer on the Networking Host Project for the Governance project's service agents (gcp-sa-agentgateway, gcp-sa-aiplatform-re, and default Cloud Services SA).
    • Multi-Project Terraform Engine: Configures dual providers (google and aliased google.networking) with user_project_override = true and explicit per-project billing_project, eliminating local ADC quota conflicts and cross-project permission collisions.
  • Cross-Project Egress Network Governance via Agent Gateway & ACT (CENTRALIZED):

    • Provisions a Google-managed Agent Gateway in AGENT_TO_ANYWHERE mode connected across projects via a Private Service Connect (PSC-Interface) Network Attachment deployed in the Governance project but bound to the Shared VPC subnet in the Host project.
    • Links the gateway to an AgentConnectivityTemplate with vpcEgress: ALL_TRAFFIC and native deployment_model = "CENTRALIZED" (utilizing the latest hashicorp/google provider features), routing 100% of runtime egress through the Shared VPC.
    • Generates template names dynamically with random_id (e.g., agent-gateway-act-<hex>), completely isolating fresh deployments from pre-GA alpha tombstone reference locks.
    • Orchestrates the gateway lifecycle directly through the Network Services v1 REST API (scripts/manage_agent_gateway.sh), eliminating gcloud alpha (v1alpha1) LRO timeout bugs and achieving provisioning in ~4 minutes.
  • Dual-Path Routing & Private Google Access Bypass (PBR):

    • PBR 1500 (Bypass SWP): Intercepts 199.36.153.8/30 (resolved by private Cloud DNS *.googleapis.com. in Host project) and routes internal Google API traffic (Vertex AI, Cloud Trace, IAM) directly via Private Google Access (DEFAULT_ROUTING), avoiding NAT and SSL handshake failures.
    • PBR 2000 (Redirect to SWP): Steers all public outbound container traffic (such as MCP calls to *.a.run.app) to the internal IP of the Secure Web Proxy (10.0.0.2:443).
  • L7 Inspection, Content Safety & Reserved Static Egress IP:

    • Secure Web Proxy operates in NEXT_HOP_ROUTING_MODE with a GatewaySecurityPolicy allowing vetted hosts.
    • Cloud Router and Cloud NAT run in MANUAL_ONLY mode with an external reserved static IP (<name_prefix>-nat-ip), ensuring remote servers and firewalls can verify the agent's deterministic identity.
    • Model Armor Integration: Provisions request and response sanitization templates (RAI filters, prompt injection, and jailbreak detection) along with Cloud DLP inspection templates for enterprise AI security compliance.
  • Agent Gateway Default-Deny Override (Allow-All AuthzPolicy):

    • Attaches an allow-all AuthzPolicy (action = ALLOW, when = 'true') to the Agent Gateway via declarative lifecycle scripts. This delegates L7 inspection and policy enforcement to the customer Shared VPC SWP and Model Armor, overriding the gateway's internal default_denied drop.
  • Software Bug Agent & Cloud Run MCP Service:

    • software-bug-agent: Deploys an ADK agent with SafeAdkApp patching socket.getaddrinfo to force pure IPv4 resolution, preventing aiohappyeyeballs socket unreachable errors (Errno 101).
    • bug-tickets-mcp: Cloud Run service running FastMCP 3.x with min_instance_count = 1 and session_affinity = true. The agent client sets timeout = 30.0 to eliminate cold-start timeouts.
  • Resilient Multi-Project Teardown Automation:

    • network_attachment_drain: Polls connectionEndpoints every 10s (up to 45 attempts) to guarantee PSC interfaces fully detach before deleting the Network Attachment across projects, preventing Error 412.
    • manage_agent_gateway.sh delete: Implements a 30-attempt retry loop on HTTP 400 (FAILED_PRECONDITION) to wait for Vertex AI Reasoning Engine decoupling before deleting the gateway.

Testing

  • Day-0 Multi-Project Clean Deployment:
    • Deployed cleanly across two separate GCP projects via two-phase apply:
      1. Phase 1: terraform apply (Foundation, Shared VPC, cross-project IAM, SWP, Cloud NAT, Cloud Run MCP, Model Armor, Agent Registry, Agent Gateway with ACT).
      2. Agent build: uv run python deploy_agent.py --build-only.
      3. Phase 2: terraform apply -var deploy_reasoning_engine=true (Vertex AI Reasoning Engine in Governance project).
  • End-to-End Query Verification:
    • Tested the agent in Vertex AI Agent Engine with prompt:
      List all open P1 bugs assigned to alice@quantumroast.example.
    • Verified live invocation of list_tickets tool through the Governance project Agent Gateway, Host project Shared VPC PSC-I subnet, SWP, and Cloud NAT, returning the formatted Markdown table of P1 bugs.
    • Confirmed Cloud Run request logs show caller IP matches the Cloud NAT reserved static external IP (terraform output -raw nat_static_ip).
    • Confirmed L7 audit logging in networkservices.googleapis.com/gateway_requests (HTTP 200 CONNECT/POST) and Model Armor safety telemetry.
  • Automated Teardown & Replay:
    • Verified full teardown via terraform destroy -var deploy_reasoning_engine=true with zero dangling resources across both Host and Service projects.
    • Validated that cross-project Network Attachment drain cleanly unbinds PSC-I interfaces without blocking Host subnet destruction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant