Stream metadata responses above Cloud Run's non-streamed size cap

[MaxGhenis]

Part of #3797 (the /us/metadata symptom only; the simulation connection-reset symptom is separate).

Root cause

GET /us/metadata serializes to ~70 MB (71,532,620 bytes on 2026-08-22). Cloud Run drops HTTP/1 responses above 32 MiB unless they use Transfer-Encoding: chunked or streaming (Cloud Run quotas: "Maximum HTTP/1 response size: 32 MiB if not using Transfer-Encoding: chunked or streaming mechanisms"). Both the native FastAPI route and the Flask fallback send the body with Content-Length, so since the Cloud Run cutover any client that does not negotiate gzip gets an empty 500 generated by the Google Frontend (content-type: text/html, content-length: 0, no x-policyengine-* headers — the app never sees a failure).

Clients that negotiate gzip were never affected: GZipMiddleware compresses the body to ~10 MB, under the cap. Browsers and python requests/httpx negotiate gzip by default, which is why the app kept working while plain curl (and any non-gzip monitor) saw 500s. Verified live on 2026-08-22:

GET /us/metadata                          -> 500, empty body, Google Frontend
GET /us/metadata (Accept-Encoding: gzip)  -> 200, content-encoding: gzip, 9,956,182 bytes
HEAD /us/metadata                         -> 200, content-length: 71,532,620
GET /uk/metadata (1.18 MB)                -> 200 either way

Fix

Serialize once exactly as before, but deliver bodies at or above 20 MiB as a streamed response with no Content-Length — chunked on the wire, which Cloud Run accepts at any size. Applied to both the native FastAPI route (what production serves; ROUTE_IMPL_METADATA=fastapi_native) and the Flask fallback, sharing one helper. Responses under the threshold keep their exact current framing, so UK/CA/NG/IL metadata and every other route are byte-identical. GZipMiddleware still compresses streamed bodies for gzip-negotiating clients.

Alternatives considered:

• Forcing gzip regardless of Accept-Encoding breaks clients that genuinely cannot decode it.
• Shrinking the payload (US metadata is 60× UK's) is worth doing separately, but is an app-contract change, not an incident fix.

Tests

• tests/unit/test_streaming_json.py: threshold boundary and chunk reassembly.
• tests/unit/test_stage6_native_metadata.py: large bodies stream without Content-Length for identity clients, still gzip for negotiating clients, and small bodies keep Content-Length. Full tests/unit run locally: 1071 passed (the 3 pre-existing collection errors in test_gcp_logging.py / ai_prompts reproduce identically on clean master in the same environment).

After deploy, the incident repro flips: curl -s -o /dev/null -w "%{http_code}" https://api.policyengine.org/us/metadata should print 200.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 91.30435% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.13%. Comparing base (f45a7ab) to head (4f8ae0f).
⚠️ Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
policyengine_api/routes/metadata_routes.py	50.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3798      +/-   ##
==========================================
+ Coverage   85.11%   85.13%   +0.01%     
==========================================
  Files         109      110       +1     
  Lines        6290     6309      +19     
  Branches     1093     1096       +3     
==========================================
+ Hits         5354     5371      +17     
- Misses        630      631       +1     
- Partials      306      307       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.