feat(epp): observable request lifecycle, routing phases, and reservation callbacks

[0z5a]

Part of #11661 (Milestone 9 — Embedded EPP Observability). This is the first batch: request/phase instrumentation on the current EPP path. It does not close the milestone.

What this adds

Extends the EPP's existing private Prometheus registry (deploy/inference-gateway/ext-proc/src/metrics.rs) with the request, phase, and lifecycle families. The exporter, the /metrics listener, the startup/discovery model-label binding, and dynamo_epp_cached_tokens are reused unchanged — no second registry, no second model labeling, no runtime initialization.

Metric	Type	Labels	Observed at
dynamo_epp_requests_total	counter	model, outcome	once per ext_proc attempt, at its terminal state
dynamo_epp_request_duration_seconds	histogram	model, outcome	same instant, measured from attempt start
dynamo_epp_streams_inflight	gauge	model	incremented when an attempt starts, decremented at its terminal state
dynamo_epp_phase_duration_seconds	histogram	model, phase, outcome	when a phase the attempt entered returns
dynamo_epp_first_response_body_seconds	histogram	model	on the first non-empty response body chunk
dynamo_epp_lifecycle_callbacks_total	counter	model, operation, result	when a picker lifecycle callback returns
dynamo_epp_cached_tokens	histogram	model	unchanged

outcome ∈ {response_eos, upstream_http_error, early_reject, ext_proc_error, incomplete}; phase ∈ {render_tokenize, selection}; operation ∈ {prefill_complete, request_complete}; result ∈ {ok, error}. There is no cancelled result — a call site cannot reliably tell a cancelled call from a failed one, so it reports the failure rather than guessing.

The metric glossary lives in the module docs: exact observation points, what each family includes and does not include, the zero/absent/unknown rules, and the list of things that are not observable at this layer.

Terminal classification

result.is_ok() is deliberately not consulted. The gRPC handler returns Ok(()) both for a completed response and for an immediate rejection it already wrote, so the outcome is derived from the control flow, in a fixed priority order: early rejection → protocol error → upstream HTTP error status → completed response → incomplete.

• incomplete is what a disconnect or force shutdown actually is. It is not reported as a client cancellation, which this layer cannot distinguish.
• upstream_http_error uses the :status pseudo-header when the gateway forwards response headers, so a 5xx backend is not folded into response_eos.
• An attempt that never sees a response body records no first_response_body_seconds sample.
• Backend-reported cached_tokens = 0 remains a real zero sample; a response with no usage information records nothing rather than a fabricated zero.

Exactly-once terminal accounting

One RequestObservation owns one attempt. finish() is consuming, so an attempt records exactly one terminal; if the owning future is dropped first, Drop records Incomplete instead. A dropped stream therefore cannot leak the inflight gauge or disappear from the counters, and no shared request-keyed map is needed.

Reservation lifecycle: calls, not state

dynamo_epp_lifecycle_callbacks_total counts callback invocations and their return. It is explicitly not an active-reservation gauge: free_reservation and prefill_complete are idempotent and return success for an unknown id without releasing anything, so callback == Ok must not decrement a state counter. Authoritative Applied/NoChange transitions need a core-level observer and are not claimed here.

Structured events

epp.request_finished (finite outcome, duration_ms) and epp.routing_failed (request_id, finite stage, finite code, reservation_taken) carry bounded fields only. No prompt, token, authorization, raw handoff, worker address, or error message text is logged. These are tracing events on the existing subscriber; this PR does not add a JSON sink.

Cardinality

model remains the startup/discovery-bound label; it is never read from a request body. Nothing request-scoped (request id, attempt id, reservation/plan/session id, prompt, cache salt, token, worker endpoint, peer IP, path/query/header) is a label. A test drives every label combination the schema allows and asserts the series count per family is exactly the enumeration product.

Behavioural neutrality

Instrumentation does not change routing, status, headers, body, callback order, or cleanup semantics. The pre-existing wire-level integration test and all pre-existing unit tests pass unchanged.

Tests

cargo test -p dynamo-ext-proc --lib          # 177 passed; 0 failed   (baseline 160; +17)
cargo test -p dynamo-ext-proc --test ext_proc_test   # 1 passed; 0 failed
cargo clippy -p dynamo-ext-proc --all-targets -- -D warnings   # clean
cargo fmt -p dynamo-ext-proc -- --check                        # clean
git diff --check                                              # clean

New coverage:

• terminal taxonomy: response_eos, upstream_http_error, early_reject, incomplete; exactly-once terminal; inflight back to zero; body-free stream; two streams sharing a client request id;
• first body: recorded once, on the first non-empty chunk, never when no body arrives;
• recorder: finish + Drop records one terminal, not two; isolated registries do not share samples; exposition declares HELP/TYPE for every family;
• cardinality: every outcome, phase, operation, and result combination is driven once, and each family's series count equals its label product.

Follow-up commits

• b8974c7c48 condenses the module documentation into one glossary plus the readings most likely to be
  misread, and drops two tests that asserted properties the compiler or the test helper already
  guarantees. No runtime path changed; 175 lib tests pass.

Base

402c2611ffcb3dbdeea344677d757dfed5469912

Not in this PR

• Authoritative reservation transitions and scheduler load snapshots (needs a reviewed core observer; HostTelemetry is a candidate, its snapshot granularity needs checking first).
• Replica connection/sync-lag signals. The discovery set does not prove a peer registered or caught up, so no such series is published; discovered / register / deregister operation outcomes follow separately.
• Sidecar and paired P/D stages, which wait on feat(vllm): add NIXL handoff adapter to the standalone decode sidecar #13404 and feat(epp): add role-scoped worker catalogs for standalone disaggregated routing #13406/feat(epp): add paired prefill/decode selection and reservation lifecycle #13407 respectively.
• Real TTFT/ITL and per-token progress: no token-boundary signal exists at this layer.
• Output-growth accounting and worker filtering.
• A JSON log sink (the current fmt subscriber does not imply one).

Notes for review

• ExtProcServer::new now takes the recorder instead of reaching a global, which is what lets the tests assert on an isolated registry. EppRouter::from_selector takes it for the same reason.
• The :status capture only fires when the gateway forwards response headers; when it is absent the attempt is classified without it, and the glossary says so.
• selection includes queue wait and render_tokenize includes renderer latency. They are named for the phase, not for compute, on purpose.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi 0z5a! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀