Repository navigation
Replay goldfive events.jsonl from disk: harmonograf-replay + zicato handoff - #292
Merged
Merged
Conversation
A finished goldfive run is a file: the recorder writes one goldfive.v1.Event per line to an events.jsonl next to the run. Harmonograf could terminate a live event stream but had no way to ingest one of those files after the fact, so a finished run could not be re-rendered as a session. harmonograf-replay closes that gap as a thin console script over a new harmonograf_client.replay module. It reads an events.jsonl and feeds every event through the existing HarmonografSink, reusing the live path's LLM-call-to-span translation, agent-id canonicalization, and dict-envelope handling unchanged -- replay is just a non-live producer, no new wire surface or server endpoint. Both on-disk shapes are accepted: proto-JSON (parsed with ignore_unknown_fields so a file written by a newer goldfive than the pinned submodule still replays) and the kind+payload dict envelope. Corrupt and blank lines are skipped rather than fatal. The session id is preserved from Event.session_id so a replayed run is stable and addressable by its run id.
Add docs/zicato-handoff.md describing how a zicato run's events.jsonl is replayed into harmonograf as that run's execution view, the deep-link URL shape (/#/session/<run-id>), and what zicato must emit for the link to resolve. Calls out the conceptual split: harmonograf renders one run over time, zicato's own dashboard renders the competition across runs. Honest note included: the frontend hash router does not yet parse a session id from the URL, so the run id resolves as the session id the operator selects in the picker until the small #/session/<id> route lands. Link the new doc from the README documentation table and docs/index.md.
`harmonograf-replay` is fire-and-forget on the wire — events are pushed onto the transport buffer and a background send loop drains them. Previously `_run` returned 0 regardless of delivery, so a replay against a down server / wrong --server / rejected token printed a 'success' summary (with `session: (unassigned)`) and exited green — a zicato hook shelling out to replay would treat a dropped run as landed. `_run` now keys delivery on `client.session_id`: the transport's assigned session id is set only when the server answers Hello with a Welcome, so an empty value after shutdown means no server was reached. (Buffer emptiness is not a sound signal — the send loop pops envelopes off the ring into an in-flight gRPC call before that call can fail, so the ring can read empty with nothing acked.) A buffer-overflow drop during replay is also surfaced. Either case logs an actionable error and exits 3.
Adds the test cases the original 9 left open: - a JSON object that parses but is not a valid goldfive Event (a type mismatch ignore_unknown_fields cannot absorb) — exercises the skipped_unparseable counter, a path distinct from the corrupt-JSON skip and previously untested. - _run exit codes: missing file and directory argument return 2, empty and blank-only files return 1, and a run that never reaches a server (dead port) returns 3 rather than a misleading green exit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Smooths the zicato to harmonograf handoff. A zicato run produces a goldfive
events.jsonl(onegoldfive.v1.Eventper line); zicato's dashboard wants to deep-link each run into harmonograf as that run's execution view. Harmonograf could terminate a live event stream but had no way to ingest one of those files after the fact.harmonograf-replay-- a new console script (overharmonograf_client.replay) that reads a goldfiveevents.jsonlfrom disk and feeds every event through the existingHarmonografSink, so a finished run renders as a harmonograf session. No new wire surface, no new server endpoint -- replay is just a non-live producer that reuses the live path's LLM-call-to-span translation, agent-id canonicalization, and dict-envelope handling unchanged.docs/zicato-handoff.md-- the handoff contract: the deep-link URL shape, what zicato must emit, and the conceptual split (harmonograf = one run over time; zicato's dashboard = the competition across runs).What it handles
Both on-disk event shapes goldfive writes:
ignore_unknown_fields=True, so a file written by a newer goldfive than the pinned submodule still replays (unknown event kinds drop, known ones render).kind+payload) -- passed to the sink verbatim; the sink owns dict-to-proto translation.Corrupt and blank lines are skipped, never fatal. The session id is preserved from
Event.session_id, so a replayed run is stable and addressable by its run id (ingest is idempotent on(session_id, run_id, sequence)).Verification
Verified end-to-end against a real zicato run's
events.jsonland a liveharmonograf-server: 116 events emitted per run (107 proto + 9 dict, 13steeringDecisionMadeevents newer than the pinned goldfive skipped cleanly), landing as a session with 2 agent rows, 24 spans, 21 goldfive events, and 3 refine attempts -- exactly what the frontend Gantt/Trajectory consume.Deep-link shape
<harmonograf-frontend-origin>/#/session/<run-id>where<run-id>is the goldfive run id zicato already has. The doc includes an honest note: the frontend hash router does not yet parse a session id from the URL (today it routes#/and#/stress; session selection is in-app state). Wiring#/session/<id>tosetCurrentSessionon load is a small isolated follow-up; the URL shape is the agreed target so zicato can emit the final URL now.Test plan
make client-test-- 362 passed (9 new intest_replay.py)ruff checkclean on new filesharmonograf-replay <real zicato events.jsonl>against a live server -- session renders with spans, goldfive events, refine attempts#/session/<id>hash route (follow-up, out of scope here)