Conversation
…r read methods Phase 1 of the primitives refactor (no behavior change): - ExperimentSession bundles the shared client, API wrappers, identity, queue, stats, and background worker; created once in Experiment.__init__ and exposed via the .session property. - ArtifactsApi now shares the single LitRestClient. - New read wrappers for the resume path: MetricsApi.get_metric_values, ArtifactsApi.list_experiment_artifacts, MediaApi.list_media. - File._log_artifact/_bind_remote_artifact now require a client: no LitRestClient construction remains in media.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 2 (part 1): primitives own their serialization and remote writes. - Primitive protocol: log(session) synchronous write + enqueue(session) background hand-off. - Metric: enqueue feeds the existing batching queue unchanged; log is a new synchronous single-value append with worker-equivalent auto-stepping. - Metadata: log performs the full-tag read-modify-write (fresh remote read, from_code filter, phase=RUNNING); enqueue defers it to the worker. - Support shims delegate; RMW tests re-seeded one seam up (get_experiment_metrics_by_name) with unchanged behavioral assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 2 (part 2): File/Image/Video/Text/Model own their remote writes.
- File gains log(session)/enqueue(session) plus stamped placement
(_log_key, _series_index, _series_step); static uploads use the key,
series elements key/{index}.
- _log_artifact/_bind_remote_artifact replaced by session-based
_upload_artifact/_bind_remote; media.py no longer touches ArtifactsApi
or LitRestClient.
- Image/Video/Text share a _MediaFile.log override (media API, bare key
+ step); Model.log auto-versions series elements and uploads via the
registry, keeping the experiment= linkage kwarg.
- Support shims reduce to placement stamping + dispatch; finalize()
uploads console_output.txt through the session (still unregistered,
no stats bump).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 2 (part 3): each primitive owns its restore path.
- Metric._restore_values via MetricsApi.get_metric_values;
Metadata._current_tags serves both restore and the metadata property.
- File._restore_all (artifact listing + key/{index} regex + store
fallback) and File._restore_media (media listing, V1-type wrapping,
regex + arity/step heuristic) return RestoredFiles; restore passes take
a key-type snapshot and track in-pass claims instead of mutating
experiment state.
- Model._resolve/_from_version own lazy registry lookup; the positive/
negative caches stay on Experiment.
- rebuild_state is now thin orchestration (metadata -> metrics ->
artifacts -> media) with unchanged overwrite/skip semantics; rebuild
tests re-seeded against the API wrappers instead of raw client calls.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 3: Experiment now owns only public API, coercion/dispatch, local registration, resume orchestration, and lifecycle. - New _coerce_static_value/_coerce_series_value classify inputs and stamp primitive placement; __setitem__ and the Series seam dispatch straight to primitive log()/enqueue(). - experiment_support.py deleted; its delegator shims on Experiment removed (_set_static_file, _set_metadata_value, _upload_media_value, _upload_model_value, _wrap_media_file, _media_type_to_v1, _code_tags, _update_metrics_store -> session.refresh_metrics_store, etc.). - Additive public API: Metric, Metadata, Primitive, ExperimentSession exported; Experiment.session property; test_public_api.py guards the export surface. - Unit-test factories now attach a live ExperimentSession view over their mock infrastructure; shim-level assertions re-expressed against the API wrappers with unchanged behavioral expectations; refresh coverage moved to test_session.py. tests/integrations/ untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 4 (approved behavioral change): the dict API enqueues everything. - The experiment queue is now a thread queue (queue.Queue): queued file primitives must reach the worker as the caller's objects, not pickled copies, so name/download bindings land on what the user holds. - _BackgroundThread executes _QueuedWrite items one by one via the session, keeps metric batching unchanged, and on failure drains leftover items so queue.join() cannot deadlock. - __setitem__ and file-series appends dispatch via enqueue; metadata RMWs are serialized by the single worker. Legacy log_media/log_model* stay synchronous (no key registration to hang a barrier on). - Read-your-writes via read barriers: File.save/Model.load flush the queue first (barrier attached at enqueue), as do the metadata property and lazy model resolution. finalize()'s join now covers file writes and surfaces late worker failures instead of swallowing them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…line Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Experiment logging behavior is currently split between
Experiment, broad support helpers, media classes, and independently constructed API clients. This makes individual data types harder to evolve and leaves metadata and file-like writes blocking the caller.This introduces a shared primitive/session architecture and extends write-behind logging to every dict-style value.
Metric,Metadata,Primitive, andExperimentSession.Previously, only metrics used the background queue. Metadata and dict-style file, media, and model writes completed on the caller’s thread, while their write and restore behavior was coordinated through
experiment_support.py.Experimentnow focuses on public API dispatch, local registration, restoration orchestration, and lifecycle. Each primitive implements synchronouslog(session)and asynchronousenqueue(session)behavior against a sharedExperimentSession.The queue now carries both metric batches and non-metric primitives. It uses an in-process thread queue so the worker receives the caller’s wrapper object and can attach its remote name and download/load behavior after upload.
Dict assignments therefore return after enqueueing.
Experiment.metadata,File.save(),Model.load(), lazy model lookup, andfinalize()provide barriers that wait for queued writes and surface worker failures. The keyless legacy media and model methods remain synchronous.