Repository navigation
SF-12: manual coordinator sign-off before a model reaches labs - #88
Merged
Merged
Conversation
Decided on 2026-10-16 from the BabelBrain FL meeting: model updates are not released automatically; the coordinator tests each aggregated model against a test set before release. The evaluation gate still runs automatically and publishes accepted models, but a published version now starts as pending. Router: - ModelVersion gets status (pending, approved, rejected), reviewed_by, reviewed_at and review_note. Migration 0013 marks existing rows approved, reviewed_at = created_at, so nothing already released disappears. - POST registry/approve/ and registry/reject/ with version and an optional note. Reviewers are the superuser and, with site tokens, the site that coordinates the version's project. A decision is final: the same decision again returns 200, the other one 409. - GET registry/ and registry/latest/ return approved versions only. The status filter pending, rejected or any is for reviewers; others get 403. A token coordinator sees unapproved versions of its own projects only. - GET registry/file/ serves a pending or rejected version only to a reviewer, so the coordinator can download it to test by hand. Controller: list_models and latest_model take an optional status. The coordinator takes the parent of a new version from status=any, since training continued from the newest registered model, released or not. e2e: the registry test approves the run's newest pending model before a lab lists and downloads it. TASK_GUIDE documents the sign-off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
stedrew
added a commit
that referenced
this pull request
Oct 17, 2026
The gate now decides on relative l2 alone, D8, so the intro no longer says peak distance blocks by default; --peak-margin-mm adds it back as an opt-in. Gate-approved models now wait in the registry as pending, D7: the demo lists them, signs off the newest as the coordinator and shows it approved. Needs PR #87 and PR #88 merged first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Decided on 2026-10-16 from the BabelBrain FL meeting: model updates are not released automatically. The coordinator tests each aggregated model against a test set before release. The gate still runs and publishes accepted models, but each version now starts as pending.
Router
ModelVersion:status(pending, approved, rejected),reviewed_by,reviewed_at,review_note. Migration0013_modelversion_sign_offmarks existing rows approved, withreviewed_at = created_at.makemigrations --checkreports no changes.POST registry/approve/andPOST registry/reject/, body{"version": "250k-v0003", "note": "..."}. Reviewers are the superuser and, with site tokens, the site that coordinates the version's project. A decision is final: repeating it returns 200, the opposite decision returns 409.GET registry/andGET registry/latest/return approved versions only.status=pending|rejected|anyis for reviewers only; anyone else gets 403. A token coordinator sees unapproved versions of its own projects only.GET registry/file/serves a pending or rejected version only to a reviewer, so the coordinator can download it and test it.Controller
transfer.list_modelsandtransfer.latest_modeltake an optionalstatus.status=any, since training continued from the newest registered model whether or not it was released.Tests
starfish-controller-babelbrainimage: 580 pass, 1 skipped.e2e/test_babelbrain_workflow.py: the registry test now approves the run's newest pending model first. Not run here; it needs the full workbench stack.Not in this PR
🤖 Generated with Claude Code