Phase 5: import mint-ensemble-manager into the single repository - #162
Merged
Merged
Conversation
…atalog_parameter DYNAMO v2.0 migration renamed the object relationship on thread_model_parameter from 'model_parameter' to 'modelcatalog_parameter'. Hasura was rejecting GraphQL queries with: field 'model_parameter' not found in type: 'thread_model_parameter'. Updates parameter_bindings selections in thread/get.graphql, emulator/model-executions.graphql, emulator/thread-executions.graphql, and the matching tm/pb readers in graphql_adapter.ts.
…name field modelcatalog_parameter has no 'name' column -- the migration uses 'label' (consistent with other modelcatalog_* tables). Hasura was rejecting: field 'name' not found in type: 'modelcatalog_parameter'. Updates parameter_bindings selections in thread/get.graphql, emulator/model-executions.graphql, emulator/thread-executions.graphql, and the emulator branch of executionFromGQL in graphql_adapter.ts.
The catch block in POST /executionEngines/tapis returned an error JSON without writing anything to stdout, so failures were invisible in pod logs. Add console.error in both branches (HttpError vs unknown) and narrow the unknown error type before reading .message to satisfy strict TypeScript.
submitSingleExecution submitted a job successfully and then called
subscriptionsClient.subscribe directly. tapis-typescript runtime
throws the raw fetch Response on non-2xx, so the catch block printed
String(Response) ("[object Response]") and discarded the JSON error
body, leaving real failures invisible in the pod logs.
Wrap subscribe in errorDecoder so 500/4xx responses surface as
HttpError(decoded.message). Log the jobUuid and webhook delivery
target before subscribing and the decoded message on failure. Wrap
updateExecutionRunId with a labelled catch so the next reproduction
pinpoints which post-submit step fails.
…ilure handle_failed_connection_ensemble previously deleted every thread_model_execution junction row for the thread when a Tapis submission failed (subscribe 500, etc.). The execution row was correctly marked status=FAILURE, but the UI's executions_for_thread_model query joins through that junction and returned no rows, leaving the run page stuck on the "Downloading software image and data..." placeholder instead of rendering the failed run. Drop the delete_thread_model_execution clause from the mutation. Summary delete+reinsert is preserved; only the junction deletion was harmful.
Add optional tapis_webhook_base_url preference so Tapis can deliver job
notifications to a publicly reachable target (e.g. webhook.site) when the
ensemble manager ingress hostname is not resolvable from outside the
cluster. Falls back to ${ensemble_manager_api}/tapis when unset.
Migrate model catalog integration from SDK to direct Hasura GraphQL
Add explicit tag rules to docker/metadata-action so pushes to master also publish mintproject/ensemble-manager:latest in addition to the branch and SHA tags.
Hasura relation on thread_model_io was renamed from model_io to modelcatalog_dataset_specification. Use a GraphQL alias so the response shape stays the same and existing consumers keep working.
modelcatalog_dataset_specification exposes label and has_format, not name and format. Alias them so consumers continue to see model_io.name and model_io.format.
feat(schema): add input can be optional on model configuration
* fix(tapis): tolerate empty-string parameter bindings in submitExecutions POST /executionEngines/tapis crashed with "Cannot read properties of undefined (reading 'match')" when an input parameter had no default value and the user-supplied binding was an empty string. The empty string is falsy, so neither branch in getInputsParameters assigned parameters[ip.id], and the subsequent regex check on the value threw. The crash propagated into handleSubmissionFailure, which then crashed on this.seeds.length because seeds was never initialized before seedExecutions threw. The cleanup mutation never ran and the original error was masked. - Guard the geojson regex with optional chaining so undefined values pass through unchanged. - Initialize this.seeds = [] at the start of submitExecutions so the failure cleanup path always has a valid array. - Add unit tests covering empty-string, populated, and default-value parameter cases. * fix: use node 24
* fix(tapis): tolerate empty-string parameter bindings in submitExecutions POST /executionEngines/tapis crashed with "Cannot read properties of undefined (reading 'match')" when an input parameter had no default value and the user-supplied binding was an empty string. The empty string is falsy, so neither branch in getInputsParameters assigned parameters[ip.id], and the subsequent regex check on the value threw. The crash propagated into handleSubmissionFailure, which then crashed on this.seeds.length because seeds was never initialized before seedExecutions threw. The cleanup mutation never ran and the original error was masked. - Guard the geojson regex with optional chaining so undefined values pass through unchanged. - Initialize this.seeds = [] at the start of submitExecutions so the failure cleanup path always has a valid array. - Add unit tests covering empty-string, populated, and default-value parameter cases. * fix: use node 24
FIXED inputs are locked by the app definition; Tapis injects them from the app at submission. createJobFileInputsFromSeed previously required every app fileInput to have a matching model.input_files entry, throwing "Component input not found" for FIXED inputs since they are not exposed to the model component. Skip FIXED inputs in the job request (Tapis fills them in).
matchTapisOutputsToMintOutputs crashed with TypeError reading 'name' on undefined
because getModelOutputsByModelId returned the raw Hasura response
(outputs[].output.{id,label,has_format}) while typed as ModelOutput[]
({position, model_io:{id,name,format,variables}}). The matcher accessed
mintOutput.model_io.name on the raw shape and threw.
Extracted mapHasuraOutputsToModelOutputs and added unit tests covering the
real Hasura payload, null/undefined input, and missing fields. Function now
returns [] instead of null on errors to match its declared return type.
#119) processInputFiles unconditionally pushed every model input id into the list passed to cartProd, even when an optional input had no user binding and no fixed value. cartProd then read `undefined.length` and the POST to /executionEngines/tapis crashed before the stage-2 skip in TapisJobService could run. Skip optional inputs (is_optional=true) that have neither a binding nor a fixed value: omit the id from inputIds and do not seed an undefined binding. Required inputs and optional inputs with bindings are unchanged. Downstream Tapis/localex adapters already tolerate the missing key.
* build: commit package-lock.json The lockfile was ignored. Dependencies floated on every install. @apollo/client moved from ^3.7.1 to 3.14.1 with no record. npm ci now installs a fixed tree. * build: set skipLibCheck in tsconfig tsc --noEmit reported 71 errors. All 71 are in node_modules. None are in src/. skipLibCheck turns the typecheck green.
A path cannot be a gitlink and a tree at the same time. The next commit merges the service history at this path, so the gitlink goes first. This is the last service import. After it, .gitmodules holds only ui and helm-charts.
364 commits, mainline only. Branches and tags do not cross: 60 of the 78 refs hold commits that are not on main. The archive is their only copy. Authors are normalised through the shared mailmap. Five names collapse to two people. "Your Name <you@example.com>" stays unmapped: two commits, both 2020-12-11, and the committer field gives no signal. Three credentials were stripped from the history during the rewrite. GitHub push protection refused the import without this. - src/config/mint-full-4dc148cc116b.json, a Google Cloud service account key with its private key. Present from the initial commit until 2019. Deleted from every commit. - src.bak/, an accidentally committed backup folder holding an AWS key pair. Its own commit says "accidently added a backup folder". Deleted from every commit; that commit is now empty and drops out, which is why 364 commits cross and not 365. - The AWS access key id and secret access key in src/config/config.json, replaced with ***REMOVED*** in every commit that held them. HEAD already carries CHANGEME placeholders, so no working tree changes. The imported tree at this commit is byte-identical to mint-ensemble-manager main. Only history changed. The source repository still holds the original commits, so these credentials must be rotated regardless of this import.
Node 24, to match mint-ensemble-manager/Dockerfile. Testing on a Node the image does not use gives a green build that fails in the cluster. No path filter, so every single-repo commit builds an image for every service. The Dockerfile does COPY . ., so the build context is the service directory, not the repository root. No lint or format job. The service carries 475 eslint errors and 74 unformatted files. Adding one now would block every pull request. The image moves from Docker Hub mintproject/ensemble-manager to ghcr.io/mintproject/ensemble-manager. Also delete mint-ensemble-manager/.github/workflows/docker-publish.yml. GitHub reads workflows only at the repository root, so after the import that file never runs.
Delete .husky/ and the "prepare" script. core.hooksPath holds one value for the whole repository, and ui-react holds it. Two services cannot both install husky; the last npm install wins and nothing reports it. This is mutual exclusion, not a scoping problem, so the ui-react pattern does not help. Before this commit, npm ci in mint-ensemble-manager printed a husky usage error and carried on. Delete lint-staged: the dependency, the script and the config block. Its only caller was the deleted hook. Also drop the stale line in mint-ensemble-manager/CLAUDE.md that named the script. Delete .release-it.json and VERSION. release-it is not in devDependencies. The config was last touched 2019-12-19 and drives a tool that cannot run. Add "private": true to package.json. The package is never published to npm. prettier:fix and eslint:fix stay, for manual use. Verified after the change: npm ci is silent, core.hooksPath is still ui-react/.husky/_, tsc --noEmit reports 0 errors and 80 tests pass.
Root skills end at two, plus one directory-scoped skill under model-catalog-api/. bump-image-tag replaces dynamo-bump-from-branch and update-helm-image-tags, deleted in Phase 3. It resolves one tag from mintproject/monorepo and writes it to global.imageTag. The component_key|owner/repo|branch table disappears: one single-repo commit builds all four services, so one tag names the whole system state. It aborts on the two silent failures the old scripts could not see. A per-service components.<name>.image.tag wins over the global, and the chart pins all four by default, so the script clears them. A tag from mintproject/monorepo does not exist on Docker Hub, so a repository that is not the GHCR one is an error, not a warning. It also proves every one of the four images is in GHCR at that tag before it writes. test-on-k8s-dev was written on the submodule model throughout. Its description, its component table and its claim that CI in model-catalog-api pushes on every branch push are all rewritten for the single repository. The table gains ui-react, container names and the live ingress hosts, verified against the cluster on 2026-08-30. It now points at bump-image-tag.
dependabot.yml is a first setup. No dependabot.yml exists anywhere in this repository today. One github-actions entry at the root, plus one npm entry per service directory. Monthly, and every entry grouped, so this is four pull requests a month rather than four per dependency. Ungrouped weekly updates are what produced 8 open and 0 merged on mint-ui-lit. The case for it is github-actions: mint-ensemble-manager arrived pinned to actions/checkout@v2 and docker/login-action@v1. release-please gives the repository one version. Every single-repo commit builds all four services, so one tag names the whole system state. Two settings come off after the first release: release-as, which fixes 0.1.0, and bootstrap-sha, which stops the changelog walk at the last commit before the three service histories were imported. The re-tag workflow adds the release version to the four manifests that already built and passed, and asserts the digest is unchanged. It does not rebuild. A rebuild would ship an artifact no test saw. Both release workflows are unused until Phase 8.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
checkout -B ran from whatever branch the values repository happened to be on, so the pull request carried that branch's other commits too. Check the branch and abort instead. Also let gh infer the repository from the working directory, rather than parsing the origin URL with a regex.
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 5 of the single-repo cutover runbook
(#143, #145, #149).
This is the last import. After it,
.gitmodulesholds two stanzas:uiandhelm-charts.MERGE WITH A MERGE COMMIT. NEVER SQUASH. NEVER REBASE.
A squash flattens 364 commits into one and destroys the import.
The revert trap. After you revert a merge commit, re-merging this branch
brings nothing back. Git considers it already merged. You must revert the revert
first.
Branched off
develop. Not stacked on anything.Credentials stripped from the history
GitHub push protection refused the import.
mint-ensemble-managercarries threecredentials in old commits. They were removed during the rewrite, not allowlisted.
src/config/mint-full-4dc148cc116b.jsonsrc.bak/config/config.jsonsrc/config/config.json***REMOVED***HEADalready carriedCHANGEMEplaceholders, so no working tree changes. Theimported tree is byte-identical to
mint-ensemble-managermain. Only history changed.Rotate these credentials. The source repository is public and still holds the
original commits. This PR stops the monorepo re-publishing them; it does not
un-publish them.
The import
git filter-repo --to-subdirectory-filter mint-ensemble-manager/ --mailmap <shared>,then
git merge --allow-unrelated-histories.Mainline only. 364 commits from
main. The runbook says 726; that is theall-refs count. 60 of the 78 refs hold commits that are not on
main, includingfive touched in 2026. The archive is their only copy.
main(f72fa9b) is one commit ahead of the old submodule pin (9cf3607). Thatcommit is mint-ensemble-manager#120,
the Phase 0.4 precondition.
The mailmap. Seven author identities collapse to two people.
Your Name <you@example.com>stays unmapped: two commits, both 2020-12-11, andthe committer field gives no signal.
Assertions
git log --follow -- mint-ensemble-manager/src/config/app.tsInitial Commit.diff -ragainstmint-ensemble-managermain: identical.git config core.hooksPathafternpm ciui-react/.husky/_scripts/deploy-hasura.sh --dry-rungit ls-files -s | awk '$1==160000'helm-charts,uinpm ci && npx tsc --noEmit && npm testAssertion 2 (image totality) now passes for all four. Anonymous
docker manifest inspectat this branch head5c6e32e:The
ensemble-managerGHCR package was created by this branch's first publishand is anonymously readable, so it needs no Actions-access grant. That is the
first commit in the cutover where one tag names the whole system.
The CI commit
.github/workflows/mint-ensemble-manager.yml. Node 24, matching the Dockerfile.npm ci,tsc --noEmit,npm test. No path filter, so every single-repo commitbuilds an image for every service.
No lint or format job. The service carries 475 eslint errors and 74
unformatted files.
eslint:fixandprettier:fixstay for manual use.The image moves from Docker Hub
mintproject/ensemble-managertoghcr.io/mintproject/ensemble-manager. It also deletesmint-ensemble-manager/.github/workflows/docker-publish.yml; GitHub readsworkflows only at the repository root.
5a — the cleanup commit
.husky/and thepreparescript.core.hooksPathholds one value for thewhole repository and
ui-reactholds it. Before this commit,npm ciinmint-ensemble-managerprinted a husky usage error and carried on.lint-staged: the dependency, the script and the config block. Its only callerwas the deleted hook.
.release-it.jsonandVERSION.release-itis not indevDependencies."private": true.5b — the skills commit
bump-image-tagreplacesdynamo-bump-from-branchandupdate-helm-image-tags,deleted in Phase 3. It resolves one tag from
mintproject/monorepoand writes itto
global.imageTag. Thecomponent_key|owner/repo|branchtable disappears.It aborts on the two silent failures the old scripts could not see:
components.<name>.image.tagwins over the global, and thechart pins all four by default. The script clears them.
mintproject/monorepodoes not exist on Docker Hub. The chart stillnames Docker Hub for
ui_reactandensemble_manager(mint#109), so the script refuses
to write until that is fixed.
It also proves every one of the four images is in GHCR at that tag before writing.
test-on-k8s-devwas written on the submodule model throughout. Rewritten for thesingle repository, with
ui-reactadded and the ingress hosts verified against thedev cluster on 2026-08-30.
Root skills end at two, plus
model-catalog-api/.claude/skills/run-e2e-hasura.5c — the CI infrastructure commit
.github/dependabot.yml. A first setup: nodependabot.ymlexists anywheretoday. Monthly, every entry grouped. The case for it is
github-actions—mint-ensemble-managerarrived pinned toactions/checkout@v2anddocker/login-action@v1.release-please. One version for the whole repository.release-as: 0.1.0andbootstrap-shacome off after the first release; withoutbootstrap-shathefirst changelog replays 700 commits from the source repositories.
passed and asserts the digest is unchanged. It does not rebuild.
Both release workflows are unused until Phase 8.
After merge
.gitmodulesholdsuiandhelm-charts.ensemble-managerGHCR package already exists, created by this branch'sfirst publish. No Actions-access grant is needed.