Skip to content
This repository was archived by the owner on Aug 30, 2026. It is now read-only.

Point thread and task indicator columns at modelcatalog_standard_variable - #15

Merged
mosoriob merged 1 commit into
mainfrom
fix/106-response-variable-standard-variable-fk
Aug 30, 2026
Merged

Point thread and task indicator columns at modelcatalog_standard_variable#15
mosoriob merged 1 commit into
mainfrom
fix/106-response-variable-standard-variable-fk

Conversation

@mosoriob

Copy link
Copy Markdown
Collaborator

Fixes the Variables step of the thread wizard, which cannot save any indicator.
See mintproject/monorepo#106.

The defect

VariablesStep writes a standard variable id — a URI such as
https://w3id.org/okn/i/mint/DRAWDOWN — to thread.response_variable_id.
The foreign key referenced public.variable, whose ids are bare CSDMS-style
names such as drawdown. At TACC 0 of 668 standard variable ids are a valid
variable.id, so every save failed:

ApolloError: Foreign key violation. insert or update on table "thread"
violates foreign key constraint "thread_response_variable_id_fkey"

Why the constraint moves, not the client

The read side already keys on standard variable ids (ModelsStep, the picker
itself). public.variable is frozen — etl/load.py writes modelcatalog_*
tables only, and nothing has written variable since the DYNAMO migration — so
mapping the client onto it would tie the feature to a table that cannot grow.

What the migration does

  • Drops the four indicator constraints (thread and task × response and
    driving), migrates the data, then re-adds them against
    modelcatalog_standard_variable. The drop comes first because the old
    constraint refuses the new ids.
  • Maps existing values by label, the only bridge between the two id schemes.
    A value maps only when exactly one standard variable carries that label;
    62 labels are shared by up to 6 standard variables, so those are nulled rather
    than guessed. A wrong id shows a silent No models found.; a null shows the
    step as unset.
  • Moves the four variable array relationships to
    modelcatalog_standard_variable. Without this the migration leaves four
    inconsistent metadata objects — observed live, see below.

Verified on the MicroK8s dev cluster

Seeded four cases, ran up.sql, then down.sql, then up.sql again:

stored value after up.sql
air__daily_max_of_temperature (unique label) .../AIR__DAILY_MAX_OF_TEMPERATURE
soil_water__volume_fraction (2 standard variables) NULL
crop__planting_end_time (no standard variable) NULL
total_water_storage (the one real row) .../TOTAL_WATER_STORAGE

All four constraints report confrelid = modelcatalog_standard_variable.
down.sql restores the mapped values to their labels and the constraints to
public.variable; values that up.sql nulled stay null, which is lossy by
construction and documented in the file.

Metadata after the migration and the relationship move:
get_inconsistent_metadatais_consistent: true. thread.response_variable
resolves to modelcatalog_standard_variable and returns the label. Writing a
URI now succeeds; writing the bare drawdown is now the one refused.

Data cost

At TACC one row: thread nzGQkhtRsudfSEHkDGct, whose total_water_storage
maps uniquely. No task carries either column. ISI is unmeasured, which is why
the mapping exists rather than a blanket null.

Follow-up, not in this PR

The Ensemble Manager validates response_variables against the variable table
on POST /v1/problemStatements/*/subtasks (subTasksService.ts:209,219). After
this lands that route rejects standard variable URIs. ui-react does not call
it — it writes threads through Hasura. Filed separately.

…standard_variable

The Variables step writes a standard variable id -- a URI -- while the foreign
key referenced the legacy public.variable table, whose ids are bare CSDMS-style
names. At TACC 0 of 668 standard variable ids are a valid variable.id, so every
save failed with a foreign key violation.

The read side already keys on standard variable ids, and public.variable is
frozen: the ETL writes modelcatalog_* tables only. So the constraint moves
rather than the client.

Existing values migrate by label, the only bridge between the two id schemes.
Labels shared by several standard variables are nulled instead of guessed.

Metadata moves with the constraint: the four array relationships on variable
would otherwise go inconsistent, as observed on the dev cluster.

Refs mintproject/monorepo#106
@mosoriob
mosoriob merged commit 06492bd into main Aug 30, 2026
5 checks passed
@mosoriob
mosoriob deleted the fix/106-response-variable-standard-variable-fk branch August 30, 2026 00:36
mosoriob added a commit to mintproject/mint that referenced this pull request Aug 30, 2026
The chart asked for ghcr.io/mintproject/graphql-engine:latest, but the
graphql_engine workflow publishes only :main and :<full-sha>. It has never
pushed :latest, so that tag is a stale image no merge updates -- it matches
none of the last six commits on main, and carries neither the indicator FK
migration (mintproject/monorepo#106) nor the cascade migration
(mintproject/monorepo#99). pullPolicy: Always cannot help when the tag itself
never moves.

Pin 06492bd, the merge of mintproject/graphql_engine#15. Migrations are baked
into this image, so the tag decides which schema a deployment can reach at all.

Same treatment model_catalog_api got in #106.
mosoriob added a commit to mintproject/monorepo that referenced this pull request Aug 30, 2026
Picks up 1771200018000_response_variable_standard_variable_fk and the metadata
move of the four array relationships onto modelcatalog_standard_variable, merged
as mintproject/graphql_engine#15. Also picks up the cascade migration (#14),
which the pointer had not yet caught up with.
mosoriob added a commit to mintproject/monorepo that referenced this pull request Aug 30, 2026
* chore: bump graphql_engine to main for the indicator FK migration

Picks up 1771200018000_response_variable_standard_variable_fk and the metadata
move of the four array relationships onto modelcatalog_standard_variable, merged
as mintproject/graphql_engine#15. Also picks up the cascade migration (#14),
which the pointer had not yet caught up with.

* fix(ui-react): reload the executions after fetching results (#110)

Registration writes the execution_result rows server-side, but the results
table renders the executions held in MintThread's state, and
handlePublishResults only refetches GetThreadExecution -- the pipeline and
its counters, not the execution rows that carry the results.

So a successful fetch left the table reading "No results available" until
the user pressed Reload. That is #110's own symptom, still present on the
first click.

MintResults now reloads the current page of executions after a successful
publish, through the same onFetchRuns path the Reload button uses. A failed
publish still reloads nothing.

Verified live against TACC on thread jx8ibyilmsmg1pvt: the registered row
was deleted to restore the "before" state, and one click on Fetch results
rendered the output with no Reload.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant