Skip to content

Fixing the required tag cleanup - #1438

Merged
VisLab merged 5 commits into
hed-standard:mainfrom
VisLab:validation_cleanup
Sep 30, 2026
Merged

VisLab merged 5 commits into
hed-standard:mainfrom
VisLab:validation_cleanup

Conversation

@VisLab

@VisLab VisLab commented Sep 30, 2026

Copy link
Copy Markdown
Member

No description provided.

Copilot review of PR hed-standard#1437. A sidecar string is a fragment of a row, so the sidecar stage no longer runs the required-tag check on it; required tags split between a sidecar string and the HED column validate clean. The unique-tag check still runs there, since a unique tag repeated inside a fragment is repeated in the row and hed-tests has a sidecar-only TAG_NOT_UNIQUE case. run_full_string_checks takes required_tags and unique_tags in place of whole_row; the HED-cell pass passes both False.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The public method signature introduces breaking and silently changed behavior for existing callers.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Updates validation so required tags are checked only after complete row assembly.

Changes:

  • Adds independent required/unique-tag validation controls.
  • Defers sidecar and cell required-tag checks.
  • Adds regression coverage for fragments and assembled rows.
File Description
hed/​validator/​hed_validator.py Adds separate row-tag check flags.
hed/​validator/​spreadsheet_validator.py Defers fragment-level row checks.
hed/​validator/​sidecar_validator.py Skips required-tag checks for sidecar fragments.
tests/​validator/​test_hed_validator.py Tests the new validation flags.
tests/​validator/​test_staged_validation.py Tests sidecar/cell assembly behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread hed/validator/hed_validator.py Outdated
SchemaComparer iterated sets of row keys, extras section names, and attribute names, so the order of the 'Row ... missing in first schema' lines and of an entry's attribute changes changed with the hash seed from run to run. The changelog this feeds (hed-schemas PRERELEASE_CHANGES.md) is a generated, committed file, so every regeneration showed spurious reorderings. The three loops now run in sorted order. Tests pin the order for one-sided rows through _compare_dataframes and through gather_schema_changes.
Copilot review of PR hed-standard#1438. required_tags and unique_tags on HedValidator.run_full_string_checks can no longer be passed positionally, so a positional call cannot silently change meaning. The whole_row flag they replace lived on main for one day between PRs hed-standard#1437 and hed-standard#1438 and never reached a release, so no compatibility parameter is kept for it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Two non-assembly paths incorrectly suppress definite unique-tag violations.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Full-string validation drops definite uniqueness violations

hed/​validator/​spreadsheet_validator.py:400

With full_string=True, this method is explicitly serving a caller that will not assemble rows, so disabling unique_tags drops a definite TAG_NOT_UNIQUE violation. A duplicate unique tag inside this fragment remains duplicated in any assembled row; keep only the required-tag check disabled. The normal staged path uses full_string=False, so this will not duplicate stage 4 reports.

This issue also appears on line 411 of the same file.

Copilot re-review of PR hed-standard#1438. The cell pass that runs when no assembly follows, and validate_hed_column(full_string=True), now skip only the required-tag check. A unique tag repeated inside one cell is repeated in every assembled row, so the check cannot false-alarm there; it is reported once per distinct string with a row count. The two paths never run together with assembly, so nothing is reported twice.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The behavior is internally consistent, documented, and covered by focused regression tests.

Review effort: Balanced
Findings: None

@VisLab
VisLab merged commit b62bf64 into hed-standard:main Sep 30, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants