Skip to content

feat: FHIRPath 3.0.0 functions, instance selectors, and pathname() - #122

Merged
fjsj merged 17 commits into
mainfrom
fjsj/tackle-new-3.0.0-issues
Oct 9, 2026
Merged

fjsj merged 17 commits into
mainfrom
fjsj/tackle-new-3.0.0-issues

Conversation

@fjsj

@fjsj fjsj commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

This PR implements the seven FHIRPath 3.0.0 issues (spec). Each behavior follows the 3.0.0 text and its examples, and every spec example is a test.

Builds on #110 (merged): the repeat() limit, scalar-value string functions, the Date + hour error, and EqualityIndex. None of #110's behavior changes here; main is merged in with its last two commits.

Closes #111, closes #112, closes #113, closes #114, closes #115, closes #116, closes #117.

Changes

Issue Change
#111 matches(), matchesFull(), and replaceMatches() take an optional flags argument: i ignores case and m makes ^/$ match at line breaks. Other flags throw, and the analyzer reports a literal one (invalid-argument). The runtime and analyzer share one rule. The matches-flags-argument quirk family is removed, and its 10 fhirpath.js cases now pass. A custom EvaluateOptions.regex engine receives i/m.
#112 repeatAll() keeps duplicates. It shares MAX_REPEAT_ITEMS (10,000) with repeat(), so the spec's unsafe examples ('abc'.repeatAll(replace('a', 'A')), Questionnaire.repeatAll('item')) throw. The analyzer signature matches repeat()'s (unordered).
#113 duration() and difference() follow the spec's precision table. Weeks start on Sunday. Timezone offsets are normalized for hour and smaller precisions. The result is empty when an operand is less precise than the requested precision. A precision the operand types don't allow throws, and the analyzer reports a literal one.
#114 toDate(), toDateTime(), convertsToDate(), and convertsToDateTime() take a format for a String input. All the spec's required codes are supported, plus yy, M, d, h, H, m, s, MMM, and MMMM (English). Other inputs ignore the format. The analyzer reports a literal format that can't compile.
#115 pathname([short]) (forked work, reviewed). Navigation records an item's origin (parent, element, JSON index) only when the evaluation can reach a pathname() call. That includes vars, expression-function bodies, DTO columns, and instance-selector values. The benchmark shows no measurable cost with tracking on or off. Paths cover children(), descendants(), and extension(), plus resolve() into contained resources and Bundle entries. A new "Element paths" section in docs/api.md describes them.
#116 Instance selectors (forked work, reviewed). The change touches the lexer (: token), parser, AST, printer, evaluator, analyzer, and type-level parser. With a model, unknown types, unknown elements, wrong value types, and several values for a non-repeating element are errors at runtime and in the analyzer, through one shared rule. A choice element takes its key from the value's type (Observation { value: 5 'mg' } sets valueQuantity). Type inference gives a selector unknown[] and keeps the surrounding expression typed: the generated interfaces list required-binding codes that the runtime doesn't check in a built value (#133).
#117 combine(other, preserveOrder) and encode('ascii'). combine() keeps its sources' order whatever preserveOrder says. ascii gives one ? per Unicode scalar value above 127, so an emoji is one ?. decode('ascii') stays unsupported, as in the spec.

Decisions for review

  • combine(), union(), and | keep their sources' order, a deliberate spec deviation. The spec says "when preserveOrder is false, or not supplied, there is no expectation of order", and gives union() no order. The runtime appends in order, and the analyzer treats all three results as ordered, so x.combine(y).first() and (a | b).first() are not order-dependent errors. A new "Deviations from the specification" section in docs/conformance.md records this, and docs/static-checking.md links to it.
  • repeatAll() outputs only projected items, not the input, as repeat() does. fhirpath-rs testRepeatAllFiniteSequence expects 10.repeatAll(iif($this > 1, $this - 2, {})).count() = 6; this engine gives 5 (8, 6, 4, 2, 0). fhirpath-rs testRepeatAllBasic expects Patient.name.repeatAll('test') to succeed, but the spec lists that form as one that never terminates, and here it hits the limit.
  • duration()/difference() details the spec leaves open:
    • Only singular precision words are accepted ('years' throws), matching the spec's list.
    • A Date measured against a DateTime counts as a DateTime.
    • Components below the less precise operand are ignored.
    • At hour and smaller precisions, a DateTime with an offset against one without gives empty, as comparison does.
  • Date formats:
    • Two-digit years use the spec's suggested pivot: 00–49 is 20xx, 50–99 is 19xx, which matches the spec's '12-27'.toDate('MM-yy') → @2027-12.
    • The time zone name code z reads an IANA id in Area/Location form, or UTC/GMT, and takes the zone's offset at that date and time through Intl.DateTimeFormat. Abbreviations (PST) and names (Pacific Standard Time) don't match, even where ICU accepts them. A wall time a DST change repeats takes the earlier instant, as Temporal's compatible disambiguation does. A skipped wall time, or an offset that isn't whole minutes (local mean time, such as New York in 1850), gives empty.
    • A format with no year, a gap (a day without a month), h without a, or a zone code without an hour throws.
    • Month names and AM/PM markers are read in English, case-insensitive. Literal characters match exactly.
  • matchesFull() with m matches when any one line matches whole (^(?:pattern)$ with multi-line ^/$), as fhirpath.js expects. The extended R5 suite in reason-healthcare/rh expects the whole value instead; the spec's "completely matches" and its m definition pull both ways.
  • Instance selector input: an empty focus gives empty, as the spec says. So r4.evaluate("Coding { code: 'x' }") with no input returns []. The model has no minimum cardinalities, so a selector that omits a required element (e.g. Observation {:}) is not reported; Instance selectors: report missing required elements #124 tracks that. %factory stays deferred in the README.
  • Instance selector values match FHIR's primitive patterns. Each primitive a selector writes must match its type's regex from the R4 definitions, generated into src/r4/generated/primitive-patterns.ts and served by a new optional ModelProvider.valuePattern(). So unsignedInt { value: -1 }, Coding { code: ' a' }, HumanName { family: '' }, and Observation { effective: @2020-01-01T10:00 } (a time without seconds and a time zone) throw a FhirPathRuntimeError, and the static checkers report a literal one (invalid-value). Values read from data are checked too. The message names the element, type, and pattern, never the value, which may be patient data. The generator rewrites two things for a JS RegExp, and fails if a source pattern changes or a bare \s remains:
    • \s/\S use Java's whitespace set, the dialect of HAPI's validator. A JS \s also matches U+00A0 and U+3000, so a valid name such as Yamada Taro would fail string.
    • FHIR's own base64Binary pattern backtracks exponentially in a JS RegExp (a 120-character invalid value took over a minute), so it gets an equivalent that matches in linear time; a test checks every pattern stays linear.
  • Instance selector gap: backbone elements can't be built yet; fhirpath.js accepts BackboneElement { ... } there (Instance selectors: build backbone elements with BackboneElement { ... } #132).
  • pathname() roots:
    • A root with a resourceType starts the path with that type, with no indexer.
    • A root without one gives relative paths (given[0]), and the root itself gives no path.
    • resolve('#') now returns the root item itself.
  • Type-level cost: the API surface is +4.7% over its baseline and the full language +4.3% (limit 5%), up from +3.6% on Fix six behaviors found by the upstream test audit #110. The extra comes from four new function names in the generated unions and the selector tokenizer rule. A first parser-branch version of the selector rule reached +5.4%, so detection moved into the tokenizer (recorded in AGENTS.md). The budget is not raised.
  • Corpus gaps: no reference suite calls difference, duration, pathname, or repeatAll yet. corpus-audit.test.ts lists them next to convertsToLong; their unit tests and the generated function capabilities cover them.

Upstream validation

The official suites (FHIR/fhir-test-cases R4, R4B, R5, R6) have no 3.0.0 cases, and the HL7/FHIRPath repo hasn't changed these sections since 3.0.0. So every 3.0.0 case I could find in other engines' tests was run against this branch:

Source 3.0.0 cases Result
#110's audit (fhirpath.js, fhirpath-py, fhirpath-rs, Helios, HL7 Java core, Medplum, kotlin, HAPI) 37 34 pass; 3 are the fhirpath-rs repeatAll() differences above
Josh Mandel's fhirpath.zig and fhirpath.wasm tests 29 all pass
Extended R5 suite in reason-healthcare/rh 58 54 pass; the other 4 contradict the spec or their fixture, or are the matchesFull() question above
gofhir, Pathling, Ignixa, fhirpath.js pathname/selector tests ~280 pass except engine design differences and tests the 3.0.0 text contradicts

The cases the suite lacked are now in it: src/reference-crosschecks.test.ts (fhirpath-rs, Helios, fhirpath.zig) and test-data/crosschecks/fhirpath-3.0.0.json (179 cases from gofhir, Pathling, Ignixa, and fhirpath.js).

Then every evaluation the test suite makes for these features went through the analyzer the way strict evaluation runs it, comparing diagnostics and result types with the runtime. Together the two passes found these bugs, all fixed here:

  • combine()'s preserveOrder and pathname()'s short accepted any value as true; a non-Boolean now throws, as the analyzer already reported.
  • Element names on Object.prototype (Patient.constructor, Coding { constructor: 'x' }) crashed the runtime and the analyzer with a JS TypeError. This predates the PR, in the R4 model's element lookup; unescape('html') had the same problem with &constructor; (Fix six behaviors found by the upstream test audit #110).
  • unsignedInt { value: 5 } and positiveInt { value: 1 } were rejected, because the R4 model types their value as System.String.
  • duration() returned -0 for less than one period backwards.
  • The analyzer missed a Time measured against a Date, unsupported literal encode()/decode() formats and escape()/unescape() targets, and typed 2147483648 as Integer where the runtime gives a Long.
  • The analyzer read a repeat()/repeatAll() projection against the first round only, so QuestionnaireResponse.repeat(item | answer.item) was an unknown-element error. It now types $this over every round.

The remaining differences are the analyzer's usual strictness (unknown elements, wrong input types, and possibly-many values that the runtime turns into empty or tolerates), plus misses it can't see: an untyped env value, and gender.repeatAll(length()), whose first round is valid. Follow-ups found on the way, outside this PR: #127 (sort() asc/desc), #128 (min()/max() on dates and strings), #129 (ignored escapes), #130 (as(Quantity) drops Age), #131 (= and element ids), #132 (BackboneElement selectors).

Review loop

An independent review pass (Opus, read-only) found two blockers, both fixed: selector types claimed codes the runtime never checks (now unknown[], #133 for the code checks), and the patterns' JS \s rejected valid Unicode-spaced strings (now Java's whitespace). It also led to one shared loop for repeat()/repeatAll(), one systemTypeOfName() helper in place of three copies, and four text fixes. The second pass approved.

Validation

  • pnpm typecheck, lint, test (8,649 passed, 1,446 skipped), check:fhirpath, check:inference (2,116 precise, 0 conflicts), check:type-perf, coverage (statements 99.43%, branches 96.97%), build, and check:package (TypeScript 5.9.3 and 5.4.5) all pass.
  • The fhirpath.js corpus is now 2,306 passing and 1,363 skipped, with 236 quirk entries. docs/conformance.md has the new counts, which also include Fix six behaviors found by the upstream test audit #110's changes.
  • The demo has no AST switches. Its Monaco declarations are generated at build time and gitignored. I did not run the demo's own typecheck, because its dependencies aren't installed here.

🤖 Generated with Claude Code

fjsj added 5 commits October 8, 2026 22:49
…e formats, combine preserveOrder, encode ascii

- matches/matchesFull/replaceMatches take an optional flags argument (i, m);
  other flags are an error at runtime and for a literal in the analyzer (#111).
  The fhirpath.js matches-flags-argument quirk family is gone; its 10 cases run.
- repeatAll() keeps duplicates and shares repeat()'s 10,000-item limit (#112).
- duration() and difference() over Date, DateTime, and Time, with the spec's
  precision table, Sunday-start weeks, and offset normalization for hour and
  smaller precisions; the analyzer checks a literal precision (#113).
- toDate/toDateTime/convertsToDate/convertsToDateTime take an optional format
  for a String input (spec format codes; English month names; no 'z') (#114).
- combine(other, preserveOrder); the analyzer treats combine() as unordered
  unless preserveOrder is true. encode('ascii') replaces each character above
  127 with one '?' (#117).
pathname([short]) returns the path of each input item inside the input
resource, written with element names and indexers, such as
Observation.component[0].code[0].coding[0]. With short = true, elements
that are not arrays in the data or the model have no indexer.

Navigation, children(), descendants(), extension(), and resolve() record
each item's parent, element name, and JSON array position. Items that
pass through unchanged keep it; computed values and env values have none
and are left out. A resolved contained resource or Bundle entry of the
input has a path. A root without a resourceType gives relative paths.

Origins are recorded only when the evaluation can call pathname(): in
the expression, a vars body, or a host expression function body,
including a DTO column's own function table. Other evaluations create
the same items as before.

Closes #115
# Conflicts:
#	CHANGELOG.md
#	src/typed/corpus-audit.test.ts
#	src/typed/generated/corpus-audit.ts
#	src/typed/generated/full-language-perf.types.ts
`Coding { system: 'http://loinc.org', code: '8480-6' }` builds one FHIR value
from the current focus, and `Period {:}` builds one with no elements.

- Lexer: `:` is punctuation. Parser: a qualified identifier followed by `{` at
  term position is an `instance` node; the printer and fuzz round trip cover it.
- Runtime (src/engine/instance-selector.ts): empty focus gives empty, more than
  one item throws; each value runs against the focus in its own variable scope;
  empty values are left out. With a model, the type, element names, value types,
  and cardinality are checked, repeating elements become arrays, choice
  elements take the key of the value's type, FHIR primitives keep their
  `_field` siblings, and System values become FHIR JSON (quantities with UCUM
  codes). Without a model, any name builds a plain object.
- Analyzer: the same type and element rules (shared resolveInstanceType and
  acceptingElementType), reported as unknown-type, unknown-element,
  operand-type, and singleton-required.
- Type-level: Tokenize collapses each selector into one token; the named type is
  inferred when it requires nothing but resourceType, otherwise unknown[].
  API surface +0.3%, within budget.
….0-issues

# Conflicts:
#	src/typed/generated/full-language-perf.types.ts
Base automatically changed from fjsj/fix-new-upstream-found-issues to main October 9, 2026 13:15
fjsj added 5 commits October 9, 2026 10:16
combine(), union(), and | keep their sources' order in the analyzer, a
documented deviation from FHIRPath 3.0.0. The date format code z reads an
IANA time zone id and takes its offset at the date and time read.
…-issues

# Conflicts:
#	src/functions/filtering.ts
#	src/functions/functions-core.test.ts
#	src/functions/tree.ts
The upstream audit's 3.0.0 cases missing from the suite: nine fhirpath-rs
repeatAll() cases (three test the spec where fhirpath-rs differs) and three
Helios regex flag cases.
The 19 cases from Josh Mandel's hand-written fhirpath.zig tests for
duration(), difference(), regex flags, and repeatAll() that the suite did
not already have.
combine() and pathname() read their Boolean argument with the implicit-true
rule of singleton evaluation, so combine(x, 1) and pathname('yes') ran as if
given true. An argument of another type is an error (FHIRPath 3.0.0,
Functions), as the analyzer already reported.
fjsj added 4 commits October 9, 2026 11:15
…and fhirpath.js

176 cases from other engines' tests for the 3.0.0 functions and syntax that
this engine agrees with and the suite did not already have, with their
sources, commits, and licenses in test-data/crosschecks.
Running every evaluation the suite makes through the analyzer, the way
strict evaluation does, found these divergences and bugs:

- duration()/difference() between a Time and a Date or DateTime throws at
  runtime; the analyzer now reports it (operand-type), with or without a
  literal precision.
- encode()/decode() formats and escape()/unescape() targets come from one
  table per function; the runtime throws and the analyzer reports a
  literal one with the same message (decode('ascii') included).
- An integer literal past 32 bits is a Long at runtime (Decimal past 64
  bits); integerLiteral() types it for both, so the analyzer no longer
  says Integer.
- repeat() and repeatAll() run their projection on its own results; the
  analyzer now types $this over every round, so
  QuestionnaireResponse.repeat(item | answer.item) is no longer an
  unknown-element error, while an element no round has still is.
- Element names on Object.prototype (Patient.constructor, Coding
  { constructor: 'x' }) crashed the runtime and the analyzer with a JS
  TypeError; the R4 model now reads own keys only. unescape('html')
  likewise ignored &constructor;.
- unsignedInt { value: 5 } and positiveInt { value: 1 } were rejected:
  the R4 model types their value as System.String. selectorElement()
  gives a primitive's value the System type the runtime reads.
- duration() returned -0 for less than one period backwards.

The three cross-check cases left out because of these bugs are back.
A selector now checks each primitive value it writes against the FHIR
type's value pattern, so unsignedInt { value: -1 }, Coding { code: ' a' },
HumanName { family: '' }, and a dateTime with a time but no seconds or
time zone throw (FhirPathRuntimeError). The static checkers report a
literal that breaks its pattern (invalid-value). The message names the
element, type, and pattern, never the value, which may be patient data.

The patterns come from the regex extension in the R4 definitions,
generated into src/r4/generated/primitive-patterns.ts and served by the
new optional ModelProvider.valuePattern(). FHIR's base64Binary pattern
backtracks exponentially in a JS RegExp (a 120-character invalid value
took over a minute), so the generator writes an equivalent that matches
in linear time and fails if the source pattern changes.
Findings from the first review pass:

- Blocker 2: the R4 value patterns ran with JavaScript's \s, which also
  matches Unicode spaces (U+00A0, U+3000), so a valid FHIR string such as a
  Japanese name with an ideographic space failed `string`. The generator now
  rewrites \s and \S to Java's whitespace sets (the reference validator's
  dialect), in one JS_PATTERNS table with the linear base64Binary rewrite,
  and fails when a pattern still uses a bare \s.
- repeat() and repeatAll() share one loop (repeatProjection) and one limit
  check instead of two rules.
- systemTypeOfName() replaces the identical systemTwin() and
  systemTypeName() helpers and the inline copy in type-compat.ts.
- Text: the function-table JSDoc is back on `functions`; the changelog says
  "whether an element repeats" instead of "cardinality"; the linear-time
  test covers every generated pattern; the corpus-audit comment names the
  inventory's sources.
fjsj added 2 commits October 9, 2026 13:33
Blocker 1 from the review: InstanceState typed a selector as its named
interface, which lists the codes of required bindings, while the runtime
does not check those codes in a built value. Patient { gender: 'x' }.gender
was typed as the gender code union and returned 'x', and every type reaches
codes through extension. Settled with the user (2026-10-09): selectors are
typed unknown[] in this PR, the tokenizer still collapses them so the
surrounding expression stays typed, and #133 tracks checking required codes
so built values can be typed again.

README, CHANGELOG, AGENTS.md, and docs/static-checking.md say so; the
selector.required-elements capability is gone since both selector cases now
infer the same type.
The README and docs/conformance.md say the engine follows FHIRPath 3.0.0.
Spec links use 3.0.0's permanent address (hl7.org/fhirpath/STU3/) instead
of the 2.0.0 normative release (N1) or the unversioned URL, and name the
section instead of 2.0.0's "§11", since 3.0.0 does not number sections.
The README's gaps section lists the 3.0.0 behavior that does not work yet
(#127, #128, #129, #132) and links the ordering deviation. Code comments
that said "ballot STU" name 3.0.0 and its trial-use status.
3.0.0 does not number its sections, so comments, test names, manifest
notes, and the analyzer's diagnostics name the section instead
(spec "Singleton Evaluation of Collections", "FHIRPath strict evaluation").
Several numbers pointed at the wrong 2.0.0 section (§6.6.2 for &, §6.6.7
for date arithmetic, §6.1.3 for ~, §5.1 for tree navigation); the names
fix those. Comment links to the spec use 3.0.0's permanent address.

The ceiling() quirk note and the power() comment now say 3.0.0 differs
from the engine there, tracked with log(), overflow, and mixed-type
aggregates in #134, which the README's gaps list adds.
@fjsj
fjsj merged commit a36adcd into main Oct 9, 2026
3 checks passed
@fjsj
fjsj deleted the fjsj/tackle-new-3.0.0-issues branch October 9, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment