Repository navigation
feat: FHIRPath 3.0.0 functions, instance selectors, and pathname() - #122
Merged
Merged
Conversation
…e formats, combine preserveOrder, encode ascii - matches/matchesFull/replaceMatches take an optional flags argument (i, m); other flags are an error at runtime and for a literal in the analyzer (#111). The fhirpath.js matches-flags-argument quirk family is gone; its 10 cases run. - repeatAll() keeps duplicates and shares repeat()'s 10,000-item limit (#112). - duration() and difference() over Date, DateTime, and Time, with the spec's precision table, Sunday-start weeks, and offset normalization for hour and smaller precisions; the analyzer checks a literal precision (#113). - toDate/toDateTime/convertsToDate/convertsToDateTime take an optional format for a String input (spec format codes; English month names; no 'z') (#114). - combine(other, preserveOrder); the analyzer treats combine() as unordered unless preserveOrder is true. encode('ascii') replaces each character above 127 with one '?' (#117).
pathname([short]) returns the path of each input item inside the input resource, written with element names and indexers, such as Observation.component[0].code[0].coding[0]. With short = true, elements that are not arrays in the data or the model have no indexer. Navigation, children(), descendants(), extension(), and resolve() record each item's parent, element name, and JSON array position. Items that pass through unchanged keep it; computed values and env values have none and are left out. A resolved contained resource or Bundle entry of the input has a path. A root without a resourceType gives relative paths. Origins are recorded only when the evaluation can call pathname(): in the expression, a vars body, or a host expression function body, including a DTO column's own function table. Other evaluations create the same items as before. Closes #115
# Conflicts: # CHANGELOG.md # src/typed/corpus-audit.test.ts # src/typed/generated/corpus-audit.ts # src/typed/generated/full-language-perf.types.ts
`Coding { system: 'http://loinc.org', code: '8480-6' }` builds one FHIR value
from the current focus, and `Period {:}` builds one with no elements.
- Lexer: `:` is punctuation. Parser: a qualified identifier followed by `{` at
term position is an `instance` node; the printer and fuzz round trip cover it.
- Runtime (src/engine/instance-selector.ts): empty focus gives empty, more than
one item throws; each value runs against the focus in its own variable scope;
empty values are left out. With a model, the type, element names, value types,
and cardinality are checked, repeating elements become arrays, choice
elements take the key of the value's type, FHIR primitives keep their
`_field` siblings, and System values become FHIR JSON (quantities with UCUM
codes). Without a model, any name builds a plain object.
- Analyzer: the same type and element rules (shared resolveInstanceType and
acceptingElementType), reported as unknown-type, unknown-element,
operand-type, and singleton-required.
- Type-level: Tokenize collapses each selector into one token; the named type is
inferred when it requires nothing but resourceType, otherwise unknown[].
API surface +0.3%, within budget.
….0-issues # Conflicts: # src/typed/generated/full-language-perf.types.ts
combine(), union(), and | keep their sources' order in the analyzer, a documented deviation from FHIRPath 3.0.0. The date format code z reads an IANA time zone id and takes its offset at the date and time read.
…-issues # Conflicts: # src/functions/filtering.ts # src/functions/functions-core.test.ts # src/functions/tree.ts
The upstream audit's 3.0.0 cases missing from the suite: nine fhirpath-rs repeatAll() cases (three test the spec where fhirpath-rs differs) and three Helios regex flag cases.
The 19 cases from Josh Mandel's hand-written fhirpath.zig tests for duration(), difference(), regex flags, and repeatAll() that the suite did not already have.
combine() and pathname() read their Boolean argument with the implicit-true
rule of singleton evaluation, so combine(x, 1) and pathname('yes') ran as if
given true. An argument of another type is an error (FHIRPath 3.0.0,
Functions), as the analyzer already reported.
…and fhirpath.js 176 cases from other engines' tests for the 3.0.0 functions and syntax that this engine agrees with and the suite did not already have, with their sources, commits, and licenses in test-data/crosschecks.
Running every evaluation the suite makes through the analyzer, the way
strict evaluation does, found these divergences and bugs:
- duration()/difference() between a Time and a Date or DateTime throws at
runtime; the analyzer now reports it (operand-type), with or without a
literal precision.
- encode()/decode() formats and escape()/unescape() targets come from one
table per function; the runtime throws and the analyzer reports a
literal one with the same message (decode('ascii') included).
- An integer literal past 32 bits is a Long at runtime (Decimal past 64
bits); integerLiteral() types it for both, so the analyzer no longer
says Integer.
- repeat() and repeatAll() run their projection on its own results; the
analyzer now types $this over every round, so
QuestionnaireResponse.repeat(item | answer.item) is no longer an
unknown-element error, while an element no round has still is.
- Element names on Object.prototype (Patient.constructor, Coding
{ constructor: 'x' }) crashed the runtime and the analyzer with a JS
TypeError; the R4 model now reads own keys only. unescape('html')
likewise ignored &constructor;.
- unsignedInt { value: 5 } and positiveInt { value: 1 } were rejected:
the R4 model types their value as System.String. selectorElement()
gives a primitive's value the System type the runtime reads.
- duration() returned -0 for less than one period backwards.
The three cross-check cases left out because of these bugs are back.
A selector now checks each primitive value it writes against the FHIR
type's value pattern, so unsignedInt { value: -1 }, Coding { code: ' a' },
HumanName { family: '' }, and a dateTime with a time but no seconds or
time zone throw (FhirPathRuntimeError). The static checkers report a
literal that breaks its pattern (invalid-value). The message names the
element, type, and pattern, never the value, which may be patient data.
The patterns come from the regex extension in the R4 definitions,
generated into src/r4/generated/primitive-patterns.ts and served by the
new optional ModelProvider.valuePattern(). FHIR's base64Binary pattern
backtracks exponentially in a JS RegExp (a 120-character invalid value
took over a minute), so the generator writes an equivalent that matches
in linear time and fails if the source pattern changes.
Findings from the first review pass: - Blocker 2: the R4 value patterns ran with JavaScript's \s, which also matches Unicode spaces (U+00A0, U+3000), so a valid FHIR string such as a Japanese name with an ideographic space failed `string`. The generator now rewrites \s and \S to Java's whitespace sets (the reference validator's dialect), in one JS_PATTERNS table with the linear base64Binary rewrite, and fails when a pattern still uses a bare \s. - repeat() and repeatAll() share one loop (repeatProjection) and one limit check instead of two rules. - systemTypeOfName() replaces the identical systemTwin() and systemTypeName() helpers and the inline copy in type-compat.ts. - Text: the function-table JSDoc is back on `functions`; the changelog says "whether an element repeats" instead of "cardinality"; the linear-time test covers every generated pattern; the corpus-audit comment names the inventory's sources.
Blocker 1 from the review: InstanceState typed a selector as its named
interface, which lists the codes of required bindings, while the runtime
does not check those codes in a built value. Patient { gender: 'x' }.gender
was typed as the gender code union and returned 'x', and every type reaches
codes through extension. Settled with the user (2026-10-09): selectors are
typed unknown[] in this PR, the tokenizer still collapses them so the
surrounding expression stays typed, and #133 tracks checking required codes
so built values can be typed again.
README, CHANGELOG, AGENTS.md, and docs/static-checking.md say so; the
selector.required-elements capability is gone since both selector cases now
infer the same type.
The README and docs/conformance.md say the engine follows FHIRPath 3.0.0. Spec links use 3.0.0's permanent address (hl7.org/fhirpath/STU3/) instead of the 2.0.0 normative release (N1) or the unversioned URL, and name the section instead of 2.0.0's "§11", since 3.0.0 does not number sections. The README's gaps section lists the 3.0.0 behavior that does not work yet (#127, #128, #129, #132) and links the ordering deviation. Code comments that said "ballot STU" name 3.0.0 and its trial-use status.
3.0.0 does not number its sections, so comments, test names, manifest notes, and the analyzer's diagnostics name the section instead (spec "Singleton Evaluation of Collections", "FHIRPath strict evaluation"). Several numbers pointed at the wrong 2.0.0 section (§6.6.2 for &, §6.6.7 for date arithmetic, §6.1.3 for ~, §5.1 for tree navigation); the names fix those. Comment links to the spec use 3.0.0's permanent address. The ceiling() quirk note and the power() comment now say 3.0.0 differs from the engine there, tracked with log(), overflow, and mixed-type aggregates in #134, which the README's gaps list adds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements the seven FHIRPath 3.0.0 issues (spec). Each behavior follows the 3.0.0 text and its examples, and every spec example is a test.
Builds on #110 (merged): the
repeat()limit, scalar-value string functions, the Date + hour error, andEqualityIndex. None of #110's behavior changes here;mainis merged in with its last two commits.Closes #111, closes #112, closes #113, closes #114, closes #115, closes #116, closes #117.
Changes
matches(),matchesFull(), andreplaceMatches()take an optionalflagsargument:iignores case andmmakes^/$match at line breaks. Other flags throw, and the analyzer reports a literal one (invalid-argument). The runtime and analyzer share one rule. Thematches-flags-argumentquirk family is removed, and its 10 fhirpath.js cases now pass. A customEvaluateOptions.regexengine receivesi/m.repeatAll()keeps duplicates. It sharesMAX_REPEAT_ITEMS(10,000) withrepeat(), so the spec's unsafe examples ('abc'.repeatAll(replace('a', 'A')),Questionnaire.repeatAll('item')) throw. The analyzer signature matchesrepeat()'s (unordered).duration()anddifference()follow the spec's precision table. Weeks start on Sunday. Timezone offsets are normalized for hour and smaller precisions. The result is empty when an operand is less precise than the requested precision. A precision the operand types don't allow throws, and the analyzer reports a literal one.toDate(),toDateTime(),convertsToDate(), andconvertsToDateTime()take aformatfor a String input. All the spec's required codes are supported, plusyy,M,d,h,H,m,s,MMM, andMMMM(English). Other inputs ignore the format. The analyzer reports a literal format that can't compile.pathname([short])(forked work, reviewed). Navigation records an item's origin (parent, element, JSON index) only when the evaluation can reach apathname()call. That includesvars, expression-function bodies, DTO columns, and instance-selector values. The benchmark shows no measurable cost with tracking on or off. Paths coverchildren(),descendants(), andextension(), plusresolve()into contained resources and Bundle entries. A new "Element paths" section in docs/api.md describes them.:token), parser, AST, printer, evaluator, analyzer, and type-level parser. With a model, unknown types, unknown elements, wrong value types, and several values for a non-repeating element are errors at runtime and in the analyzer, through one shared rule. A choice element takes its key from the value's type (Observation { value: 5 'mg' }setsvalueQuantity). Type inference gives a selectorunknown[]and keeps the surrounding expression typed: the generated interfaces list required-binding codes that the runtime doesn't check in a built value (#133).combine(other, preserveOrder)andencode('ascii').combine()keeps its sources' order whateverpreserveOrdersays.asciigives one?per Unicode scalar value above 127, so an emoji is one?.decode('ascii')stays unsupported, as in the spec.Decisions for review
combine(),union(), and|keep their sources' order, a deliberate spec deviation. The spec says "when preserveOrder is false, or not supplied, there is no expectation of order", and givesunion()no order. The runtime appends in order, and the analyzer treats all three results as ordered, sox.combine(y).first()and(a | b).first()are notorder-dependenterrors. A new "Deviations from the specification" section in docs/conformance.md records this, and docs/static-checking.md links to it.repeatAll()outputs only projected items, not the input, asrepeat()does. fhirpath-rstestRepeatAllFiniteSequenceexpects10.repeatAll(iif($this > 1, $this - 2, {})).count() = 6; this engine gives 5 (8, 6, 4, 2, 0). fhirpath-rstestRepeatAllBasicexpectsPatient.name.repeatAll('test')to succeed, but the spec lists that form as one that never terminates, and here it hits the limit.duration()/difference()details the spec leaves open:'years'throws), matching the spec's list.'12-27'.toDate('MM-yy')→@2027-12.zreads an IANA id in Area/Location form, orUTC/GMT, and takes the zone's offset at that date and time throughIntl.DateTimeFormat. Abbreviations (PST) and names (Pacific Standard Time) don't match, even where ICU accepts them. A wall time a DST change repeats takes the earlier instant, as Temporal'scompatibledisambiguation does. A skipped wall time, or an offset that isn't whole minutes (local mean time, such as New York in 1850), gives empty.hwithouta, or a zone code without an hour throws.matchesFull()withmmatches when any one line matches whole (^(?:pattern)$with multi-line^/$), as fhirpath.js expects. The extended R5 suite in reason-healthcare/rh expects the whole value instead; the spec's "completely matches" and itsmdefinition pull both ways.r4.evaluate("Coding { code: 'x' }")with no input returns[]. The model has no minimum cardinalities, so a selector that omits a required element (e.g.Observation {:}) is not reported; Instance selectors: report missing required elements #124 tracks that.%factorystays deferred in the README.regexfrom the R4 definitions, generated intosrc/r4/generated/primitive-patterns.tsand served by a new optionalModelProvider.valuePattern(). SounsignedInt { value: -1 },Coding { code: ' a' },HumanName { family: '' }, andObservation { effective: @2020-01-01T10:00 }(a time without seconds and a time zone) throw aFhirPathRuntimeError, and the static checkers report a literal one (invalid-value). Values read from data are checked too. The message names the element, type, and pattern, never the value, which may be patient data. The generator rewrites two things for a JSRegExp, and fails if a source pattern changes or a bare\sremains:\s/\Suse Java's whitespace set, the dialect of HAPI's validator. A JS\salso matches U+00A0 and U+3000, so a valid name such asYamada Tarowould failstring.base64Binarypattern backtracks exponentially in a JSRegExp(a 120-character invalid value took over a minute), so it gets an equivalent that matches in linear time; a test checks every pattern stays linear.BackboneElement { ... }there (Instance selectors: build backbone elements with BackboneElement { ... } #132).pathname()roots:resourceTypestarts the path with that type, with no indexer.given[0]), and the root itself gives no path.resolve('#')now returns the root item itself.difference,duration,pathname, orrepeatAllyet.corpus-audit.test.tslists them next toconvertsToLong; their unit tests and the generated function capabilities cover them.Upstream validation
The official suites (FHIR/fhir-test-cases R4, R4B, R5, R6) have no 3.0.0 cases, and the HL7/FHIRPath repo hasn't changed these sections since 3.0.0. So every 3.0.0 case I could find in other engines' tests was run against this branch:
repeatAll()differences abovematchesFull()question abovepathname/selector testsThe cases the suite lacked are now in it:
src/reference-crosschecks.test.ts(fhirpath-rs, Helios, fhirpath.zig) andtest-data/crosschecks/fhirpath-3.0.0.json(179 cases from gofhir, Pathling, Ignixa, and fhirpath.js).Then every evaluation the test suite makes for these features went through the analyzer the way strict evaluation runs it, comparing diagnostics and result types with the runtime. Together the two passes found these bugs, all fixed here:
combine()'spreserveOrderandpathname()'sshortaccepted any value as true; a non-Boolean now throws, as the analyzer already reported.Object.prototype(Patient.constructor,Coding { constructor: 'x' }) crashed the runtime and the analyzer with a JSTypeError. This predates the PR, in the R4 model's element lookup;unescape('html')had the same problem with&constructor;(Fix six behaviors found by the upstream test audit #110).unsignedInt { value: 5 }andpositiveInt { value: 1 }were rejected, because the R4 model types theirvalueasSystem.String.duration()returned-0for less than one period backwards.encode()/decode()formats andescape()/unescape()targets, and typed2147483648as Integer where the runtime gives a Long.repeat()/repeatAll()projection against the first round only, soQuestionnaireResponse.repeat(item | answer.item)was anunknown-elementerror. It now types$thisover every round.The remaining differences are the analyzer's usual strictness (unknown elements, wrong input types, and possibly-many values that the runtime turns into empty or tolerates), plus misses it can't see: an untyped env value, and
gender.repeatAll(length()), whose first round is valid. Follow-ups found on the way, outside this PR: #127 (sort()asc/desc), #128 (min()/max()on dates and strings), #129 (ignored escapes), #130 (as(Quantity)dropsAge), #131 (=and element ids), #132 (BackboneElement selectors).Review loop
An independent review pass (Opus, read-only) found two blockers, both fixed: selector types claimed codes the runtime never checks (now
unknown[], #133 for the code checks), and the patterns' JS\srejected valid Unicode-spaced strings (now Java's whitespace). It also led to one shared loop forrepeat()/repeatAll(), onesystemTypeOfName()helper in place of three copies, and four text fixes. The second pass approved.Validation
pnpm typecheck,lint,test(8,649 passed, 1,446 skipped),check:fhirpath,check:inference(2,116 precise, 0 conflicts),check:type-perf,coverage(statements 99.43%, branches 96.97%),build, andcheck:package(TypeScript 5.9.3 and 5.4.5) all pass.🤖 Generated with Claude Code