Blog · Standards

Where the standard stops

In November 2026 we measured five weaknesses in the FHIR rules and in the SUSHI tool and filed them with HL7. The FHIR community's answers draw a line that matters for every data transfer.

What we measured

We checked five points where the FHIR specification or its tool SUSHI guarantees less than users often assume. Measured with SUSHI 3.21.0 and the HL7 Validator 6.10.4, on synthetic profiles and instances, no patient data.

  1. A profile can bind a value set, but it cannot state which source value a code must have come from. Two senders can conform to the same profile and deliver the same code for different facts. The two extensions that link a binding to a ConceptMap are informational only: no conformance meaning, no validator behaviour.
  2. With extensible bindings, the decision “no applicable concept” is taken at the sender. It is not in the instance. The validator issues the same warning for an outside code whether it is justified or not.
  3. Obligations have 69 codes, but none that says: this element is populated from a named source system through a declared mapping.
  4. With its dedicated rules, SUSHI checks that a derived profile only tightens its parent. The same weakening through a caret path passes without error or warning, in four measured cases. The HL7 Validator rejects all four.
  5. The FSH specification does not say, in its Caret Paths section, that the constraining rules apply there too, and it uses meaningWhenMissing as an example although R4 forbids it in profiles.

Filed on 16 November 2026: one bug report in the SUSHI project (FHIR/sushi #1663), four change requests with HL7 (FHIR-59871 to FHIR-59874). The reproduction material is public: github.com/fmatten/fhir-profile-constraint-checks.

What the community answered

The answers came the same day, on chat.fhir.org and in the SUSHI project, from the people who carry the standard and the tool. We quote verbatim because the wording settles the matter.

On point 4, Chris Moesel, maintainer of SUSHI, closed the report as “working as-designed”: SUSHI is not a validator, it checks the common path only, validation belongs in the IG Publisher. Lloyd McKenzie in the same vein: “So long as the issue is picked up at some point in the build cycle, I think we're ok?”

On points 1 to 3, Grahame Grieve wrote: “How can a profile author make a rule for what is not in scope of the agreements the community has?” and “in our context, this sounds out of scope”. Lloyd McKenzie: “The profile represents what's standardized. What is mapped from the internal data representation represents what isn't standardized.” And on whether implementations get mappings wrong: “absolutely. Is there anything the standard can do about it - not really.” John Moehrke pointed out that a value set should be precise enough for differences at the source to remain visible at the target.

The discussion is public (links below). The four change requests are with the work groups; we do not expect the specification to change.

What it means

This is not a rejection, it is a boundary, and an honest one. A profile governs which instances are acceptable. Whether an acceptable code is the right one for the fact lies outside the standard, by design. Two laboratories can map the same analyser flag to H and to N; both instances pass validation against the same profile, and no computable rule lets the receiver notice.

This does not devalue the validator. It checks what is checkable: structure, cardinalities, bindings, invariants, and it reliably catches the cases from point 4. It does not ask whether the meaning of the source has arrived, because the source is not part of its check. We described this in “What FHIR Validation Does NOT Check”; the community has now confirmed it in its own words.

We take two practical hints from the discussion. First: the source value can be carried today, as a second Coding with userSelected = true, plus the text the user saw. Second: the two extensions for permitted values and mappings have existed for years but, according to Lloyd McKenzie, are not used in production, because a standard rarely has the authority to prescribe internal mappings.

What we do with it

Whether two senders map alike, whether an interval became an instant, whether a negation vanished on the way: no target-side validation answers that, only a comparison of source and target on an agreed path. That is the SILD audit. It does not replace the validator; it starts behind it, with a different question.

For profile authors, the recommendation from “What Your FHIR Profile Does Not Say” stands: what the contract does not fix is not agreed. Anyone who wants mappings to stay comparable across organisational boundaries needs a second layer for that, explicitly agreed, and a measurement that checks it.

Evidence

Related posts: What Your FHIR Profile Does Not Say · What FHIR Validation Does NOT Check

About ISCaD

In the SILD audit we review one defined data transfer against review rules agreed in writing — using released test data or effectively anonymised data, with a report and a findings list. If you want to know what arrives on your transfer: see the SILD audit.

The formal foundation is documented at aion-clinical.eu; the software behind the measurement method is licensed commercially — enquiries to licensing@iscad-it.de.

← Back to the blog