- What is this implementation?
- Quick Start
- Overview
- Test Kit
- Adaptation
- Security & Privacy Considerations
- Performance Considerations
- Support
DHIS2 Tracker programs can be configured to support case-based disease surveillance and EMR functions such as patient health record. Often laboratory results need to be entered into such programs to inform the decision-making of health professionals. In contrast to manual data entry, sending electronically the laboratory results to DHIS2 helps speed up this decision-making, eliminate manual transcribing errors between systems, as well as improve the information availability to all health and management staff.
A Laboratory Information System (LIS) is typically the primary source of laboratory results destined to other information systems. With the generous support of the U.S. Centers for Disease Control and Prevention, HISP Centre developed this reference implementation to demonstrate the electronic transmission of lab results from a LIS to DHIS2, improving the timeliness and quality of such results in Tracker programs.
As defined in Laboratory Information Systems Project Management: A Guidebook for International Implementations, a LIS is a computer-based information management systems created specifically for laboratories, to support workflow, track data from the start to the end of the testing process, store data, and provide correct and complete information to laboratory staff, managers, and customers in a timely manner allowing for decision making by clinicians, epidemiologists and other stakeholders.
This reference implementation demonstrates, in a sandbox environment, the import of laboratory reports from a LIS into a DHIS2 Tracker program designed for case-based disease surveillance. The import is accomplished by (1) fetching laboratory diagnostic reports from a mock LIS conforming to the HL7 Laboratory FHIR Implementation Guide, (2) transforming the diagnostic reports into Tracker events, and then (3) transmitting the events to the DHIS2 Web API.
The data exchange between the health information systems is mediated thanks to a DHIS2-driven Interoperability Layer (IOL) component which also bridges the differences between the FHIR and DHIS2 resources. These differences can be divided into:
- structural due to the mismatch between the DHIS2 and FHIR JSON formats, and
- semantic due to the terminology mismatch where the LIS employs LOINC, whereas the DHIS2 Tracker program has its own custom metadata codes for the data elements and option set values capturing the laboratory tests and results.
The expected audience of this reference implementation is enterprise and solution architects, integrators, and implementation engineers. This is a self-contained, working example meant to technically guide you in developing your own integration between an LIS and DHIS2. It SHOULD NOT be used directly in production without adapting it to your local context. Prior to studying the software artefact, it is important to read the implementation guidance on lab interoperability.
- From the machine where you intend to run the reference implementation:
- Install a recent version of the Java Development Kit to be able to build the IOL
- Install Node so that you can run Yarn
- Install Yarn to facilitate the building and running of the project
- Install Docker Desktop which provides the tooling required to bring up the sandbox environment
- Install the Git client which is a source code management tool
- Within a terminal, run the command shown next to download the reference implementation repository:
git clone https://github.com/dhis2/reference-dhis2-tracker-lab-result-integration.git
- Change the current directory in your terminal to
reference-dhis2-tracker-lab-result-integrationand run:Running these commands will:yarn install --frozen-lockfile yarn build yarn start
- Install the test development dependencies
- Build the IOL application
- Stand-up the components which include:
- a DHIS2 instance reachable from http://localhost:8080/
- a mock LIS which is a HAPI FHIR server reachable from http://localhost:8081/
- the IOL running as a background process
- From your browser, type the following in the address bar to open the enrollment form for the case surveillance program: http://localhost:8080/apps/capture#/new?orgUnitId=DiszpKrYNg8&programId=N07iEegH3Hw. Alternatively, follow these steps:
- Open the Capture app from the DHIS2 dashboard in your local DHIS2 instance on http://localhost:8080/
- Expand the Program drop-down box and pick Case Surveillance
- Expand the Organisation unit down-down box and type Ngelehun CHC before proceeding to select it
- Press the Create new person button
- In the enrollment form, expand the Initial Diagnosis drop-down box and pick
Ebola - Press the Save person button, located at the bottom of the form
- From the enrollment dashboard, click on New Lab request event
- Choose a date from the Date of data entry date picker
- Insert a random identifier like
123456in the Specimen ID field (the specimen ID must always be unique across all lab request events) - Press the
Completebutton, located at bottom of the form
- From a terminal, run
yarn simulateto simulate the laboratory instrument. Wait until the command completes before moving on to the next step. - Wait at least a minute or two before refreshing the DHIS2 enrollment dashboard in order to give time for the LIS laboratory report to be imported into DHIS2. After the refresh, an event should appear under the Lab report section of the enrollment dashboard. Try refreshing the page a couple of more times if the event does not show up.
- Finally, open the lab report event to view the laboratory diagnosis confirming or refuting the initial Ebola diagnosis.
The subsequent diagram conceptualises the architecture of this reference implementation:
What follows is a brief overview of the architectural components:
The role assigned to DHIS2 in this reference implementation is that of an integrated surveillance and outbreak response system based on the Africa CDC Toolkit for Surveillance and Outbreak Response. The DHIS2 instance shipped with the sandbox is preconfigured with Tracker programs covering case surveillance and contact tracing. The laboratory result integration is focused on the case surveillance program which has its workflow depicted below:
The following sections drill down into the stages that are relevant to the lab report integration.
A disease surveillance case in DHIS2 starts with enrolment of a person having a suspect disease. The surveillance officer needs to select the initial diagnosis before they can enrol the person into the program. In the enrolment form shown below, the initial diagnosis can be either cholera, Ebola, or mpox.
The lab request stage is used to report the laboratory order and link the LIS laboratory result to the surveillance case. The link is established thanks to the specimen ID which is entered into this stage's data entry form shown next:
The mandatory specimen ID field shown is expected to be unique for each lab request, even across cases. In other workflows, instead of the specimen ID, alternative or additional unique linking identifiers could be required such as the patient name or the case ID, each with their own tradeoffs.
Completing the lab request form does not trigger a laboratory order. It is assumed that the laboratory test itself is ordered at a prior point in the overall disease surveillance workflow (e.g., during initial clinical diagnosis). However, to facilitate testing and demoing, accompanying the reference implementation is a lab instrument simulator that fetches the completed lab requests of in-progress cases from DHIS2, generates corresponding laboratory reports, and transmits the reports to the mock LIS.
After the lab request is the lab report program stage. As described in the Interoperability Layer section, the IOL populates this stage with the results originating from the LIS. That is, automatically, a lab result is imported into the ongoing case when a laboratory report that has a specimen ID linking it to the lab request in DHIS2 becomes available in the LIS. The outcome is a completed lab report data entry form, like the following, for the surveillance officer to review:
In this illustration, a case can have multiple lab requests and reports but a lab request can only have a single lab report. Lab report updates represent corrections or amendments to the original laboratory report. The lab result status change is reflected in the event notes section like what is presented here:
As part of the lab report integration, it is DHIS2 itself that drives the transformation and terminology mapping in the IOL, allowing the lab report to be imported into DHIS2. More concretely, the FHIR-to-DHIS2 JSON transformation script is kept in the DHIS2 data store which the IOL retrieves and executes with DataSonnet. DataSonnet is a JSON-extended template that lends well to JSON-to-JSON transformations. The subsequent screenshot demonstrates the DataSonnet script as viewed from the DHIS2 data store app:
In terms of terminology mapping, DHIS2 binds the data elements and option set values to lab terminology via attributes. For example, the following option set value config maps the LOINC code LA6576-8 to the option set value POSITIVE:
The DHIS2 implementer benefits from having the terminology mapping driven by DHIS2 because it permits the implementer to revise the LOINC-to-DHIS2 code mappings without ever leaving DHIS2. Moreover, an implementer proficient in both DataSonnet and the DHIS2 Web API, can adjust the transformation script in the DHIS2 data store when the lab result program stage or the LIS API is modified. The overall implication is that the DHIS2 implementer does need to enlist the IOL technical team when the mapping or transformation rules change.
The LIS is the source of the lab reports in the DHIS2 case surveillance program. In the real world, one or more laboratory instruments would run tests on the specimen and then report their results to the LIS for storage and analysis. However, for this reference implementation, a simulator is used instead to fake the results and transmit them to a mock LIS. These results are in turn read by the IOL as described in the next section.
The mock LIS is powered by HAPI FHIR: a popular open-source server implementation of FHIR. FHIR (Fast Healthcare Interoperability Resources) is a modern, adaptable health data exchange standard that allows us to keep the integration decoupled from any particular LIS interface.
The FHIR server is configured to conform to the universal Laboratory Report Implementation Guide. The guide is still in draft stage at the time of writing. Nevertheless, it was selected to represent the lab report exchange thanks to its broad scope due to the participation of experts from several countries, projects, and initiatives.
The IG profiles several FHIR resources though the following are exchanged with DHIS2 in this integration:
- Specimen: holds the specimen ID and the date the specimen was received at the lab
- Observation: contains the LOINC codes or free-form text, when LOINC codes are unavailable, identifying the test carried out and its result
- Patient: the test subject which can be anonymous, safeguarding patient data
- DiagnosticReport: bundles together the specimen, observation, and patient resources
The interoperability layer (IOL) is a low-code and customisable Apache Camel background application running inside a Java Virtual Machine (JVM) that bridges the LIS diagnostic report to the DHIS2 lab report program stage. Its processing is broadly broken down in the following steps:
-
The application routinely fetches active enrollments from DHIS2 having program ID
N07iEegH3Hw(i.e., case surveillance program) with the subsequent GET HTTP call:.../api/tracker/enrollments?program=N07iEegH3Hw&status=ACTIVE&fields=enrollment,events. -
If there are active cases, the IOL proceeds to:
-
Fetch from DHIS2:
- data element codes that have LOINC code attributes present using the GET HTTP call
.../api/dataElements?LqVVfNVy594:!null&fields=code,attributeValues - option set value codes have LOINC code attributes present using the GET HTTP call
.../api/options?LqVVfNVy594:!null&fields=code,attributeValues - the DataSonnet script from the data store using the GET HTTP call
.../api/dataStore/iol/diagnosticReportTransformScript
- data element codes that have LOINC code attributes present using the GET HTTP call
-
Search for events within the fetched active cases such that the event program stage ID is equal to
N07iEegH3Hw(i.e., the lab request program stage) and the status is equal toCOMPLETED. -
For each lab request, extract its specimen ID and attempt to retrieve the corresponding lab report within the enrollment matching the specimen ID. If the corresponding lab report within the case is retrieved, then this means that a diagnostic report for the lab request already exists in the LIS.
-
Record the
updatedAttimestamp of the DHIS2 lab report should one exist. In the next step, the IOL uses this timestamp to check whether the corresponding LIS diagnostic report was updated since the last time it was processed. -
Search for final, amended, appended, or corrected diagnostic reports by the lab request specimen ID in the LIS. A key constraint in the reference implementation is that the specimen ID is unique across laboratory orders so the IOL assumes that the LIS returns at most a single diagnostic report for a given specimen ID. The IOL behaviour is undefined when multiple diagnostic reports are in the search results. The GET HTTP call to search the reports is
.../fhir/DiagnosticReport?status=final,amended,appended,corrected&specimen.accession=[specimenId]&_include=DiagnosticReport:result&_include=DiagnosticReport:specimen&_lastUpdated=gt[labReportUpdatedAt]where:[specimenId]is substituted with the lab request specimen ID, and[labReportUpdatedAt]is substituted with theupdatedAtof the most recent lab result for[specimenId]. The IOL defaults[labReportUpdatedAt]to0000-01-01if no lab results exist to indicate that the diagnostic report should be retrieved regardless of when its was updated.
-
Transform the diagnostic report, if found, into a DHIS2 event using the DataSonnet script fetched in step 2ia and map the LOINC codes into data element and option set value codes by looking up the mappings downloaded from step 2ib and 2ic.
-
Import the event into DHIS2 with an HTTP POST sent to the endpoint
.../api/tracker?async=false&importStrategy=CREATE_AND_UPDATE&dataElementIdScheme=CODEwhere the:asyncquery parameter is set tofalseso that the event is imported synchronously leading to any import errors being reported and logged immediately.importStrategyquery parameter is set toCREATE_AND_UPDATEso that lab report event is updated should one exist.dataElementIdSchemeis set toCODEsince the transformation result from step 6 references the data elements by code.
-
The IOL exposes its metrics through JMX. A JMX client like VisualVM can be used to observe these metrics, however, the IOL comes bundled with Hawtio so that the system operator can easily monitor and manage the application's runtime operations without prior setup.
From the Hawtio web console, apart from browsing application logs, the system operator can manage Camel routes and endpoints, check the application health status, collect CPU and memory diagnostics, as well as view application settings:
You can log into the Hawtio console locally from http://localhost:9070/management/hawtio using the username user. The password is printed in the IOL logs when it is booting up for the very first time.
The IOL configuration is expressed through YAML files, Java properties files, or command-line arguments. The subsequent table lists common parameters that can be configured in the IOL:
| Parameter Name | Description |
|---|---|
| run.interval | Quartz expression denoting the schedule for polling DHIS2 lab requests |
| dhis2.api.url | Web API base path of the DHIS2 server |
| dhis2.api.username | Username of the DHIS2 Web API user. Required when not using PAT authentication |
| dhis2.api.password | Password of the DHIS2 Web API user. Required when not using PAT authentication |
| dhis2.api.pat | PAT of the DHIS2 server Web API user. Required when not using basic access authentication |
| dhis2.api.readTimeoutMs | Time to wait for a web response from DHIS2 before giving up |
| dhis2.loincCodesAttribute.id | ID of the attribute capturing the LOINC code |
| dhis2.program.id | ID of the DHIS2 Tracker program holding |
| dhis2.program.specimenDataElement.id | ID of the specimen data element holding the specimen ID |
| dhis2.program.labRequestProgramStage.id | ID of the DHIS2 lab request program stage |
| dhis2.program.labReportProgramStage.id | ID of the DHIS2 lab report program stage |
| lis.api.url | URL pointing to the LIS server |
The test kit is composed of end-to-end automated tests and scripts that simulate the laboratory instrument sending its results to the LIS. Ensure that you run yarn install from your terminal prior to running the end-to-end tests or simulating the laboratory instrument.
The end-to-end tests are located in the tests project directory. Playwright is the end-to-end test runner. Execute the following to have Playwright run the tests:
yarn testThe tests depend on the services as declared in the docker-compose.yml config. Playwright will automatically bring up the Docker containers if they are unavailable before running the tests.
The scripts creating FHIR diagnostic reports in the LIS are located in the tests/create-fake-lab-diagnostic-report-collection project directory path. Bruno is the API client that runs these scripts. Execute the following to simulate the laboratory instrument sending diagnostic reports to the mock LIS:
yarn simulateThe DHIS2-LIS reference implementation needs to be adapted to fit your local needs before it can be piloted. What follows are typical places where one would want to customise in their integration.
The DHIS2 metadata needs to be localised during customisation. This includes the organisation units, data elements, option sets, attributes capturing the laboratory terminology, and the Tracker program itself. Enrol in the DHIS2 online academies if you want to learn how to configure DHIS2.
Notably, besides metadata, the script within the DHIS2 data store used to transform the lab reports within the IOL would likely need to be altered. The nature of the changes largely depend on (1) how the LIS communicates the laboratory reports to the IOL (e.g., the FHIR resources making up the laboratory report could be structured differently or the LIS does not conform to FHIR) and (2) the differences in your Tracker programme. As a side note, substantial changes to the transformation script would go hand-in-hand with changes to the IOL when:
- the LIS data format is not FHIR over JSON, or
- the laboratory reports are represented in FHIR resources that do not match the ones enumerated in the Lab Information Section.
A good understanding of Apache Camel is a prerequisite to customising the IOL. The DHIS2 developer documentation provides a gentle introduction to Apache Camel. Besides Camel, a rudimentary knowledge of Java, Spring Boot, and Maven will go a long way in helping you to tailor the project to your requirements.
The IOL source code is located in the iol directory of this project where most of its behaviour is defined in the YAML configs located in the src/main/resources/camel directory path. Below is a description of each config's role:
- main.camel.yaml - Kicks off the routine scan of active enrollments
- fetch-diagnostic-report.camel.yaml - Fetches the diagnostic report from the FHIR server.
- get-de-code-dict.camel.yaml - Builds a mapping between the LOINC codes and the DHIS2 data element codes.
- get-opt-code-dict.camel.yaml - Builds a mapping between the LOINC codes and the DHIS2 option set value codes
- get-transform-script.camel.yaml - Fetches the DataSonnet transformation script from the DHIS2 data store that transforms the FHIR resources to DHIS2 resources.
- process-enrollment.camel.yaml - Pulls out completed lab request events from the enrollment before sending the events downstream for further processing.
- process-lab-request.camel.yaml - Searches for a corresponding lab report DHIS2 event and any matching diagnostic result in the LIS prior to sending the message to be final stage of processing
- import-lab-report.camel.yaml - Imports lhe lab report into DHIS2.
Config or code changes to the IOL should always be followed by changes to the IOL unit tests present in the iol/src/test project path. The IOL is built and unit tested with the following terminal command, run from within the iol directory:
./mvnw clean packageOn Windows, run instead:
mvnw.cmd clean packageWhat follows is a Q&A for some common scenarios when adapting the IOL:
My LIS conforms to a different FHIR IG or I need to fetch different FHIR resources from the LIS for the lab report. How do I configure the IOL to read and transform the right resources?
Within the fetch-diagnostic-report.camel.yaml IOL config, change the URL path of the simple expression within the setHeader key to include or exclude the required FHIR resources from the LIS search results. The official FHIR documentation describes the search operations that can be expressed in the URL path.
How to turn the IOL from a polling consumer into an event-driven one to improve the timeliness of lab reports in DHIS2 and eliminate the performance costs tied to polling?
Change the from endpoint in the main.camel.yaml config such that it listens for events from DHIS2. With the Camel Debezium PostgreSQL component, you can listen to database events thanks to PostgreSQL Logical Replication.
When an event triggers execution in the IOL, you should factor the possibility that the diagnostic report is not yet available or the LIS is offline. Having the IOL retry every so often to fetch the diagnostic report in the event thread is not a good strategy in terms of resource allocation since it can lead to resource starvation. Instead, consider persisting or caching the lab request in the IOL and routinely dispatching a thread from a pool of threads to process the locally stored lab requests. The tradeoff being made here is building more complexity into the IOL in favour of efficiency.
At the time of writing, most LISs do not speak FHIR and facts on the ground could make it impractical to hide the LIS behind a FHIR Facade. Adapting the IOL to talk with a non-FHIR LIS entails replacing the FHIR endpoints in the IOL configs while also swapping out the unmarshal processors with ones that can unmarshal the new format (e.g., HL7v2). Apache Camel has a large catalogue of components from which you can choose to integrate with different LISs. Moreover, given Camel's open architecture, you can always develop your own component when the provided ones do not meet your needs.
Additionally, depending on the LIS data format, the choice of transformation engine (i.e., DataSonnet) might need to be revisited or pre-transformation steps added to prepare the data for transformation.
This integration was designed to exclude personal identifiable information from the data exchange. Nonetheless, the focus here is to illustrate technical interoperability. Security and privacy concerns are out of scope. It is therefore important that the architecture together with the design and code undergo a security and privacy review prior to adaptation.
The time it takes for the IOL to complete a run is
- Case surveillance enrollments in DHIS2 are left open instead of being marked as complete by the DHIS2 user. A simple solution could be to include a step in your standard operating procedures that instructs the DHIS2 user to complete the enrollment once the case is finished.
- Having too many laboratory orders (e.g., due to a disease outbreak). In such cases, if it is not viable to allocate more resources (e.g., memory) to your applications, one ought to consider re-implementing the IOL as an event-driven consumer.
Questions or feedback about this reference implementation can be posted on the DHIS2 Community of Practice. Contributions in the form of pull requests are more than welcome.