CHOP scientists build a custom tool that drastically reduces engraftment reporting error rates

Researchers at Children's Hospital of Philadelphia (CHOP) have developed a custom-built application to automate determination of engraftment, a key outcome after hematopoietic stem cell transplant (HSCT).

The application supersedes a tedious manual process and at the same time substantially improves accuracy of reported hematopoietic cell transplant engraftments. The success of the tool, described recently in the journal Transplantation and Cellular Therapy, demonstrates how a unique embedded informatics team can rapidly build and implement applications that are useful in a clinical setting.

Led by our Hematology/Oncology and Clinical Informatics fellow, David Anderson, MD, our team was rapidly able to build a custom tool that drastically reduced engraftment reporting error rates. Our experience building this application could serve as a blueprint for broad implementation of a similar tool, which would improve not only transplant reporting but also transplant research and practice."

Stephan Kadauke, MD, PhD, Senior Author, Associate Director of the Cell and Gene Therapy Laboratory and Medical Director of Cell and Gene Therapy Informatics at Children's Hospital of Philadelphia

Hematopoietic stem cell (HSC) transplant, also referred to as bone marrow or stem cell transplant, is a complex process used for the treatment of multiple types of malignant and non-malignant diseases in adults and children. The process involves wiping out a patient's own HSCs with chemotherapy, which prepares the body for receiving new cells. Once the chemotherapy has cleared out the patient's HSCs, new HSCs from a donor are infused. These new cells take time to establish themselves in the bone marrow and proliferate, a process known as engraftment.

The time from HSC infusion to engraftment is important. If HSCs engraft late, patients are at risk for infection and bleeding complications. If HSCs don't engraft at all, patients face life-threatening risks and require an urgent re-transplant. For that reason, federal law requires that clinicians report time to engraftment for all transplants to the Center for International Blood and Marrow Transplant Research (CIBMTR). This registry provides real-world data that are used to inform research and clinical guidelines directly affecting the care of transplant patients.

However, current electronic health record (EHR) systems lack the capability to calculate engraftment automatically, so staff need to manually abstract these data from patient charts. This process involves collating records from different source systems and reviewing lengthy tables of lab results, which can be difficult, time-consuming, and error-prone, as the time to engraftment can vary based on whether patients had platelet transfusions, whether their neutrophil counts dipped sometime after infusion, and other variables that might be missed in manual review.

To improve accuracy and efficiency, an informatics team embedded within CHOP's Cell Therapy Program built an application using R/Shiny, an open-source framework for developing interactive web applications that can perform complex data acquisition and manipulation tasks. The tool developed by the team extracts data from multiple sources and calculates engraftment dates based on CIBMTR rules. The application is hosted on an internal platform, and access is restricted to designated staff. It was designed to be compliant with Foundation for the Accreditation of Cellular Therapy (FACT) standards for electronic records.

Routine usage of the application to assist with CIBMTR data reporting began in April 2021. In a prospective analysis from April 2021 to April 2022, the researchers found that neutrophil engraftment reporting was found to be incorrect in only 2 of 53 cases (3.8%), both of which were due to typographical errors and not the application, versus 15% of cases when engraftments were calculated manually prior to April 2021. Over the same period, platelet engraftment reporting was found to be incorrect in 1 of 53 cases (1.9%), also due to a typographical error, a significant decrease from the 28% error rate prior to the implementation of the tool.

"The success of this tool and the speed with which we were able to build it underscores the value of having an informatics team embedded within the Cell Therapy program, rather than under organization-wide software or analytics groups," Dr. Kadauke said. "Our consistent cellular therapy focus has allowed us to move more nimbly to complete projects and pivot to new initiatives. This is a model we believe could be emulated by other academic medical centers."