Predictive technology that could Help Patients
CAR T therapy works by modifying some of a patient’s white blood cells so they can identify and destroy cancer cells. After treatment, between 35% and 93% of recipients are estimated to develop a side effect known as cytokine release syndrome. The syndrome, in mild cases, can feel like the flu. But in more severe cases, the inflammation that the syndrome causes can lead to heart, kidney, lung or liver failure, all of which can be deadly.
Methods of detecting the syndrome have often relied on the patient exhibiting symptoms, at which point the patient is more likely to develop a severe case of the syndrome, according to the students. While some predictive models exist, the students found that these systems do not precisely detail when the syndrome may develop.
The students’ machine learning system can predict the likelihood of not only cytokine release syndrome but also its onset within six- or eight-hour windows.
Before building their model, the students constructed two datasets with anonymized information from Mount Sinai’s treatment of 30 previous CAR T patients. The students included data on the patients’ blood oxygen levels, pulses and temperatures, as well as their levels of the proteins that spur cytokine release syndrome. The data was partly collected partly by nurses and devices the patients wore.
Once the students had cleaned the datasets, addressing any missing data or cases of improper structuring, they could begin creating their machine learning system. The students ran their datasets through several machine learning models to determine which was best for the project.
In the end, the students selected the Random Forest model, which makes its predictions using multiple random “decision trees.”
Using the students’ datasets, the model achieved roughly 93% accuracy in identifying patients who were likely to develop cytokine release syndrome within six hours. “It was a highly successful proof of concept, combining MISM technical learnings and newly learned clinical knowledge. The project required frequent stakeholder communication and agile planning skills, critical to the overall success,” Kowalsky said.
Alessandro Lagana, Ph.D., an assistant professor of genetics and genomic sciences at the Icahn School of Medicine, said the “collaboration offered an important opportunity to explore early modeling approaches for predicting cytokine release syndrome.”
“We appreciated the opportunity to collaborate with the student team,” Lagana continued. “The students’ work helped surface key data challenges and informed our subsequent, more comprehensive internal analysis. Their efforts contributed meaningfully to our understanding of how predictive tools in this space might be designed and evaluated.”
Navigating medical jargon, unstructured data
The team encountered several challenges during the project. For starters, most of the team members did not have a healthcare background and were unfamiliar with both CAR T and cytokine release syndrome. “It was a little hard for us to understand the therapy and all the medical terminology,” Tian said.
The students also worked through typical real-world data challenges in the Mount Sinai dataset. Because the data were collected at varying intervals, the team needed to standardize the timelines before analysis. They also addressed some expected gaps in device-recorded patient information, as not all participants wore their devices consistently.
“There were many different data sources, and some were handwritten. We had to put a lot of effort into data cleaning and integrating the different data sources into the datasets that trained our model,” Yao said.
A data scientist from Mount Sinai also worked closely with the team to help them understand the data and overcome project obstacles.
The project –– specifically the process of preparing the data for the machine learning system –– was a great learning experience for the students, Kowalsky said. By encountering and having to clean unstructured data, the students learned to navigate a common challenge data scientists face.
“The project wasn't easy, and there wasn't always a clear path to success,” Kowalsky said. “They had to be data scientists, software engineers, good communicators and project planners. They had to understand the problem from both a medical and business standpoint. They had to be able to adjust as they went along.”
He continued: “It took a special type of skill level, a special type of student focus and the special type of applied education that the MISM program provides.”
What comes next for the project
At this time, Mount Sinai has not implemented the student prototype in clinical workflows, as further rigorous validation and refinement are required before such tools can be deployed in practice. However, the project helped inform a more detailed analysis that a research team has since conducted, and the insights gained continue to shape Mount Sinai’s ongoing work in this area.
The MISM program and its required capstone prepare students for their careers, Kowalsky said.
“The capstone gives students the critical skills to work in a team, scope a project, interact with a client, measure progress toward deadlines and pivot when necessary. The teams that do very well on the capstone become well-positioned to thrive in their chosen position after graduation.”
Before she began the capstone project, Tian enrolled in a machine learning course at Heinz College. She found the class challenging at the time, but during the capstone, she applied what she had learned.
“Before taking that class, I never imagined that I could do the kind of work we did in the capstone, because I think machine learning is really difficult for me. This class gave me a lot of knowledge and background, and it helped me understand existing models,” Tian said.
Yao said the Heinz College curriculum provided him with a strong foundation to handle the problems the students encountered during the capstone project. Now a data scientist in Boston, he sees similarities between his work and that of the capstone project.
“The capstone project definitely gave us a valuable opportunity to practice skills we need to work in industry,” he said.