Longitudinal Image-based AI Models for Health and Medicine
AI can see the tip! Deep learning predicts all-cause mortality from single and sequential body composition imaging
Key Points, TLDR:
- The mixture of body composition imaging and meta-data (e.g. age, sex, grip strength, walking speed, etc) resulted in the very best 10 12 months mortality predictions
- Longitudinal or sequential models overall performed higher than single record models, highlighting the importance of modeling change and time dependencies in health data.
- Longitudinal models have the potential to offer a more comprehensive assessment of 1’s health
- Read the paper
Artificial intelligence (AI) and machine learning (ML) are revolutionizing healthcare, driving us toward the era of precision medicine. The motivation to develop AI health models is to cut back deaths and disease in addition to delay a prime quality of life. Well trained models have the power to more thoroughly analyze data that’s presented which offers a more comprehensive assessment of 1’s health.
Image-based medical AI/ML models have now reached a maturity where they often rival and even surpass human performance, adeptly identifying patterns and anomalies that would easily elude the human eye. Nevertheless, the vast majority of these models still operate on single time-point data, providing an isolated snapshot of health at one specific instance. Whether these are uni-modal or multi-modal models, they have a tendency to work with data gathered inside a comparatively similar timeframe, forming the muse of a prediction. Yet, within the broader context of AI/ML for medical applications, these single time-point models represent just step one — the proverbial ‘low hanging fruit.’ One frontier of medical AI research is longitudinal models which supply a more holistic view of an individual’s health over time.
Longitudinal models are designed to integrate data from multiple time-points, capturing a person’s health trajectory relatively than a standalone moment. These models tap into the dynamic nature of human health, where physiological changes are constant. The flexibility to map these changes to specific outcomes or health questions may very well be a game-changer in predictive healthcare. The concept of longitudinal data isn’t latest to clinical practice — it’s usually used to observe aging and predict frailty. A major example is the tracking of bone mineral density (BMD), a key marker for osteoporosis and frailty. Regular assessments of BMD can detect significant decreases, indicating potential health risks.
Historically, the event of longitudinal models has faced several significant challenges. Other than larger data volumes and computation required per individual, essentially the most critical obstacle lies within the curation of longitudinal medical data itself. Unlike single time-point data, longitudinal data involves tracking patients’ health information over prolonged periods, often across multiple healthcare institutions. This requires meticulous data organization and management, making the curation process each time-consuming and expensive. Multiple successful studies have been funded to prospectively collect longitudinal data. These studies report challenges with respect to patient retention over an extended remark period. Hence, despite the potential advantages of longitudinal models, their development has remained a posh, resource-intensive endeavor.
Changes in body composition, proportions of lean and fat soft tissue and bone, are known to be related to mortality. In our study, we aimed to make use of body composition information to raised predict all-cause mortality, in simpler terms, the likely timeline of an individual’s life. We evaluated the performance of models built on each single time-point and longitudinal data, respectively known as our ‘single record’ and ‘sequential’ models. Single record models allowed us to judge what kind of information was most predictive of mortality. Development of sequential models were for the needs of capturing change over time and evaluating how that affects mortality predictions.
The information for this study was acquired from a longitudinal often known as the Health, Aging, and Body Composition (Health ABC) study by which over 3000 older, multi-race female and male adults were followed and monitored for as much as 16 years. This study resulted in a wealthy and comprehensive longitudinal data set. As a component of this study patients received total body dual energy X-ray absorptiometry (TBDXA) imaging and a number of other pieces of meta-data were collected (see table XXX). Consistent with best modeling practices and to avoid data leakage or mitigate overfitting, the info was split right into a train, validation, and hold-out test set using a 70%/10%/20% split.
We quantify body composition using total body dual energy X-ray absorptiometry (TBDXA) imaging which has long been considered a gold standard imaging modality. Historically, patient meta-data which include variables like age, body mass index (BMI), grip strength, walking speed, etc were used to evaluate aging/mortality and used as surrogate measurement of body composition. The prevalent use of patient meta-data and surrogate measures of body composition were driven by the limited accessibility to DXA scanners. Accessibility has improved greatly as of recent with scans becoming cheaper and not needing a physician referral/order/prescription.
Three single record models were built each with different data inputs but all with the identical output which was a ten 12 months mortality probability. The primary model was built to only take patient meta-data and is a neural network with a single 32-unit, ReLU activation hidden layer and sigmoid prediction layer. The second model used only TBDXA images as input and it consisted of a modified Densenet121 which was modified to handle the 2 color channels as opposed to a few color channels (RGB) seen in most natural images. The twin energy nature of DXA leads to a high and low X-ray images that are fully registered and stacked into two image channels. The third model combines the meta-data embedding of model one with the TBDXA image embeddings of model two then passes it through a 512-unit, a 64-unit fully-connected ReLU layer to make, and lastly a sigmoid prediction layer.
Three sequential models were built and evaluated. The only record model architectures served as the bottom for every sequential model however the sigmoid prediction layers were removed in order that the output was a vector representing feature embeddings. Over the course of the study data was collected from each patient at multiple time points. The information from every time point was input into the suitable models to accumulate the corresponding feature vector. The feature vectors for every patient were ordered and stacked right into a sequence. A Long Short Term Memory (LSTM) model was trained to take the sequence of feature vectors and output a ten 12 months mortality prediction. As previously mentioned, there are several difficulties with conducting long run studies with retention and data collection being a typical problem. Our study was not absent of those problems and a few patients had more data points that others in consequence. An LSTM model was chosen because the sequence modeling approach because they aren’t constrained to make use of the identical input sequence length for every patient. I.e. LSTMs can work with sequences of various length thus eliminating the necessity to pad sequences if patients were short the complete set of information points (~10).
Area under the receiver operating characteristic (AUROC) on the hold-out test set show that metadata performs higher than using TBDXA image alone in each the one record and sequential models. Nevertheless, combining meta-data and TBDXA imaging resulted in the very best AUROCs in each modeling paradigms which indicates that imaging incorporates useful information, predictive of mortality that isn’t captured by the meta-data. One other approach to interpret that is that the meta-data aren’t a full surrogate measure of body composition with respect to predicting mortality. In the event that they were full surrogates, combining TBDXA imaging with meta-data would have resulted in no significant increase or change in AUROC. The indisputable fact that the mix resulted in higher AUROCs indicates that imaging is providing orthogonal information beyond what the meta-data capture and further justifies the utility of imaging.
Longitudinal or sequential models overall performed higher than single record models. That is true across all modeling approaches and input data types (meta-data, image only, combined meta-data and image). These results reveal the importance of modeling change and the time dependencies of health data.
We performed an Integrated Discrimination Improvement (IDI) evaluation to judge the advantages of mixing imaging with metadata, in comparison with using metadata alone. This evaluation was conducted on the sequence models, which outperformed the single-record models. The IDI was found to be 5.79, with an integrated sensitivity and specificity of three.46 and a pair of.33, respectively. This means that the mix of imaging and metadata improves the model’s ability to accurately discover those that is not going to survive the following 10 years by 3.46%, and enhances the power to accurately discover those that will survive the following 10 years by 2.33%. Overall, this means an improvement in model performance of roughly 5.8%.
Our study underscores the promising potential of longitudinal AI/ML models within the realm of predictive healthcare, specifically within the context of all-cause mortality. The comparative evaluation of single record models and longitudinal models revealed that the latter offers superior performance, indicating the critical role of modeling change over time in health data evaluation. The clinical implication of our findings include the power to offer a more precise and holistic assessment of 1’s health through models that account for a patient’s historical or longitudinal data. While the info needed for developing longitudinal health models exists, the correct infrastructure and institutional support isn’t quite oriented yet to enable efficient data curation and development of those models at scale. Nevertheless, many are working to beat these hurdles and the event of longitudinal models is certainly one of many exciting frontiers for AI in medicine.
The clinical implications of those findings are far-reaching. Longitudinal models have the potential to remodel care delivery by enabling more precise, personalized predictions a few patient’s health trajectory. Such models can inform proactive interventions, thereby enhancing care outcomes and possibly even prolonging life. Furthermore, using each metadata and imaging data sets a brand new precedent for future AI/ML models, suggesting a synergistic approach for optimal results. It reinforces the necessity for multidimensional, nuanced data to color an accurate and holistic picture of a patient’s health. These findings represent significant strides in the appliance of AI/ML in healthcare, highlighting an exciting path forward in our pursuit of precision medicine.