Type 2 diabetes is one of the most common chronic diseases and is associated with an increased risk of severe cardiovascular complications such as stroke and heart attack. These not only pose a significant health risk but also lead to high costs in the healthcare system. However, early prevention can significantly reduce the risk. Many risk prediction models currently rely on clinical data, which is often collected unsystematically. Digitalization and the use of large health datasets, such as those from health insurance providers, open up new possibilities for targeted prevention.https://www.hs.mh.tum.de/php/team/wissenschaftliche-mitarbeiterinnen/anna-janina-stephan/
This raises the question of whether risk prediction models based on these data can make valid predictions for heart attacks and strokes. The research team led by Prof. Dr. Michael Laxy, head of the Professorship of Public Health and Prevention, and his research associate Dr. Anna-Janina Stephan address this question in a new study. They use German health insurance data from the period 2014 to 2019, with 287 potentially relevant variables for predicting the 3-year risk of heart attack and stroke. The results were published under the title “Development and validation of prediction models for stroke and myocardial infarction in type 2 diabetes based on health insurance claims: does machine learning outperform traditional regression approaches?” in the medical journal “Cardiovascular Diabetology” The journal has an impact factor of 8.5.
Professor Laxy explains: “This was a very challenging project. Both nationally and internationally, there have been few studies so far that have pursued a similar approach. Our study shows that, in principle, it is possible to distinguish individuals with high cardiovascular risk from those with low cardiovascular risk based on routine health insurance data. The quality of the prediction models is comparable to existing epidemiological models that are based on clinical data.”
The research objective was to develop and validate prediction models for stroke and heart attack in patients with type 2 diabetes. Dr. Stephan explains: “The first question was: Are the health insurance data, which were originally collected for entirely different purposes, even sufficient for predicting risk? And secondly: Is it enough to use relatively simple models that have existed for a long time to develop these prediction models, or do we achieve better predictions when we use state-of-the-art machine learning methods?”
As part of the study, traditional regression methods were compared with modern machine learning techniques, including deep learning. A train-test split approach was applied, and the following modeling approaches were tested: logistic regression with/without variable selection, LASSO regularization, Random Forest (RF), Gradient Boosting (GB), Multi-Layer Perceptron (MLP), and Feature-Tokenizer-Transformer (FTT). The models were evaluated in terms of discrimination (AUPRC, AUROC) and calibration.
Overall, the study included 371,006 patients (average age 67.2 years). 3.5 percent of the recorded individuals (n = 13,030) suffered a heart attack, 3.4 percent (n = 12,701) a stroke. The results show that machine learning (including deep learning) did not achieve significantly better performance than traditional regression methods. The prediction models based on health insurance data achieved a maximum discrimination performance of about 0.09 (AUPRC) and 0.7 (AUROC). While these values are comparable to existing epidemiological models, machine learning did not offer significant advantages over traditional methods. This suggests that the complexity of the data had already been exhausted before the algorithm could make a significant difference.
“I want to emphasize that this result does not necessarily mean that machine learning fundamentally cannot work better. In selecting predictors, which we made available to the models, we relied heavily on the literature. One would need to try again with other predictors that we do not yet know are related to diabetes and diabetes complications. The models should truly be given the opportunity to develop their predictive power by identifying and incorporating patterns in the data that we do not yet understand,” Dr. Stephan explains the findings.
Future studies should accordingly examine whether other methods of feature engineering can improve prediction accuracy. Furthermore, additional external validation is needed, according to Dr. Stephan: “So far, we have only developed and tested the models using data from one health insurance provider. The next step would be to see whether the models perform just as well with insured data from another provider or with insured data from a later period.”
With the “Health Data Utilization Act” introduced in 2024, Germany has also paved the way for the targeted use of insurance data for risk screenings in the future. Regarding the law, Dr. Stephan explains: “It enables insurers to actually apply such prediction models in routine care in the future. This allows them to approach their insured members and inform them to see a doctor to check for potential risks. Essentially, the relevance of such models for application and their usability in a real healthcare context has significantly increased with the law.”
Homepage of the Professorship of Public Health and Prevention
Link to the study
Contact:
Prof. Dr. Michael Laxy
Professorship of Public Health and Prevention
Technical University of Munich
Georg-Brauchle-Ring 60/62
80992 Munich
Tel.: +49 89 289 24977
E-Mail: michael.laxy(at)tum.de
Dr. Anna-Janina Stephan
Professorship of Public Health and Prevention
Technical University of Munich
Georg-Brauchle-Ring 60/62
80992 Munich
Tel.: +49 89 289 24984
E-Mail: anna-janina.stephan(at)tum.de
Text: Jasmin Schol
Photos: Pixabay/Private