A learning risk score for patients with peripheral arterial occlusive disease using machine learning
by Thea Kreutzburg, Frederik Peters, Jenny Kuchenbecker, Ursula Marschall, and Christian-Alexander Behrendt
Corresponding adress: firstname.lastname@example.org
In times of evidence-based medicine, every patient should receive the best medical care as soon as possible, which is built on personalized medicine, high evidence grade, and the experience of qualified physicians. High medical evidence develops through research over years involving randomized and controlled clinical trials. In peripheral vascular medicine, many guidelines are based on consensus recommendations with low level of evidence due to a lack of available trial data (Conte et al. 2019). Therefore, it can be challenging to choose the best available treatment on a high level of evidence. Additionally, there is sparse knowledge about long-term outcomes after revascularizations in the lower limb. Real-world data from registries or health insurance claims can help to complement the knowledge-base using large longitudinal cohorts with only marginal selection bias. With the FINNVASC registry study a linear sum score for postoperative mortality and/or major lower-limb amputation exists (Biancari et al. 2007) but only for a prediction of 30 days after hospital discharge. With the multi-centre GermanVasc registry study in Germany, data of consecutively treated patients and one year follow-up are collected to close this lack of knowledge (Behrendt et al. 2017). Nevertheless, for five year outcomes there is no (planned) large trial, but retrospectively health insurance claims data exist for millions of patients with a follow-up up to 11 years which can be used instead. Additionally, PAOD is a chronic progressive disease typically affecting high-aged patients, which have several comorbidities, e.g. hypertension and congestive heart failure, when they got their first diagnosis of PAOD (Kreutzburg et al. 2019). All individual comorbidities and frailties have to be considered for this complex atherosclerotic disease in long-term investigations. To develop long-term prediction models a variable selection process is necessary. In the ongoing RABATT project, we use algorithm-based risk prediction to support informed consent for the best therapy choice in the healthcare of PAOD patients (Schwaneberg et al. 2019).
In the algorithms the computer learns to minimize the error in predicted and observed outcomes validated in data unseen to the algorithm (Dreiseitl and Ohno-Machado 2002). For stroke and cardiovascular risk prediction machine learning (ML) predictions exist (Chen-Ying et al. 2017, Weng et al. 2017).
The aim of this study was to develop an easy-to-use long-term risk with a high accuracy in patients with PAOD using retrospectively health insurance claims data (Biancari et al. 2007).
In health insurance claims data of Germany’s second largest insurance fund BARMER, we used the inpatient medical care provided to approximately 9.4 million German citizens (13.2% of Germany’s population). We used the coding of the International Classification of Diseases in its German Modification (ICD-10-GM) and Operations and Procedures Codes (OPS).
We included patients with symptomatic PAOD diagnosis (stages II to IV according to the Fontaine classification) from January 1, 2008 to December 31, 2018 in the BARMER cohort with the first index hospital stay and long-term outcomes. Patients with prior major amputation were excluded (see Figure 1).
We used age (in years and dichotomised 50+, 60+, …, 90+), sex, 30 Elixhauser comorbidity groups (lookback 3 years), 103 frailty variables (lookback 3 years), prior myocardial infarction or stroke, dialysis, wound infection, smoking, major amputation, atrial fibrillation, compartment syndrome, bleeding, readmission, reoperation, and others.
All codes for the utilized variables are listed in the supplement.
For the risk score, we used the outcome event-free survival after index hospital stay (first PAOD diagnosis). Event means either myocardial infarction, stroke, amputation, or death.
We stratified all prediction models by Fontaine stages: II intermittent claudication (IC) vs. III/IV chronic limb threatening ischaemia (CLTI). We used a time-to-event analysis with a Cox proportional hazards approach (Ref).
First, we separated the original dataset in training (60%) and validation data set (40%). With the Least Absolute Shrinkage and Selection Operator (LASSO) method, a risk score based on individual risk of baseline risk and hazard was performed using a penalty term (lambda) (Tibshirani 1996) and 10 times cross validation on the training data set. With this approach, variables with no significant contribution were set to no contribution and can be deleted afterwards. Afterwards, the prediction was calculated for the validation data set. With a backward variable selection we evaluated the performance of the LASSO approach.
The FINNVASC score for 30 days postoperative complications is based on four dichotomous variables: diabetes (ICD E10/11), coronary artery disease (I20-I25), foot gangrene (Fontaine stage IV), and urgent operation as an emergency treatment case in hospital. The sum score is 0, 1, 2, 3 or 4 (Biancari, Salenius et al. 2007). The linear van Walraven score was developed for the in-hospital mortality for the 30 Elixhauser groups (Elixhauser, Steiner et al. 1998, Quan, Sundararajan et al. 2005, van Walraven, Austin et al. 2009).
The importance of the variables is calculated with the chi square test and sorted for the 15th variables with the highest importance.
The predictive accuracy was assessed based on the validation data by the concordance statistics (c index, generalized area under curve, AUC value) ranging from 0 to 1 whereas a high value means high accuracy. The goodness of fit was assessed with the Grønnesby and Borgan test (p value) (Grønnesby and Borgan 1996). The net benefit of the prediction was illustrated in a decision curve (http://www.decisioncurveanalysis.org) in the supplement. For the risk group scores, the calibration was illustrated with Kaplan Meier curves.
Software. All analyses were performed with SAS version 9.04 for data processing and R version 3.3 for modelling (packages survival, Hmisc and glmnet).
The project RABATT is funded by the innovations fund of the German Federal Joint Committee (G-BA) between April, 2019 and March, 2022 (grant number 01VSF18035).