Methodology for Comparing Machine Learning Algorithms for Survival Analysis

This study presents a comparative methodological analysis of six machine learning models for survival analysis (MLSA). Using data from nearly 45,000 colorectal cancer patients in the Hospital-Based Cancer Registries of São Paulo, we evaluated Random Survival Forest (RSF), Gradient Boosting for Survival Analysis (GBSA), Survival SVM (SSVM), XGBoost-Cox (XGB-Cox), XGBoost-AFT (XGB-AFT), and LightGBM (LGBM), capable of predicting survival considering censored data. Hyperparameter optimization was performed with different samplers, and model performance was assessed using the Concordance Index (C-Index), C-Index IPCW, time-dependent AUC, and Integrated Brier Score (IBS). Survival curves produced by the models were compared with predictions from classification algorithms, and predictor interpretation was conducted using SHAP and permutation importance. XGB-AFT achieved the best performance (C-Index = 0.7618; IPCW = 0.7532), followed by GBSA and RSF. The results highlight the potential and applicability of MLSA to improve survival prediction and support decision making.

💡 Research Summary

This paper conducts a comprehensive comparative study of six machine‑learning‑based survival analysis (MLSA) models—Random Survival Forest (RSF), Gradient Boosting for Survival Analysis (GBSA), Survival Support Vector Machine (SSVM), XGBoost‑Cox, XGBoost‑Accelerated Failure Time (XGB‑AFT), and LightGBM (LGBM)—using a large, real‑world cohort of colorectal cancer (CRC) patients from the Hospital‑Based Cancer Registries of São Paulo (approximately 45 000 cases). The authors begin by describing the clinical relevance of CRC, noting its high incidence and mortality worldwide and in Brazil, and the limitations of traditional Cox proportional hazards models when proportionality assumptions are violated. They argue that MLSA methods, which natively handle censoring, are better suited for large observational datasets.

Data preparation involved selecting CRC cases (ICD‑10 C18‑C20), applying strict exclusion criteria (age < 20, non‑residents, missing staging, non‑adenocarcinoma histology, etc.), and engineering variables that capture demographic information, service characteristics, and time intervals from diagnosis to treatment initiation. The final outcome variables are time‑to‑event (months) and a binary death indicator. Categorical features (e.g., clinical stage, topography) were ordinally encoded, while continuous variables were standardized using z‑scores; this uniform scaling was applied to all models, even tree‑based ones, to improve numerical stability.

Model implementation details are thorough. RSF uses non‑parametric Nelson‑Aalen estimators within a bagging framework. GBSA follows Ridgeway’s formulation, employing the Cox partial likelihood as its loss function and sequentially fitting decision trees. SSVM extends the classic SVM by incorporating censoring and optimizing a C‑Index‑based loss. XGBoost is adapted in two ways: XGB‑Cox uses the Cox partial likelihood, while XGB‑AFT adopts an accelerated failure‑time loss that directly models survival time assuming a parametric error distribution. LightGBM, lacking a native survival extension, is employed as a regression model with death status used as a sample weight.

Hyper‑parameter optimization was performed with Optuna, testing three distinct samplers—RandomSampler (pure random search), TPESampler (Bayesian optimization via Tree‑structured Parzen Estimator), and CmaEsSampler (evolutionary strategy). Each sampler evaluated 150 hyper‑parameter configurations with 10‑fold cross‑validation, and the best‑performing sampler for each algorithm was retained for the final model. This exhaustive search, though computationally intensive, enhances robustness against over‑fitting and local minima.

Four evaluation metrics were used: (1) Concordance Index (C‑Index) for overall discriminative ability; (2) Inverse Probability of Censoring Weighting C‑Index (IPCW‑C‑Index) to correct for informative censoring; (3) mean time‑dependent Area Under the ROC Curve (AUC) to assess discrimination across follow‑up times; and (4) Integrated Brier Score (IBS) to measure calibration and overall prediction error over the entire time horizon. Notably, only RSF and GBSA provide native survival curves suitable for IBS calculation; for SSVM an approximate curve was derived via a calibrated Cox model, while XGB‑AFT, XGB‑Cox, and LGBM could not generate reliable survival functions, precluding IBS for those methods.

Results show that XGB‑AFT achieved the highest C‑Index (0.7618) and IPCW‑C‑Index (0.7532), outperforming GBSA and RSF. GBSA ranked second, with RSF third. SSVM lagged behind, reflecting its generally lower predictive power in similar studies. LightGBM and XGB‑Cox, despite strong risk predictions, suffered from the inability to produce accurate survival curves, limiting their evaluation to discrimination metrics only.

Interpretability was addressed using SHAP values and Permutation Importance. Consistently important predictors across models included clinical stage at diagnosis, delay between diagnosis and treatment initiation, patient age, and type of healthcare service. These findings align with established clinical knowledge, reinforcing the credibility of the MLSA models.

The authors also compared the MLSA models to previously developed classification models that predict binary survival at fixed horizons (1, 3, 5 years). Because MLSA retains censored cases, it offers a more comprehensive risk assessment for the entire cohort, avoiding the selection bias inherent in classification‑only approaches.

In discussion, the paper highlights the practical implications of deploying XGB‑AFT and GBSA in clinical decision‑support systems, emphasizing their superior discrimination and robustness to censoring. Limitations include the lack of external validation, the inability of some algorithms to generate calibrated survival curves, and potential over‑reliance on ordinal encoding for categorical variables. Future work is suggested to involve external cohort testing, integration of deep learning survival models, and development of user‑friendly tools for clinicians.

Overall, this study provides a rigorous methodological framework for comparing MLSA algorithms on large oncology datasets, demonstrates the superiority of gradient‑boosting‑based approaches (especially XGB‑AFT) in this context, and underscores the importance of comprehensive hyper‑parameter tuning, multi‑metric evaluation, and model interpretability for translating predictive analytics into actionable clinical insights.