A Deep Multi-Modal Method for Patient Wound Healing Assessment

Hospitalization of patients is one of the major factors for high wound care costs. Most patients do not acquire a wound which needs immediate hospitalization. However, due to factors such as delay in treatment, patient’s non-compliance or existing co-morbid conditions, an injury can deteriorate and ultimately lead to patient hospitalization. In this paper, we propose a deep multi-modal method to predict the patient’s risk of hospitalization. Our goal is to predict the risk confidently by collectively using the wound variables and wound images of the patient. Existing works in this domain have mainly focused on healing trajectories based on distinct wound types. We developed a transfer learning-based wound assessment solution, which can predict both wound variables from wound images and their healing trajectories, which is our primary contribution. We argue that the development of a novel model can help in early detection of the complexities in the wound, which might affect the healing process and also reduce the time spent by a clinician to diagnose the wound.

💡 Research Summary

The paper addresses the costly problem of wound‑related hospitalizations by proposing a unified deep multi‑modal framework that jointly leverages wound photographs and a set of clinician‑recorded variables to predict a patient’s risk of hospitalization (i.e., whether the wound will heal without further intervention or will require admission). The authors first construct five dedicated convolutional neural networks (CNNs) based on a pre‑trained Xception architecture to infer five key wound attributes directly from images: ulcer type, anatomical location, wound stage, joint necrosis exposure, and bone necrosis exposure. Because the dataset is heavily imbalanced across ulcer types (e.g., 19,773 diabetic ulcer images versus 47,541 pressure ulcer images), they employ class‑specific data augmentation (rotations, color jitter, cropping) and a 5‑fold cross‑validation scheme to mitigate over‑fitting and to obtain robust performance estimates. Single‑task CNNs are used for ulcer type and location, while a multi‑task configuration is adopted for stage and necrosis predictions, exploiting the known dependency of these variables on ulcer type. Training is performed with the Adadelta optimizer (learning rate 0.001), batch size 32, and 50 epochs; the final layer is a softmax classifier whose output probabilities serve as the predicted wound attributes.

In the second stage, the five predicted attributes are concatenated with sixteen additional clinician‑entered variables (e.g., BMI, age, gender, wound area, wound volume, comorbidities) to form a heterogeneous feature vector. This vector feeds a Light Gradient Boosted Machine (LightGBM) classifier, which is well‑suited for mixed categorical and continuous data and provides fast training and inference. The binary target is “Hospitalization‑Wound Related” versus “Treatment Complete (In active)”. To address class imbalance in the final task, class weights are adjusted during training. Evaluation on a held‑out test set yields a precision of 0.68, recall of 0.91, and F1‑score of 0.78 for the healing class, while the non‑healing class achieves precision 0.99, recall 0.79, and F1‑score 0.88. These results demonstrate that the combination of image‑derived predictions and traditional clinical data can reliably flag patients at risk of hospitalization.

The dataset comprises over 150,000 wound images collected over several years, covering five ulcer categories (diabetic, pressure, surgical, trauma, venous) and annotated with detailed wound variables by expert clinicians. The authors split the data into 70 % training, 10 % validation, and 20 % testing, and they explicitly discuss practical challenges encountered during model development: (i) occlusion by scales or clinicians’ hands, (ii) varying illumination due to smartphone capture, (iii) severe class imbalance, (iv) visual similarity between different ulcer types, and (v) deformation of the same wound across images. They address these issues through preprocessing (color correction, background removal), augmentation, and careful loss weighting. Attention heatmaps generated from the final CNN layer illustrate that the model focuses on the wound region despite the lack of explicit segmentation masks, providing a degree of interpretability.

Limitations acknowledged by the authors include reliance on a single‑institution dataset (potentially limiting external generalizability), possible labeling errors in the clinician‑provided metadata, and the exclusive use of 2‑D RGB photographs without depth, thermal, or multimodal imaging data. Future work is outlined to incorporate multi‑center data, 3‑D or multispectral imaging, longitudinal modeling of wound healing trajectories, and more sophisticated explainability techniques such as Grad‑CAM. The authors also envision integrating the model into real‑time clinical decision support tools to reduce the time clinicians spend on manual wound documentation and to enable earlier therapeutic interventions.

In summary, this study presents a practical, high‑performing solution that combines transfer‑learning‑based CNNs for wound attribute extraction with a gradient‑boosted decision tree classifier for hospitalization risk prediction. The reported performance suggests that such a multi‑modal approach can meaningfully assist clinicians in early detection of complex wound conditions, potentially lowering healthcare costs and improving patient outcomes.