From hand-crafted radiomics to deep learning: evaluating breast cancer classification methods in mammograms

Abstract

This study evaluates the performance of some machine learning (ML) and deep learning (DL) models for breast cancer tumor classification in mammography (MG) images, by training them on the BCDR dataset. The study compares the use of radiomics-based features in ML models, including Random Forest, Support Vector Machines, and XGBoost, with two deep learning approaches using Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). Radiomics features were extracted from segmented regions of interest (ROIs) and used to train the ML models, performing hyperparameter tuning and cross-validation to optimize the results. CNN and ViT models were trained using the MG location present in the ROI segmentations, to explore the impact of tumor region localization assistance on classification performance.

To examine and verify the performance of the ML models and the ViT, the area under the receiver operating characteristic curve (AUC-ROC) and the training execution time of all experiments (performed on the same device) were used.

The results indicate that, while all methods achieve good performance on the training dataset (mean AUCROC scores around 0.9), they exhibit substantial performance drops when tested on external data. Among the evaluated models, ViT achieves the highest overall AUC-ROC in both internal (0.93) and external (0.68) validation, surpassing CNNs and radiomics-based ML models. However, ViT also incurs the highest computational cost, highlighting a trade-off between accuracy and training time. These findings underscore the need for multicenter, multi-vendor data to improve model generalization and reliability, as well as for continued refinement of advanced architectures, such as transformers, to optimize breast cancer lesion classification in clinical settings.