Towards equitable deep learning in dermatology: assessing lesion classification fairness across skin tones

Abstract

Recent advances in deep learning skin lesion classifiers rose expectations that these models can be implemented in the clinical routine in the near future. However, before deploying deep learning models in such a sensitive area as healthcare, it is important to ensure their trustworthiness and mitigate any kind of discrimination. This thesis investigates discrimination by skin tone in a light-weight deep learning skin lesion classifier trained on a benchmark dataset of dermatological images and assesses the feasibility of SinGAN-generated synthetic dark skin images to improve predictions on dark skin samples in the absence of dark skin training data. The results suggest that (I) there is discrimination by skin tone, (II) a data shift from apparent light skin samples in training to apparent dark skin samples in deployment deteriorates predictions, and (III) although dark SinGAN-generated samples may improve performance, oversampling of a few dark skin samples appears more feasible. Most importantly, however, a thorough analysis of automated skin tone estimations with the Individual Topology Angle revealed that (IV) these skin tone estimations might measure the darkness of a skin image rather than the darkness of skin in the image and (V) the investigated HAM10000 dataset is less diverse than previous research suggested. This has potentially wide-ranging implications for previous publications about skin tone fairness using this dataset and emphasizes the need for further research on more diverse dermatology datasets with more reliable skin tone labels before wide-spread deployment of skin lesion classifiers.