☆ 4.7 Article

Robustness of convolutional neural networks in recognition of pigmented skin lesions

EUROPEAN JOURNAL OF CANCER (2021)

Journal

EUROPEAN JOURNAL OF CANCER

Volume 145, Issue -, Pages 81-91

Publisher

ELSEVIER SCI LTD

DOI: 10.1016/j.ejca.2020.11.020

Keywords

Artificial intelligence; Machine learning; Deep learning; Neural networks; Dermatology; Skin neoplasms; Melanoma; Nevus

Funding

Federal Ministry of Health, Berlin, Germany

Ask authors/readers for more resources

Protocol

Community support

Reagent

Community support

Automated Summary New
Abstract

A high robustness is a basic requirement for AI-based image analysis systems in clinical practice. This study quantified the impact of minor image perturbations on CNN-mediated skin lesion classification and evaluated three possible solutions. Results showed that minor image changes may affect CNNs' robustness, but existing methods can reduce brittleness to some extent.

Background: A basic requirement for artificial intelligence (AI)-based image analysis systems, which are to be integrated into clinical practice, is a high robustness. Minor changes in how those images are acquired, for example, during routine skin cancer screening, should not change the diagnosis of such assistance systems. Objective: To quantify to what extent minor image perturbations affect the convolutional neural network (CNN)-mediated skin lesion classification and to evaluate three possible solutions for this problem (additional data augmentation, test-time augmentation, anti-aliasing). Methods: We trained three commonly used CNN architectures to differentiate between dermoscopic melanoma and nevus images. Subsequently, their performance and susceptibility to minor changes ('brittleness') was tested on two distinct test sets with multiple images per lesion. For the first set, image changes, such as rotations or zooms, were generated artificially. The second set contained natural changes that stemmed from multiple photographs taken of the same lesions. Results: All architectures exhibited brittleness on the artificial and natural test set. The three reviewed methods were able to decrease brittleness to varying degrees while still maintaining performance. The observed improvement was greater for the artificial than for the natural test set, where enhancements were minor. Conclusions: Minor image changes, relatively inconspicuous for humans, can have an effect on the robustness of CNNs differentiating skin lesions. By the methods tested here, this effect can be reduced, but not fully eliminated. Thus, further research to sustain the performance of AI classifiers is needed to facilitate the translation of such systems into the clinic. (C) 2020 The Author(s). Published by Elsevier Ltd.

Authors

I am an author on this paper

Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7

Not enough ratings

Secondary Ratings

Novelty

Significance

Scientific rigor

Rate this paper

Original Research A benchmark for neural network robustness in skin cancer classification

Roman C. Maron, Justin G. Schlager, Sarah Haggenmuller, Christof von Kalle, Jochen S. Utikal, Friedegund Meier, Frank F. Gellrich, Sarah Hobelsberger, Axel Hauschild, Lars French, Lucie Heinzerling, Max Schlaak, Kamran Ghoreschi, Franz J. Hilke, Gabriela Poch, Markus Heppt, Carola Berking, Sebastian Haferkamp, Wiebke Sondermann, Dirk Schadendorf, Bastian Schilling, Matthias Goebeler, Eva Krieghoff-Henning, Achim Hekler, Stefan Frohling, Daniel B. Lipka, Jakob N. Kather, Titus J. Brinker

Summary: The study aimed to establish a benchmark for skin cancer classification on dermoscopic images to measure classifier robustness to out-of-distribution (OOD) data. By creating three datasets and evaluating the robustness of four different CNN architectures, it was found that all CNNs showed susceptibility to corruptions and perturbations.