Beginning in 2016, ISIC has sponsored annual challenges for the computer science community in association with leading computer vision conferences. Over the years, the challenges have grown in scale, complexity, and participation, using high-quality human-validated training and test sets of thousands of CC-0-licensed images and metadata. The earlier challenges were focused primarily on diagnostic accuracy for distinguishing melanoma from other benign and malignant skin lesions. By 2018, the diagnostic performance of the leading algorithms was consistently outperforming clinicians in "reader studies". Additional challenges in 2019 and 2020 were designed to address the out-of-distribution problem and assess the impact of clinical context respectively. The 2020 challenge had 3,300 participants from around the world. In addition to the annual Grand Challenges, ISIC hosts "live challenges" that allow researchers and students to benchmark the performance of their algorithms using ISIC images on an ongoing basis.
About the ISIC Archive
The International Skin Imaging Collaboration (ISIC) is an international effort to improve melanoma diagnosis, sponsored by the International Society for Digital Imaging of the Skin (ISDIS). The ISIC Archive contains the largest publicly available collection of quality controlled dermoscopic images of skin lesions.
Presently, the ISIC Archive contains over 13,000 dermoscopic images, which were collected from leading clinical centers internationally and acquired from a variety of devices within each center. Broad and international participation in image contribution is designed to insure a representative clinically relevant sample.
All incoming images to the ISIC Archive are screened for both privacy and quality assurance. Most images have associated clinical metadata, which has been vetted by recognized melanoma experts. A subset of the images have undergone annotation and markup by recognized skin cancer experts. These markups include dermoscopic features (i.e., global and focal morphologic elements in the image known to discriminate between types of skin lesions).
The software infrastructure of the ISIC Archive is built using the open-source Girder platform , and the source code for the Archive itself is freely available on GitHub .
About Melanoma
Skin cancer is a major public health problem, with over 5,000,000 newly diagnosed cases in the United States every year. Melanoma is the deadliest form of skin cancer, responsible for an overwhelming majority of skin cancer deaths. In 2015, the global incidence of melanoma was estimated to be over 350,000 cases, with almost 60,000 deaths. Although the mortality is significant, when detected early, melanoma survival exceeds 95%.
About Dermoscopy
As pigmented lesions occurring on the surface of the skin, melanoma is amenable to early detection by expert visual inspection. It is also amenable to automated detection with image analysis. Given the widespread availability of high-resolution cameras, algorithms that can improve our ability to screen and detect troublesome lesions can be of great value. As a result, many centers have begun their own research efforts on automated analysis. However, a centralized, coordinated, and comparative effort across institutions has yet to be implemented.
Dermoscopy is an imaging technique that eliminates the surface reflection of skin. By removing surface reflection, visualization of deeper levels of skin is enhanced. Prior research has shown that when used by expert dermatologists, dermoscopy provides improved diagnostic accuracy, in comparison to standard photography. As inexpensive consumer dermatoscope attachments for smart phones are beginning to reach the market, the opportunity for automated dermoscopic assessment algorithms to positively influence patient care increases.
