Task description

segmentation.jpg

Goal

Participants are asked to submit automated predictions of lesion segmentation boundaries from dermoscopic images.

Data

Lesion segmentation data includes the original image, paired with the expert manual tracing of the lesion boundaries in the form of a binary mask.

Training Data

Download Training Data

The Training Data file is a ZIP file, containing 900 dermoscopic lesion images in JPEG format. All images are named using the scheme ISIC_<image_id>.jpg, where <image_id> is a 7-digit unique identifier. EXIF tags in the images have been removed; any remaining EXIF tags should not be relied upon to provide accurate metadata.

Training Ground Truth

Download Training Ground Truth

The Training Ground Truth file is a ZIP file, containing 900 binary mask images in PNG format. All masks are named using the scheme ISIC_<image_id>_Segmentation.png, where <image_id> matches the corresponding Training Data image for the mask. All mask images will have the exact same dimensions as their corresponding lesion image. Mask images are encoded as single-channel (grayscale) 8-bit PNGs (to provide lossless compression), where each pixel is either: * 0: representing the background of the image, or areas outside the lesion * 255: representing the foreground of the image, or areas inside the lesion

Notes

Masks were created by an expert clinician, using either a semi-automated process (using a user-provided seed point, a user-tuned flood-fill algorithm, and morphological filtering) or a manual process (from a series of user-provided polyline points).

Participants are not strictly required to utilize the training data in the development of their lesion segmentation algorithm and are free to train their algorithm using external data sources.

Submission Format

Test Data

Given the Test Data file, a ZIP file of 379 images of the exact same format as the Training Data, participants are expected to generate and submit a file of Test Results.

The Test Data file should be downloaded via the "Download test dataset" button below, which becomes available once a participant is signed-in and opts to participate in this phase of the challenge.

Test Results

The submitted Test Results file should be in the exact same format as the Training Ground Truth file. Specifically, the Test Results file should be a ZIP file of 379 binary mask images in PNG format. Each mask should contain the participant's best attempt at a fully automated lesion segmentation of the corresponding image in the Test Data. Each mask should be named and encoded according to the conventions of the Training Ground Truth.

Submission Process

Shortly after being submitted, participants will receive a confirmation email to their registered email address to confirm that their submission was parsed and scored, or to provide a notification that parsing of their submission failed (with a link to details as to the cause of the failure). Participants should not consider their submission complete until receiving a confirmation email.

Multiple submissions may be made with absolutely no penalty. Only the most recent submission will be used to determine a participant's final score. Indeed, participants are encouraged to provide trial submissions early to ensure that the format of their submission is parsed and evaluated successfully, even if final results are not yet ready for submission.

Evaluation

Submitted Test Results segmentations will be compared to private (until after the challenge ends) Test Ground Truth. The Test Ground Truth was produced from the exact same source and methodology as the Training Ground Truth (both sets were randomly sub-sampled from a larger data pool).

Submitted segmentations will be compared using a variety of metrics, all computed at the level of single pixels, including: * sensitivity * specificity * accuracy * Jaccard index * Dice coefficient

However, participants will be ranked and awards granted based only on the Jaccard index.

Some useful resources for metrics computation include: * the ROC curve * sklearn library metric functions * jaccard * average precision

Participate

You must be logged in to participate in this task. Signup or login to continue.