The development dataset can be downloaded here.
The forged images are contained in the subfolder dev-dataset-forged and are numbered from 1 to 800. All of them are 1500×2000 color images, either in .tif or .jpg format. The corresponding tampering maps are stored in the subfolder dev-dataset-maps with the same filename.
All the images contain a forgery, which is created starting from a background image by either applying a localized processing or superimposing an external object. Except for few cases, the background images (i.e., the original images before the creation of the forgery) have been taken with 4 cameras. We also provide original flat images taken from such cameras (in the subfolders flat-camera-1, flat-camera-2, flat-camera-3 and flat-camera-4), which can be used to accurately extract the PRNU profiles to be exploited in the forensic analysis.
The performance of the algorithms will be measured in terms of F-measure (or F1-score) between the ground truth and estimated tampering map. This value combines the count of true positives (forged pixels correctly identified), false negatives (forged pixels erroneously identified as non-forged) and false positives (non-forged pixels erroneously identified as forged).
When the teams will submit the estimated tampering maps on the test dataset, we will compute the F-measure for each test image, discard the 5% of them which yield the lower F-measure values and then average the remaining ones. Such average F-measure will be used to determine the final ranking. In order for the teams to self-evaluate their algorithm, we provide a Matlab code for computing the F-measure starting from a ground truth and an estimated tampering map.
Note: do not be afraid by low values of the F-measure! Even though the algorithm provides visually interpretable tampering maps, the F-measure can decrease substantially (way lower than 0.5) due to inaccuracies at the boundaries or isolated false alarms/missed detections. Moreover, in the evaluation phase we will compute the F-measure by using both the submitted tampering maps and their inverted versions (swapping black and white pixels), and consider the maximum value obtained.
In case of issues, please contact: firstname.lastname@example.org.