By May 29, 2015, to participate in the LSUN Saliency Prediction Challenge, please download the Matlab version of data files and toolkit at the challenge website.
1. Download
Please follow the instructions in the README to download and setup the SALICON data (annotations and images). Note that the Python tools provide a script to automatically download the data and setup the environment.
Images
The current release contains 10,000 training images and 5,000 validation images with saliency ground-truth. The test set with 5,000 images is released without ground-truth. All images are selected from MS COCO 2014 release.
Training (1.5G) Validation (0.8G) Test (0.8G)
Annotations
The ground-truth saliency annotations include fixations generated from mouse trajectories. The data format is consistent with MS COCO.
Training (818M) Validation (459M)
Tools
Download our Python API and evaluation tools.
2. SALICON API
Our API inherits from the MS COCO API. Please read the specification of the MS COCO API before proceeding. The functions listed here are provided for accessing saliency annotations on the SALICON dataset. To access other annotations from MS COCO, please use the MS COCO API.
buildFixMapBuild the fixation map for the given fixation annotations. getAnnIdsGet ann ids that satisfy given filter conditions. getImgIdsGet img ids that satisfy given filter conditions. loadAnnsLoad anns with the specified ids. loadImgsLoad imgs with the specified ids. loadResLoad algorithm results and create API for accessing them. getAnnIdsGet ann ids that satisfy given filter conditions. showAnnsDisplay the specified annotations (fixation maps for grount-truth annotations and saliency maps for results). encodeImageEncode an image file (i.e., saliency map) using base64 encoding decodeImageDecode an encoded saliency map via base64 encoding
Here is a link to the Python API demo.
3. Annotation format
SALICON adds a new annotation type to the MS COCO annotations: fixations. The annotations are stored using the JSON file format. All annotations share the basic data structure below:
{“info” : info,“type” : str,“images” : [image],“annotations” : [annotation],“licenses” : [license],}
info {“year” : int,“version” : str,“description” : str,“contributor” : str,“url” : str,“date_created” : datetime,}
images[{“id” : int,“width” : int,“height” : int,“file_name” : str,“license” : int,“url” : str, “date_captured” : datetime,}]
licenses[{“id” : int,“name” : str,“url” : str,}]
Our new annotation type is “fixations”. Each fixation annotation contains a series of fields, including image_id, worker_id and fixations. The field image_id is the same as the original MS COCO image id. The field worker_id indicates the AMT worker who produced the fixations in this annotation. The field fixations contains the fixations the subject produced. The format of fixations is a list of tuples representing the image coordinates (1-indexed coordinates). The first element of the tuple represents the coordinate in the row axis, and the second element of the tuple represents the coordinate in the column axis.