1. Download

Please follow the instructions in the README to download and setup the SALICON data (annotations and images). Note that the Python tools provide a script to automatically download the data and setup the environment.

Images

The current release contains 10,000 training images and 5,000 validation images with saliency ground-truth.  The test set with 5,000 images is released without ground-truth. All images are selected from MS COCO 2014 release.

Images (3.0G)

 

Tools

Download our Python API and evaluation tools.

 Annotations

The ground-truth saliency annotations include fixations generated from mouse trajectories. The data format is consistent with MS COCO.

Training (818M)  Validation (459M)

2. SALICON API

Our API inherits from the MS COCO API. Please read the specification of the MS COCO API before proceeding. The functions listed here are provided for accessing saliency annotations on the SALICON dataset. To access other annotations from MS COCO, please use the MS COCO API.

buildFixMap
Build the fixation map for the given fixation annotations.
getAnnIds
Get ann ids that satisfy given filter conditions.
getImgIds
Get img ids that satisfy given filter conditions.
loadAnns
Load anns with the specified ids.
loadImgs
Load imgs with the specified ids.
loadRes
Load algorithm results and create API for accessing them.
getAnnIds
Get ann ids that satisfy given filter conditions.
showAnns
Display the specified annotations (fixation maps for grount-truth annotations and saliency maps for results).
encodeImage
Encode an image file (i.e., saliency map) using base64 encoding
decodeImage
Decode an encoded saliency map via base64 encoding

Here is a link to the Python API demo.

3. Annotation format

SALICON adds a new annotation type to the MS COCO annotations: fixations. The annotations are stored using the JSON file format. All annotations share the basic data structure below:

{
“info”
 : info,
“type”
 : str,
“images”
 : [image],
“annotations”
 : [annotation],
“licenses”
 : [license],
}
info {
“year”
: int,
“version”
: str,
“description”
: str,
“contributor”
: str,
“url”
: str,
“date_created”
: datetime,
}
 
images[{
“id”
: int,
“width”
: int,
“height”
: int,
“file_name”
: str,
“license”
: int,
“url”
: str,
 
“date_captured”
: datetime,
}]
licenses[{
“id”
: int,
“name”
: str,
“url”
: str,
}]

Our new annotation type is “fixations”. Each fixation annotation contains a series of fields, including image_id, worker_id and fixations. The field image_id is the same as the original MS COCO image id. The field worker_id indicates the AMT worker who produced the fixations in this annotation. The field fixations contains the fixations the subject produced. The format of fixations is a list of tuples representing the image coordinates (1-indexed coordinates). The first element of the tuple represents the coordinate in the row axis, and the second element of the tuple represents the coordinate in the column axis.

annotations[{
“id”
: int,
“image_id”
: int,
“worker_id”
: int,
“fixations”
: [[row0,col0],[row1,col1],…]
}]