1. Prepare JSON Files

Before uploading your results to the evaluation server, you will need to create two JSON files containing your results in the correct format. One file should correspond to your results on the validation dataset, and the other to the testing dataset. Your files should be named as follows:

results.zip

[type]_val2014_[alg]_results.json
[type]_test2014_[alg]_results.json

Here replace [type] with the competition type (fixations), and [alg] with the name of your algorithm. We have provided a Python tool to build the result JSON file from a folder of saliency maps. Your two files [type]_val2014_[alg]_results.json and [type]_test2014_[alg]_results.json should then be placed into a zip file for upload. Note that both files are required for submission.

2. Create CodaLab Account and Join the Competition

Next you will need to create an account on CodaLab. Please follow the MS COCO challenge tutorials to participate in the SALICON challenge.

3. Download Your Results

You will have the option to download your evaluation results once the evaluation is finished. The zip file will contain five evaluation result files:

[type]_val2014_[alg]_evalImgs.json % per image evaluation on val
[type]_val2014_[alg]_eval.json % aggregated evaluation on val
[type]_test2014_[alg]_eval.json % aggregated evaluation on test
metadata % automatically generated (safe to ignore)
scores.txt % automatically generated (safe to ignore)

The format of the evaluation files is described on the evaluation page. Please note that the *_evalImgs.json file is only available for download on the validation dataset, and not the test set.