Eye tracking is commonly used in visual neuroscience and cognitive science to answer related questions such as visual attention and decision making. Computational models that predict where to look have direct applications to a variety of computer vision tasks. Due to the inherently complex nature of both the stimuli and the human cognitive process, we envision that bigger eye-tracking data can advance the understanding of these questions and emulating the way humans do. The scale of current eye-tracking experiments, however, are limited as it requires a customized device to track gaze accurately. With our novel psychophysical and crowdsourcing paradigm, SALICON dataset offers a large set of saliency annotations on the popular Microsoft Common Objects in Context (MS COCO) image database. These data complement the task-specific annotations to advance the ultimate goal of visual understanding.