DeepGaze III: Using Deep Learning to Probe Interactions Between Scene Content and Scanpath History in Fixation Selection
Many animals make eye movements to gather relevant visual information about the environment. How fixation locations are selected has been debated for decades in neuroscience and psychology. One hypothesis states that "priority" or "saliency" values are assigned locally to image locations, independent of saccade history, and are only later combined with saccade history and other constraints to select the next fixation location. A second hypothesis is that there are interactions between saccade history and image content that cannot be summarised by a single value. Here we discriminate between these possibilities in a data driven manner. Using transfer learning from the VGG deep neural network, we train a model of scanpath prediction "DeepGaze III" on human free viewing eye scanpath data. DeepGaze III can either be forced to use a single saliency map or can be allowed to learn complex interactions via multiple saliency maps. We find that using multiple saliency maps gives no advantage in scanpath prediction compared to a single saliency map. This suggesest that at least for free viewing no complex interactions between scene content and scanpath history exist and a single saliency map may exist that does not depend on either current or previous gaze locations.