Can Peripheral Representations Improve Clutter Metrics on Complex Scenes?
Previous studies have proposed image-based clutter measures that correlate with human search times and/or eye movements. However, most models do not take into account the fact that the effects of clutter interact with the foveated nature of the human visual system: visual clutter further from the fovea has an increasing detrimental influence on perception. Here, we introduce a new foveated clutter model to predict the detrimental effects in target search utilizing a forced fixation search task. We use Feature Congestion (Rosenholtz et al.) as our non foveated clutter model, and we stack a peripheral architecture on top of Feature Congestion for our foveated model. We introduce the Peripheral Integration Feature Congestion (PIFC) coefficient, as a fundamental ingredient of our model that modulates clutter as a non-linear gain contingent on eccentricity. We show that Foveated Feature Congestion (FFC) clutter scores (r(44) = −0.82 ± 0.04, p < 0.0001) correlate better with target detection (hit rate) than regular Feature Congestion (r(44) = −0.19 ± 0.13, p = 0.0774) in forced fixation search; and we extend foveation to other clutter models showing stronger correlations in all cases. Thus, our model allows us to enrich clutter perception research by computing fixation specific clutter maps. Code for building peripheral representations is available 1 .
The paper proposes a new clutter model that integrates peripheral information on top of an existing clutter model (Feature Congestion). Their model was developed for predicting target detection rate, with targets situated at different eccentricities of a fixation point.  The paper also proposed an experimental design for collecting the behavioral data in a forced fixation search experiment. The results showed that the foveated version of the feature congestion model was able to better explain the target detection rates at different eccentricities than the original feature congestion model.
This work suffers from several issues in my opinion: problem statement, novelty and technical contribution, and experiments.  Problem statement: The author states that most clutter models produce a global metric to measure clutter perception, but the fovea effect have not been taken into account. Then they argue that the fovea effect should be a part of the clutter model, where the problem is framed as a target hit-rate modeling study. I wonder, why is this an interesting problem? As the current clutter models (other than feature congestion) have achieved high correlations with human clutter rankings, why would it even be important to add in the peripheral structure into an existing model, where such existing model is a good margin behind the current state-of-the-art (feature congestion). Also, the problem is centered around the “forced fixation search” experiment, which means the subject is fixating at a specified location while the target is somewhere in the image. This leads to the foveated clutter model to take in a fixation-location input parameter, in order to generate the peripheral structure as part of the pipeline. However, this is disturbing because I do not see under any circumstance would a viewer being forced to fixate at some part of an UI, control panel, advertisement, or any visual display, because in a real world scenario people perform free viewing on the displays. Therefore, in terms of the application of clutter perception models, the forced-fixation points should at least be chosen meaningfully, such as with a fixation-prediction algorithm, or obtain human fixations from an initial free-viewing experiment, then use those as the forced-fixation locations for the actual search experiment.  Novelty and technical contribution: This work has very limited technical scope, as it is addressing a very specific scope of clutter perception modeling: forced fixation search problem. The work combines an existing peripheral structure work onto an existing clutter model (feature congestion) in a very limited scope of clutter perception. The proposed forced-fixation search experiment is also very specific that does not offer sufficient general contribution to the NIPS community. In addition, the feature congestion model is a good margin behind the current state-of-the-art in predicting global human clutter perception, it would be more desirable for the authors to come up with an approach to base their method on the state-of-the-art clutter model, such as the proto-object model.   Experiments: As mentioned previously, if the authors want to build on top of an existing work, it would be more desirable to be adopting the state-of-the-art. For the experiments to be more convincing, the authors should try to propose a more general framework that is able to be combined with most of the clutter model, so that the experiments can include the foveated feature congestion, foveated proto-object model, foveated scale invariance model, etc. With the current experiments of comparing the proposed model against the original feature congestion model, the results are underwhelming because the authors are comparing both methods to the forced-fixation search data, where the original feature congestion model was never designed for that specific task anyway, so of course it would not do well as compared to the proposed method that is designed specifically to solve this issue. A better claim should be showing the foveated model can do better than other clutter models in terms of the general clutter ranking task, not at a task that is specifically designed for the foveated model. Therefore, the comparison part is weak and should include more models in a more general scope.