Towards Metamerism via Foveated Style Transfer
The problem of visual metamerism is defined as finding a family of perceptually indistinguishable, yet physically different images. In this paper, we propose our NeuroFovea metamer model, a foveated generative model that is based on a mixture of peripheral representations and style transfer forward-pass algorithms. Our gradient-descent free model is parametrized by a foveated VGG19 encoder-decoder which allows us to encode images in high dimensional space and interpolate between the content and texture information with adaptive instance normalization anywhere in the visual field. Our contributions include: 1) A framework for computing metamers that resembles a noisy communication system via a foveated feed-forward encoder-decoder network – We observe that metamerism arises as a byproduct of noisy perturbations that partially lie in the perceptual null space; 2) A perceptual optimization scheme as a solution to the hyperparametric nature of our metamer model that requires tuning of the image-texture tradeoff coefficients everywhere in the visual field which are a consequence of internal noise; 3) An ABX psychophysical evaluation of our metamers where we also find that the rate of growth of the receptive fields in our model match V1 for reference metamers and V2 between synthesized samples. Our model also renders metamers at roughly a second, presenting a ×1000 speed-up compared to the previous work, which now allows for tractable data-driven metamer experiments.
 Summary This paper proposes a NeuroFovea (NF) model for generation of point-of-fixation metamers. As opposed to previous algorithms which use gradient descent to match the local texture and image statistics, NF proposed to use a style transfer approach via an Encoder-Decoder style architecture, which allows it to produce metamers in a single forward pass, allowing it to achieve a significant speed-up as compared to early approaches. Pros -The paper tackles a very intriguing topic. -The paper is very well written using concise and clear language allowing it to present a large -amount of information in the 10 pages + appendix. -The paper provides a thorough discussion of both the problem, related work and the model itself. -A single forward pass nature of the model allows it to achieve a 1000x speed-up in generating metamers as opposed to previous GD based approaches. -The authors provide enough details to allow for reproducibility. Cons -(Not necessarily a negative) Requires a very careful reading as the paper provides a lot of information (though as mentioned it is very well written) -The quantitative evaluation is somewhat lacking in that there are no quantitative psychophysical experiments to compare this model to competing ones across different observers. For example, it would have been interesting to compare the ability of observers to distinguish between original images and metamers generated by different models. Additional comments On page 10., you show Fig. 13 however you mention at the end of the first paragraph you further elaborate on Fig 13. in the Supplementary Materials. I think it would be better to either provide more discussion in the text and refer to the figure, or just move it fully to Supplementary materials. Also, in the qualitative comparison of various models you mention that SideEye runs in milliseconds whereas NF runs in seconds. It would be interesting to discuss the potential trade-off between speed and the quality of generated metamers between the models.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Towards Metamerism via Foveated Style Transfer
The problem of visual metamerism is defined as finding a family of perceptually indistinguishable, yet physically different images. In this paper, we propose our NeuroFovea metamer model, a foveated generative model that is based on a mixture of peripheral representations and style transfer forward-pass algorithms. Our gradient-descent free model is parametrized by a foveated VGG19 encoder-decoder which allows us to encode images in high dimensional space and interpolate between the content and texture information with adaptive instance normalization anywhere in the visual field. Our contributions include: 1) A framework for computing metamers that resembles a noisy communication system via a foveated feed-forward encoder-decoder network – We observe that metamerism arises as a byproduct of noisy perturbations that partially lie in the perceptual null space; 2) A perceptual optimization scheme as a solution to the hyperparametric nature of our metamer model that requires tuning of the image-texture tradeoff coefficients everywhere in the visual field which are a consequence of internal noise; 3) An ABX psychophysical evaluation of our metamers where we also find that the rate of growth of the receptive fields in our model match V1 for reference metamers and V2 between synthesized samples. Our model also renders metamers at roughly a second, presenting a ×1000 speed-up compared to the previous work, which now allows for tractable data-driven metamer experiments.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Summary This paper proposes a NeuroFovea (NF) model for generation of point-of-fixation metamers. As opposed to previous algorithms which use gradient descent to match the local texture and image statistics, NF proposed to use a style transfer approach via an Encoder-Decoder style architecture, which allows it to produce metamers in a single forward pass, allowing it to achieve a significant speed-up as compared to early approaches. Pros -The paper tackles a very intriguing topic. -The paper is very well written using concise and clear language allowing it to present a large -amount of information in the 10 pages + appendix. -The paper provides a thorough discussion of both the problem, related work and the model itself. -A single forward pass nature of the model allows it to achieve a 1000x speed-up in generating metamers as opposed to previous GD based approaches. -The authors provide enough details to allow for reproducibility. Cons -(Not necessarily a negative) Requires a very careful reading as the paper provides a lot of information (though as mentioned it is very well written) -The quantitative evaluation is somewhat lacking in that there are no quantitative psychophysical experiments to compare this model to competing ones across different observers. For example, it would have been interesting to compare the ability of observers to distinguish between original images and metamers generated by different models. Additional comments On page 10., you show Fig. 13 however you mention at the end of the first paragraph you further elaborate on Fig 13. in the Supplementary Materials. I think it would be better to either provide more discussion in the text and refer to the figure, or just move it fully to Supplementary materials. Also, in the qualitative comparison of various models you mention that SideEye runs in milliseconds whereas NF runs in seconds. It would be interesting to discuss the potential trade-off between speed and the quality of generated metamers between the models.
assistant
 Summary This paper proposes a NeuroFovea (NF) model for generation of point-of-fixation metamers. As opposed to previous algorithms which use gradient descent to match the local texture and image statistics, NF proposed to use a style transfer approach via an Encoder-Decoder style architecture, which allows it to produce metamers in a single forward pass, allowing it to achieve a significant speed-up as compared to early approaches. Pros -The paper tackles a very intriguing topic. -The paper is very well written using concise and clear language allowing it to present a large -amount of information in the 10 pages + appendix. -The paper provides a thorough discussion of both the problem, related work and the model itself. -A single forward pass nature of the model allows it to achieve a 1000x speed-up in generating metamers as opposed to previous GD based approaches. -The authors provide enough details to allow for reproducibility. Cons -(Not necessarily a negative) Requires a very careful reading as the paper provides a lot of information (though as mentioned it is very well written) -The quantitative evaluation is somewhat lacking in that there are no quantitative psychophysical experiments to compare this model to competing ones across different observers. For example, it would have been interesting to compare the ability of observers to distinguish between original images and metamers generated by different models. Additional comments On page 10., you show Fig. 13 however you mention at the end of the first paragraph you further elaborate on Fig 13. in the Supplementary Materials. I think it would be better to either provide more discussion in the text and refer to the figure, or just move it fully to Supplementary materials. Also, in the qualitative comparison of various models you mention that SideEye runs in milliseconds whereas NF runs in seconds. It would be interesting to discuss the potential trade-off between speed and the quality of generated metamers between the models.