Relative Pixel Prediction For Autoregressive Image Generation
In natural images, transitions between adjacent pixels tend to be smooth and gradual, a fact that has long been exploited in image compression models based on predictive coding. In contrast, existing neural autoregressive image generation models predict the absolute pixel intensities at each position, which is a more challenging problem. In this paper, we propose to predict pixels relatively, by predicting new pixels relative to previously generated pixels (or pixels from the conditioning context, when available). We show that this form of prediction fare favorably to its absolute counterpart when used independently, but their coordination under an unified probabilistic model yields optimal performance, as the model learns to predict sharp transitions using the absolute predictor, while generating smooth transitions using the relative predictor. Experiments on multiple benchmarks for unconditional image generation, image colorization, and super-resolution indicate that our presented mechanism leads to improvements in terms of likelihood compared to the absolute prediction counterparts.
In this paper the authors present a new way to use autoregressive modeling to generate images pixel by pixel where each pixel is generated by modeling the difference between the current pixel value and the preexistent ones. In order to achieve that, the authors propose a copy and adjustment mechanism that select an existing pixel, and then adjust its sub-pixel (channel values) to generate the new pixel. The proposed model is demonstrated with a suite of experiments in classic image generation benchmark. The authors also demonstrate the use of their technique in Image to Image translation. Overall, although the paper explain clearly the intuition and the motivation of the proposed technique, I think that the paper in its present state have low novelty, weak related work analysis review and insufficient experiments to support a publication at ICLR. **Novelty, contribution and related work** The authors should highlight better their main contribution novelty of the proposed method compared to their baseline. **Result and conducted experiments** the correctness of the proposed approach is not proved by the conducted experiment in fact: The experiments do not provide the details of the used architecture compared to your baseline. In Table 1 you report the results using your technique on several computer vision tasks (generation, colorization and super-resolution) but you*re not comparing with the SoA of each of these tasks. The results reported in Tables 1 and 2 are not convincing when compared to existing approaches (using only CIFAR10 in Table2). There are so many missing details specially to validate Image-To-image translation Figure 3 is confusing and not clear **Minor comments** In references section : (Kingma & Dhariwal, 2018) is not in a proper format (nips 2018) Bad quality of illustrations and images Be coherent with the position of captions (figure 3)
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Relative Pixel Prediction For Autoregressive Image Generation
In natural images, transitions between adjacent pixels tend to be smooth and gradual, a fact that has long been exploited in image compression models based on predictive coding. In contrast, existing neural autoregressive image generation models predict the absolute pixel intensities at each position, which is a more challenging problem. In this paper, we propose to predict pixels relatively, by predicting new pixels relative to previously generated pixels (or pixels from the conditioning context, when available). We show that this form of prediction fare favorably to its absolute counterpart when used independently, but their coordination under an unified probabilistic model yields optimal performance, as the model learns to predict sharp transitions using the absolute predictor, while generating smooth transitions using the relative predictor. Experiments on multiple benchmarks for unconditional image generation, image colorization, and super-resolution indicate that our presented mechanism leads to improvements in terms of likelihood compared to the absolute prediction counterparts.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



In this paper the authors present a new way to use autoregressive modeling to generate images pixel by pixel where each pixel is generated by modeling the difference between the current pixel value and the preexistent ones. In order to achieve that, the authors propose a copy and adjustment mechanism that select an existing pixel, and then adjust its sub-pixel (channel values) to generate the new pixel. The proposed model is demonstrated with a suite of experiments in classic image generation benchmark. The authors also demonstrate the use of their technique in Image to Image translation. Overall, although the paper explain clearly the intuition and the motivation of the proposed technique, I think that the paper in its present state have low novelty, weak related work analysis review and insufficient experiments to support a publication at ICLR. **Novelty, contribution and related work** The authors should highlight better their main contribution novelty of the proposed method compared to their baseline. **Result and conducted experiments** the correctness of the proposed approach is not proved by the conducted experiment in fact: The experiments do not provide the details of the used architecture compared to your baseline. In Table 1 you report the results using your technique on several computer vision tasks (generation, colorization and super-resolution) but you*re not comparing with the SoA of each of these tasks. The results reported in Tables 1 and 2 are not convincing when compared to existing approaches (using only CIFAR10 in Table2). There are so many missing details specially to validate Image-To-image translation Figure 3 is confusing and not clear **Minor comments** In references section : (Kingma & Dhariwal, 2018) is not in a proper format (nips 2018) Bad quality of illustrations and images Be coherent with the position of captions (figure 3)
assistant
In this paper the authors present a new way to use autoregressive modeling to generate images pixel by pixel where each pixel is generated by modeling the difference between the current pixel value and the preexistent ones. In order to achieve that, the authors propose a copy and adjustment mechanism that select an existing pixel, and then adjust its sub-pixel (channel values) to generate the new pixel. The proposed model is demonstrated with a suite of experiments in classic image generation benchmark. The authors also demonstrate the use of their technique in Image to Image translation. Overall, although the paper explain clearly the intuition and the motivation of the proposed technique, I think that the paper in its present state have low novelty, weak related work analysis review and insufficient experiments to support a publication at ICLR. **Novelty, contribution and related work** The authors should highlight better their main contribution novelty of the proposed method compared to their baseline. **Result and conducted experiments** the correctness of the proposed approach is not proved by the conducted experiment in fact: The experiments do not provide the details of the used architecture compared to your baseline. In Table 1 you report the results using your technique on several computer vision tasks (generation, colorization and super-resolution) but you*re not comparing with the SoA of each of these tasks. The results reported in Tables 1 and 2 are not convincing when compared to existing approaches (using only CIFAR10 in Table2). There are so many missing details specially to validate Image-To-image translation Figure 3 is confusing and not clear **Minor comments** In references section : (Kingma & Dhariwal, 2018) is not in a proper format (nips 2018) Bad quality of illustrations and images Be coherent with the position of captions (figure 3)