Quantifying the Cost of Reliable Photo Authentication via High-Performance Learned Lossy Representations
Detection of photo manipulation relies on subtle statistical traces, notoriously removed by aggressive lossy compression employed online. We demonstrate that end-to-end modeling of complex photo dissemination channels allows for codec optimization with explicit provenance objectives. We design a lightweight trainable lossy image codec, that delivers competitive rate-distortion performance, on par with best hand-engineered alternatives, but has lower computational footprint on modern GPU-enabled platforms. Our results show that significant improvements in manipulation detection accuracy are possible at fractional costs in bandwidth/storage. Our codec improved the accuracy from 37% to 86% even at very low bit-rates, well below the practicality of JPEG (QF 20).
This paper presents a learned image compression method that is able to be robust under a variety of tasks. The results aren*t state of the art in terms of rate-distortion performance, but this paper has a very good analysis of the results, and has produced a very fast codec. In that sense, this is a very interesting paper that may lead to other fast methods (the other fast method they compared the runtime against - WaveOne never published a complete description). This paper should be likely accepted, but the authors should town down the claims a bit. The results presented do NOT show that this method is better than the best hand engineered approach, despite what they claim. Even compared to BPG, which is NOT state of the art, the results are a mixed bag. We would like to point out to the authors that the VVC codec has shown much stronger performance than BPG, and similarly the AV1 codec has surpassed the performance of BPG. Moreover, even Pik has also surpassed the performance of BPG, so just showing stronger performance than BPG is not grounds to make the claim that this method is superior to hand engineered approaches. Moreover, as I stated earlier, this method is not even better all the time, therefore weakening the claim. On the positives: - the paper fully describes the architecture, unlike WaveOne - the runtime numbers are impressive (as far as I know, there is no faster published method) - the authors consider applications other than compression performance (such as classification performance in forensic analysis) On the negatives, which I highly suggest that the authors fix if this paper is to be taken seriously by the community: - please be sure to explain that SSIM is computed in <RGB | grayscale> - please be more explicit about which loss is used during training for distortion (i.e., *we use MSE for the training loss, but stop training when SSIM converges*) - please provide PSNR numbers for the method; and ideally MS-SSIM (in decibels) instead of PSNR - please add other neural compression methods to the graphs - please clarify that you create a file and decode a file for each image used to create the graphs (very important topic), as opposed to using the estimated file size - tone down the claims w.r.t. beating classical codecs
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Quantifying the Cost of Reliable Photo Authentication via High-Performance Learned Lossy Representations
Detection of photo manipulation relies on subtle statistical traces, notoriously removed by aggressive lossy compression employed online. We demonstrate that end-to-end modeling of complex photo dissemination channels allows for codec optimization with explicit provenance objectives. We design a lightweight trainable lossy image codec, that delivers competitive rate-distortion performance, on par with best hand-engineered alternatives, but has lower computational footprint on modern GPU-enabled platforms. Our results show that significant improvements in manipulation detection accuracy are possible at fractional costs in bandwidth/storage. Our codec improved the accuracy from 37% to 86% even at very low bit-rates, well below the practicality of JPEG (QF 20).
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper presents a learned image compression method that is able to be robust under a variety of tasks. The results aren*t state of the art in terms of rate-distortion performance, but this paper has a very good analysis of the results, and has produced a very fast codec. In that sense, this is a very interesting paper that may lead to other fast methods (the other fast method they compared the runtime against - WaveOne never published a complete description). This paper should be likely accepted, but the authors should town down the claims a bit. The results presented do NOT show that this method is better than the best hand engineered approach, despite what they claim. Even compared to BPG, which is NOT state of the art, the results are a mixed bag. We would like to point out to the authors that the VVC codec has shown much stronger performance than BPG, and similarly the AV1 codec has surpassed the performance of BPG. Moreover, even Pik has also surpassed the performance of BPG, so just showing stronger performance than BPG is not grounds to make the claim that this method is superior to hand engineered approaches. Moreover, as I stated earlier, this method is not even better all the time, therefore weakening the claim. On the positives: - the paper fully describes the architecture, unlike WaveOne - the runtime numbers are impressive (as far as I know, there is no faster published method) - the authors consider applications other than compression performance (such as classification performance in forensic analysis) On the negatives, which I highly suggest that the authors fix if this paper is to be taken seriously by the community: - please be sure to explain that SSIM is computed in <RGB | grayscale> - please be more explicit about which loss is used during training for distortion (i.e., *we use MSE for the training loss, but stop training when SSIM converges*) - please provide PSNR numbers for the method; and ideally MS-SSIM (in decibels) instead of PSNR - please add other neural compression methods to the graphs - please clarify that you create a file and decode a file for each image used to create the graphs (very important topic), as opposed to using the estimated file size - tone down the claims w.r.t. beating classical codecs
assistant
This paper presents a learned image compression method that is able to be robust under a variety of tasks. The results aren*t state of the art in terms of rate-distortion performance, but this paper has a very good analysis of the results, and has produced a very fast codec. In that sense, this is a very interesting paper that may lead to other fast methods (the other fast method they compared the runtime against - WaveOne never published a complete description). This paper should be likely accepted, but the authors should town down the claims a bit. The results presented do NOT show that this method is better than the best hand engineered approach, despite what they claim. Even compared to BPG, which is NOT state of the art, the results are a mixed bag. We would like to point out to the authors that the VVC codec has shown much stronger performance than BPG, and similarly the AV1 codec has surpassed the performance of BPG. Moreover, even Pik has also surpassed the performance of BPG, so just showing stronger performance than BPG is not grounds to make the claim that this method is superior to hand engineered approaches. Moreover, as I stated earlier, this method is not even better all the time, therefore weakening the claim. On the positives: - the paper fully describes the architecture, unlike WaveOne - the runtime numbers are impressive (as far as I know, there is no faster published method) - the authors consider applications other than compression performance (such as classification performance in forensic analysis) On the negatives, which I highly suggest that the authors fix if this paper is to be taken seriously by the community: - please be sure to explain that SSIM is computed in <RGB | grayscale> - please be more explicit about which loss is used during training for distortion (i.e., *we use MSE for the training loss, but stop training when SSIM converges*) - please provide PSNR numbers for the method; and ideally MS-SSIM (in decibels) instead of PSNR - please add other neural compression methods to the graphs - please clarify that you create a file and decode a file for each image used to create the graphs (very important topic), as opposed to using the estimated file size - tone down the claims w.r.t. beating classical codecs