Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds
Point clouds, as a form of Lagrangian representation, allow for powerful and flexible applications in a large number of computational disciplines. We propose a novel deep-learning method to learn stable and temporally coherent feature spaces for points clouds that change over time. We identify a set of inherent problems with these approaches: without knowledge of the time dimension, the inferred solutions can exhibit strong flickering, and easy solutions to suppress this flickering can result in undesirable local minima that manifest themselves as halo structures. We propose a novel temporal loss function that takes into account higher time derivatives of the point positions, and encourages mingling, i.e., to prevent the aforementioned halos. We combine these techniques in a super-resolution method with a truncation approach to flexibly adapt the size of the generated positions. We show that our method works for large, deforming point sets from different sources to demonstrate the flexibility of our approach.
Summary: This paper proposed a deep network for point cloud sequence super-resolution/upsampling. Building on the basis of pointNet and PU-net, the main contribution of the paper is identifying the problem of temporal incoherence in the process of upsampling a point cloud shape representation as well as a training loss to encourage temporal coherence. In the cases showed in the paper, the proposed method seems effective comparing to previous work which is not done on sequence data. My main concern about the work is that the experimental evaluation is limited. Strengths: Interesting problem and novel idea. The proposed method is technically sound. From the provided results, the newly introduced training loss seems effective: the result sequences are visually more plausible and smooth. Weaknesses: Qualitative results are limited and in most cases seemingly simple. In the paper as well as the companion video, there are very few examples provided. The scale of the evaluation demonstration is not convincing enough for the readers that this work could be generalized to more complicated testing scenarios. Quantitative results are also limited. Since the method is handling the coherence of shape deformation over time, it would be much more convincing and helpful to introduce a dense-correspondence evaluation as a benchmark. For example, one can create ground-truth correspondence from parametric morphable models and evaluate the coherence of the sequence by comparing the generated results with the ground truth.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds
Point clouds, as a form of Lagrangian representation, allow for powerful and flexible applications in a large number of computational disciplines. We propose a novel deep-learning method to learn stable and temporally coherent feature spaces for points clouds that change over time. We identify a set of inherent problems with these approaches: without knowledge of the time dimension, the inferred solutions can exhibit strong flickering, and easy solutions to suppress this flickering can result in undesirable local minima that manifest themselves as halo structures. We propose a novel temporal loss function that takes into account higher time derivatives of the point positions, and encourages mingling, i.e., to prevent the aforementioned halos. We combine these techniques in a super-resolution method with a truncation approach to flexibly adapt the size of the generated positions. We show that our method works for large, deforming point sets from different sources to demonstrate the flexibility of our approach.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



Summary: This paper proposed a deep network for point cloud sequence super-resolution/upsampling. Building on the basis of pointNet and PU-net, the main contribution of the paper is identifying the problem of temporal incoherence in the process of upsampling a point cloud shape representation as well as a training loss to encourage temporal coherence. In the cases showed in the paper, the proposed method seems effective comparing to previous work which is not done on sequence data. My main concern about the work is that the experimental evaluation is limited. Strengths: Interesting problem and novel idea. The proposed method is technically sound. From the provided results, the newly introduced training loss seems effective: the result sequences are visually more plausible and smooth. Weaknesses: Qualitative results are limited and in most cases seemingly simple. In the paper as well as the companion video, there are very few examples provided. The scale of the evaluation demonstration is not convincing enough for the readers that this work could be generalized to more complicated testing scenarios. Quantitative results are also limited. Since the method is handling the coherence of shape deformation over time, it would be much more convincing and helpful to introduce a dense-correspondence evaluation as a benchmark. For example, one can create ground-truth correspondence from parametric morphable models and evaluate the coherence of the sequence by comparing the generated results with the ground truth.
assistant
Summary: This paper proposed a deep network for point cloud sequence super-resolution/upsampling. Building on the basis of pointNet and PU-net, the main contribution of the paper is identifying the problem of temporal incoherence in the process of upsampling a point cloud shape representation as well as a training loss to encourage temporal coherence. In the cases showed in the paper, the proposed method seems effective comparing to previous work which is not done on sequence data. My main concern about the work is that the experimental evaluation is limited. Strengths: Interesting problem and novel idea. The proposed method is technically sound. From the provided results, the newly introduced training loss seems effective: the result sequences are visually more plausible and smooth. Weaknesses: Qualitative results are limited and in most cases seemingly simple. In the paper as well as the companion video, there are very few examples provided. The scale of the evaluation demonstration is not convincing enough for the readers that this work could be generalized to more complicated testing scenarios. Quantitative results are also limited. Since the method is handling the coherence of shape deformation over time, it would be much more convincing and helpful to introduce a dense-correspondence evaluation as a benchmark. For example, one can create ground-truth correspondence from parametric morphable models and evaluate the coherence of the sequence by comparing the generated results with the ground truth.