Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction
Recent works have shown that exploiting multi-scale representations deeply learned via convolutional neural networks (CNN) is of tremendous importance for accurate contour detection. This paper presents a novel approach for predicting contours which advances the state of the art in two fundamental aspects, i.e. multi-scale feature generation and fusion. Different from previous works directly considering multi-scale feature maps obtained from the inner layers of a primary CNN architecture, we introduce a hierarchical deep model which produces more rich and complementary representations. Furthermore, to refine and robustly fuse the representations learned at different scales, the novel Attention-Gated Conditional Random Fields (AG-CRFs) are proposed. The experiments ran on two publicly available datasets (BSDS500 and NYUDv2) demonstrate the effectiveness of the latent AG-CRF model and of the overall hierarchical framework.
This paper proposes a gating mechanism to combine features from different levels in a CNN for the task of contour detection. The paper builds upon recent advances in using graphical models with CNN architectures [5,39] and augments these with attention. This results in large improvements in performance at the task of contour detection, achieving new state-of-the-art results. The paper also presents an ablation study where they analyze the impact of different parts of their architecture.
Cons:
1. Unclear relationship to past works which use CRFs with CNNs [5,39] and other works such as [A,B] which express CRF inference as CNNs. The paper says it is inspired from [5,39] but does not describe the points of difference from [5, 39]. Is only attention the difference and thus the novelty? Does the derivation in Section 2 follow directly from [5,39] (except taking attention into account) or it requires additional tricks and techniques beyond [5,39]? How does the derivation relate to the concepts presented in [A]? Not only will these relationships make it easier to judge novelty, it will also allow readers to contextualize the work better.
2. Intuitively, I dont understand why we need attention to combine the information from different scales. The paper is fairly vague about the motivation for this. It would help if the rebuttal could provide some examples or more intuition about what a gating mechanism can provide that a non-gating mechanism can't provide.
3. The paper presents very thorough experimental results for contour detection, but I will encourage the authors to also study at the very least one other task (say semantic segmentation). This will help evaluate how generalizable the technique is. This will also increase the impact of the paper in the community.
4. The paper is somewhat hard to read. I believe there are some undefined variables, I haven't been able to figure out what f_l^{D,C,M} in figure 2 are.
5. There are minor typos in Table 2 (performance for [14] is misreported, ODS is either 70.25 or 71.03 in [14] but the current paper reports it as 64.1), and these numbers are not consistent with the ones in Figure 4. I would recommend that the authors cross-check all numbers in the draft.
[A] Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Philipp Krähenbühl and Vladlen Koltun
NIPS 2011
[B] Conditional Random Fields as Recurrent Neural Networks. ICCV 2015