Although the state-of-the-art object detection methods are successful in detecting and classifying objects by leveraging deep convolutional neural networks (CNNs), these methods overlook the semantic context which implies the probabilities that different classes of objects occur jointly. In this work, we propose a context-aware CNN (or conCNN for short) that for the first time effectively enforces the semantics context constraints in the CNN-based object detector by leveraging the popular conditional random field (CRF) model in CNN. In particular, conCNN features a context-aware module that naturally models the mean-field inference method for CRF using a stack of common CNN operations. It can be seamlessly plugged into any existing region-based object detection paradigm. Our experiments using COCO datasets showcase that conCNN improves the average precision (AP) of object detection by 2 percentage points, while only introducing negligible extra training overheads.
This paper introduces a context-aware neural network (conCNN) that integrates context semantics into account for object detection. The proposed approach achieves this by embedding a context-aware module into the Faster R-CNN detection framework. The context-aware module simulates the learning process of Conditional Random Fields (CRF) model using a stack of common CNN operations. Specifically, this paper employs the mean-field approach of [1]. Experiments are performed on COCO dataset.
The  paper reads well. The idea of combining the strengths of both CNNs and CRF-based graphical models in a detection framework is interesting. Though the main idea is borrowed from the mean-field approach of [1], it has been successfully reformulated as a stack of common CNN layers. However, my main concern is that the experimental results (Page 8 Table 2) does not support the merits of the proposed approach. When comparing with the existing Faster R-CNN + Relation work of [2], the improvement provided by the proposed approach is negligible. In fact the proposed approach achieves inferior results on large objects (APL), compared to [2]. [1] uses attention to model the geometric relationships among objects in the same image. In addition to geometric relationships, the proposed approach leverages the inherent co-occurence relationships between object classes. However, this additional information does not seem to provide much help by looking at the results on COCO dataset. The paper does present an experiment to justify this additional information in Table 1. However, that small experiment is only on 6 object categories and does not seem to generalize when going towards full COCO dataset with 80 categories (Table 2). Therefore, the reviewer would recommend to thoroughly evaluate the merits of the proposed approach in depth on COCO and on additional datasets (i.e., Visual Genome). 
[1] Philipp Krähenbühl, Vladlen Koltun: Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials. NIPS 2011.
[2] Han Hu, Jiayuan Gu, Zheng Zhang, Jifeng Dai, Yichen Wei: Relation Networks for Object Detection. CVPR 2018.