Adversarial examples raise questions about whether neural network models are sensitive to the same visual features as humans. In this paper, we first detect adversarial examples or otherwise corrupted images based on a class-conditional reconstruction of the input. To specifically attack our detection mechanism, we propose the Reconstructive Attack which seeks both to cause a misclassification and a low reconstruction error. This reconstructive attack produces undetected adversarial examples but with much smaller success rate. Among all these attacks, we find that CapsNets always perform better than convolutional networks. Then, we diagnose the adversarial examples for CapsNets and find that the success of the reconstructive attack was proportional to the visual similarity between the source and target class. Additionally, the resulting perturbations can cause the input image to appear visually more like the target class and hence become nonadversarial. These suggest that CapsNets use features that are more aligned with human perception and address the central issue raised by adversarial examples.
This paper studies the problem of detecting and generating adversarial images using class-conditional capsule networks. Specifically, this paper first introduced a novel method that detects adversarial examples by class-conditional image reconstruction. Motivated by this defense method, this paper further proposed a novel reconstructive attack that minimizes both classification and reconstruction loss. Experimental evaluations are conducted on MNIST, FashionMNIST, SVHN, and CIFAR-10 dataset. Results demonstrate the effectiveness of the proposed defense and the novel reconstructive attack method.
Overall, this paper is well-motivated and presentation is clear. It proposed a smart way of using the class-conditional generative model to improve the adversarial robustness. Please address the following questions.
(1) Reviewer’s major concern is that this method is not very scalable to large-scale real-world datasets such as ImageNet and SUN database. First, training a class-conditional generative model on MNIST is relatively easy compared to ImageNet. The generative model could potentially create image artifacts on higher resolution images. 
-- SUN Database: Large-scale Scene Recognition from Abbey to Zoo. Xiao et al. In CVPR 2010.
Second, the proposed proxy based on l_2 image distance might not be effective at all for higher-resolution images.
-- A note on the evaluation of generative models, Theis et al. In ICLR 2016.
-- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, Zhang et al. In CVPR 2018.
(2) Reviewer is not fully convinced by the argument that features learned by CapsNets are superior to features learned by CNN baselines. To draw such conclusion, it is necessary to conduct systematic experiments with different CapsNets and CNNs architectures (e.g., number of layers) and other hyper-parameters related to the adversarial optimization.
(3) It looks like the proposed method is not specific to generative models use class labels as condition. reviewer is curious whether the method generalizes to other conditions (e.g., image-to-image translation) as well.
-- Learning Structured Output Representation using Deep Conditional Generative Models, Sohn et al. In NIPS 2015.
-- Image-to-Image Translation with Conditional Adversarial Nets, Isola et al. In CVPR 2017.
-- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, Zhu et al. In ICCV 2017.
-- Semantic Image Synthesis with Spatially-Adaptive Normalization, Park et al. In CVPR 2019.
(4) Can you possibly comment on the attack transferability compared to other existing attacks evaluated in this paper?
(5) Detection threshold: Can you possibly draw the AOC curve or report the Area Under Curve (AOC) score?