  This paper presents a novel semantic scene change detection scheme with only
weak supervision. A straightforward approach for this task is to train a
semantic change detection network directly from a large-scale dataset in an
end-to-end manner. However, a specific dataset for this task, which is usually
labor-intensive and time-consuming, becomes indispensable. To avoid this
problem, we propose to train this kind of network from existing datasets by
dividing this task into change detection and semantic extraction. On the other
hand, the difference in camera viewpoints, for example, images of the same
scene captured from a vehicle-mounted camera at different time points, usually
brings a challenge to the change detection task. To address this challenge, we
propose a new siamese network structure with the introduction of correlation
layer. In addition, we collect and annotate a publicly available dataset for
semantic change detection to evaluate the proposed method. The experimental
results verified both the robustness to viewpoint difference in change
detection task and the effectiveness for semantic change detection of the
proposed networks. Our code and dataset are available at
https://kensakurada.github.io/pscd.

Computer Vision and Pattern Recognition