  Fake news often involves semantic manipulations across modalities such as
image, text, location etc and requires the development of multimodal semantic
forensics for its detection. Recent research has centered the problem around
images, calling it image repurposing -- where a digitally unmanipulated image
is semantically misrepresented by means of its accompanying multimodal metadata
such as captions, location, etc. The image and metadata together comprise a
multimedia package. The problem setup requires algorithms to perform multimodal
semantic forensics to authenticate a query multimedia package using a reference
dataset of potentially related packages as evidences. Existing methods are
limited to using a single evidence (retrieved package), which ignores potential
performance improvement from the use of multiple evidences. In this work, we
introduce a novel graph neural network based model for multimodal semantic
forensics, which effectively utilizes multiple retrieved packages as evidences
and is scalable with the number of evidences. We compare the scalability and
performance of our model against existing methods. Experimental results show
that the proposed model outperforms existing state-of-the-art algorithms with
an error reduction of up to 25%.

Multimedia;Artificial Intelligence;Computer Vision and Pattern Recognition