  Ontologies encompass a formal representation of knowledge through the
definition of concepts or properties of a domain, and the relationships between
those concepts. In this work, we seek to investigate whether using this
ontological information will improve learning from weakly labeled data, which
are easier to collect since it requires only the presence or absence of an
event to be known. We use the AudioSet ontology and dataset, which contains
audio clips weakly labeled with the ontology concepts and the ontology
providing the "Is A" relations between the concepts. We first re-implemented
the model proposed by soundevent_ontology with modification to fit the
multi-label scenario and then expand on that idea by using a Graph
Convolutional Network (GCN) to model the ontology information to learn the
concepts. We find that the baseline Siamese does not perform better by
incorporating ontology information in the weak and multi-label scenario, but
that the GCN does capture the ontology knowledge better for weak, multi-labeled
data. In our experiments, we also investigate how different modules can
tolerate noises introduced from weak labels and better incorporate ontology
information. Our best Siamese-GCN model achieves mAP=0.45 and AUC=0.87 for
lower-level concepts and mAP=0.72 and AUC=0.86 for higher-level concepts, which
is an improvement over the baseline Siamese but about the same as our models
that do not use ontology information.

Sound;Machine Learning;Multimedia;Audio and Speech Processing