  Natural languages are powerful tools wielded by human beings to communicate
information. Among their desirable properties, compositionality has been the
main focus in the context of referential games and variants, as it promises to
enable greater systematicity to the agents which would wield it. The concept of
disentanglement has been shown to be of paramount importance to learned
representations that generalise well in deep learning, and is thought to be a
necessary condition to enable systematicity. Thus, this paper investigates how
do compositionality at the level of the emerging languages, disentanglement at
the level of the learned representations, and systematicity relate to each
other in the context of visual referential games. Firstly, we find that visual
referential games that are based on the Obverter architecture outperforms
state-of-the-art unsupervised learning approach in terms of many major
disentanglement metrics. Secondly, we expand the previously proposed Positional
Disentanglement (PosDis) metric for compositionality to (re-)incorporate some
concerns pertaining to informativeness and completeness features found in the
Mutual Information Gap (MIG) disentanglement metric it stems from. This
extension allows for further discrimination between the different kind of
compositional languages that emerge in the context of Obverter-based
referential games, in a way that neither the referential game accuracy nor
previous metrics were able to capture. Finally we investigate whether the
resulting (emergent) systematicity, as measured by zero-shot compositional
learning tests, correlates with any of the disentanglement and compositionality
metrics proposed so far. Throughout the training process, statically
significant correlation coefficients can be found both positive and negative
depending on the moment of the measure.

Computation and Language