  Humans communicate with graphical sketches apart from symbolic languages.
Primarily focusing on the latter, recent studies of emergent communication
overlook the sketches; they do not account for the evolution process through
which symbolic sign systems emerge in the trade-off between iconicity and
symbolicity. In this work, we take the very first step to model and simulate
this process via two neural agents playing a visual communication game; the
sender communicates with the receiver by sketching on a canvas. We devise a
novel reinforcement learning method such that agents are evolved jointly
towards successful communication and abstract graphical conventions. To inspect
the emerged conventions, we define three fundamental properties -- iconicity,
symbolicity, and semanticity -- and design evaluation methods accordingly. Our
experimental results under different controls are consistent with the
observation in studies of human graphical conventions. Of note, we find that
evolved sketches can preserve the continuum of semantics under proper
environmental pressures. More interestingly, co-evolved agents can switch
between conventionalized and iconic communication based on their familiarity
with referents. We hope the present research can pave the path for studying
emergent communication with the modality of sketches.

Computation and Language;Computer Vision and Pattern Recognition