summaries: The authors refer to prior work in sociolinguistic literature to state that larger communities create more systematic languages. However, they point out, this apparent correlation between language structure and population size has evaded machine learning practitioners studying language emergence. This paper claims that populations explored in machine learning have largely been homogeneous and that population heterogeneity is key to the emergence of structure in artificial agents. The authors reproduce the failure to achieve systematic languages by scaling up the population and then show that they can indeed get more structure by introducing heterogeneity. They explore introducing asymmetry between the speaker and the listener based on model capacity and learning speed, leading to an increase in language structure when the speaker is faster or has more capacity. The authors note that this effect only depends on the relative differences between the speaker and the listener and not the absolute values (there is a correlation with absolute values but the variation has a low magnitude). Further, they show that larger networks need fewer epochs to reach similar training accuracy, concluding that network capacity is a confounding factor of training speed in their setting. Finally, the authors create a population of heterogeneous agents by imbibing them with different learning speeds by updating an agent $i$ with probability $p_i$ after each round of the Lewis game. Four properties of the emergent language are studied: speakers synchronization, (negative) conditional entropy given an object, topographic similarity, and generalization. These metrics either improve or remain approximately at the same values as the population size increases. # Summary  The manuscript "On the role of population heterogeneity in emergent communication" addresses the question why results on population size in deep language emergence have not, so far, mirrored the effects it is claimed to have on natural language. In a nutshell, in natural communication population size correlates with simpler grammars and less idiosyncratic languages. The effects of population size on the structure of neural emergent communication is less explored and has hitherto not reflected what we know about natural language. The authors argue that one of the reasons for this is that artificial populations are often homogeneous. They show, through simulations, that introducing asymmetries in the training speed of speakers and listeners leads to trends that are more in line with natural language: the size of heterogeneous populations weakly correlates with more aligned languages; higher neg-entropy; better generalization when communicating about novel objects; and more compositional languages (measured as topographic similarity).  This paper analyzes the structure of emergent languages in signaling games played by _populations_ of agents, motivated by a rich body of socio- and psycho-linguistics results showing that languages spoken by more people (and with more second-language learners) tend to be grammatically simpler (e.g. have a more impoverished morphology).  The authors measure various properties of the emergent languages as proxies for how "systematic" a language is, and show several things.  First, increasing the number of agents does not correlate with any of their measures, so population size alone does not suffice.  Second, in a minimally small population, various measures of "diversity" of the two agents do correlate with their systematicity measures.  Finally, in a large but diverse population, we do see correlations with population size and _some_ of the systematicity measures.  The paper is interesting and timely, and reports on a large number of experiments.  While I think the experiments could be more closely linked to the hypotheses from the sociolingusitic literature, the paper will be of interest to many researchers in emergent communication, NLP, and cognitive science, and could spur future work in this intersection. This paper aims at solving a conflicting empirical observation and the present-state models for emerging languages. It has been observed that larger populations produce more structured languages. However, the state-of-art neural-based models have not been able to generate languages with such characteristics. This paper shows that a key ingredient is to allow population heterogeneity in the neural models rather than the current identically distributed specifications. 
abstractText: Populations have often been perceived as a structuring component for language to emerge and evolve: the larger the population, the more structured the language. While this observation is widespread in the sociolinguistic literature, it has not been consistently reproduced in computer simulations with neural agents. In this paper, we thus aim to clarify this apparent contradiction. We explore emergent language properties by varying agent population size in the speaker-listener Lewis Game. After reproducing the experimental difference, we challenge the simulation assumption that the agent community is homogeneous. We first investigate how speaker-listener asymmetry alters language structure to examine two potential diversity factors: training speed and network capacity. We find out that emergent language properties are only altered by the relative difference of factors between speaker and listener, and not by their absolute values. From then, we leverage this observation to control population heterogeneity without introducing confounding factors. We finally show that introducing such training speed heterogeneities naturally sort out the initial contradiction: larger simulated communities start developing more stable and structured languages.
references.authors: Marco Baroni
references.title: Linguistic generalization and compositionality in modern artificial neural networks
references.venue: Philosophical Transactions of the Royal Society B: Biological Sciences,
references.authors: Clay Beckner Janet B Pierrehumbert Jennifer Hay
references.title: The emergence of linguistic structure in an online iterated learning task
references.venue: Journal of Language Evolution,
references.authors: Derek Bickerton
references.title: Language evolution: A brief guide for linguists
references.venue: Lingua,
references.authors: Daniel T. Blumstein Kenneth B. Armitage
references.title: Does sociality drive the evolution of communicative complexity? a comparative test with ground-dwelling sciurid alarm calls
references.venue: The American Naturalist,
references.authors: Diane Bouchacourt Marco Baroni
references.title: Miss tools and mr fruit: Emergent communication in agents learning about object affordances
references.venue: In Proc. of the Association for Computational Linguistics (ACL),
references.authors: Henry Brighton Simon Kirby
references.title: Understanding Linguistic Evolution by Visualizing the Emergence of Topographic Mappings
references.venue: Artificial Life, 12(2):229–242,
references.authors: Ted Briscoe
references.title: Linguistic evolution through language acquisition
references.authors: Lindell Bromham Xia Hua Thomas G. Fitzpatrick Simon J. Greenhill
references.title: Rate of language evolution is affected by population size
references.venue: Proc. of the National Academy of Sciences,
references.authors: Rahma Chaabouni Eugene Kharitonov Emmanuel Dupoux Marco Baroni
references.title: Anti-efficient encoding in emergent communication
references.venue: In Proc. of Advances in Neural Information Processing Systems (NeurIPS),
references.authors: Rahma Chaabouni Eugene Kharitonov Diane Bouchacourt Emmanuel Dupoux Marco Baroni
references.title: Compositionality and generalization in emergent languages
references.venue: In Proc. of the Association for Computational Linguistics (ACL),
references.authors: Edward Choi Angeliki Lazaridou Nando de Freitas
references.title: Multi-agent compositional communication learning from raw visual input
references.venue: In Proc. of International Conference on Learning Representations (ICLR),
references.authors: Michael Clyne
references.title: Linguistic and sociolinguistic aspects of language contact, maintenance and loss
references.venue: Maintenance and loss of minority languages,
references.authors: Michael Cogswell Jiasen Lu Stefan Lee Devi Parikh Dhruv Batra
references.title: Emergence of compositional language with deep generational transmission
references.authors: Kevin Denamganaı James Alfred Walker
references.title: On (emergent) systematic generalisation and compositionality in visual referential games with straight-through gumbel-softmax estimator
references.venue: arXiv preprint arXiv:2012.10776,
references.authors: Nick C Ellis
references.title: The dynamics of second language emergence: Cycles of language use, language change, and language acquisition
references.venue: The modern language journal,
references.authors: Nicolas Fay T Mark Ellison
references.title: The cultural evolution of human communication systems in different sized populations: usability trumps learnability
references.venue: PloS one,
references.authors: Nicole Fitzgerald
references.title: To populate is to regulate
references.venue: arXiv preprint arXiv:1911.04362,
references.authors: Bruno Galantucci Simon Garrod
references.title: Experimental semiotics: a review
references.venue: Frontiers in human neuroscience,
references.authors: Simon Garrod Nicolas Fay John Lee Jon Oberlander Tracy Macleod
references.title: Foundations of representation: Where might graphical symbol systems come from
references.venue: Cognitive science, 31:961–87,
references.authors: Rick Dale Gary Lupyan
references.title: Language structure is partly determined by social structure
references.venue: PLoS ONE 5,
references.authors: Laura Graesser Kyunghyun Cho Douwe Kiela
references.title: Emergent linguistic phenomena in multi-agent communication games
references.venue: In Proc. of Empirical Methods in Natural Language Processing (EMNLP),
references.authors: Sepp Hochreiter Jürgen Schmidhuber
references.title: Long short-term memory
references.venue: Neural computation,
references.authors: Dieuwke Hupkes Verna Dankers Mathijs Mul Elia Bruni
references.title: Compositionality decomposed: how do neural networks generalise
references.venue: Journal of Artificial Intelligence Research,
references.authors: Judy Kegl
references.title: The nicaraguan sign language project: An overview
references.venue: Signpost,
references.authors: Eugene Kharitonov Rahma Chaabouni Diane Bouchacourt Marco Baroni
references.title: Egg: a toolkit for research on emergence of language in games
references.venue: In Proc. of Empirical Methods in Natural Language Processing (EMNLP),
references.authors: Eugene Kharitonov Rahma Chaabouni Diane Bouchacourt Marco Baroni
references.title: Entropy minimization in emergent languages
references.venue: In Proc. of International Conference on Machine Learning (ICML),
references.authors: Simon Kirby
references.title: Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity
references.venue: IEEE Transactions on Evolutionary Computation,
references.authors: Simon Kirby James R Hurford
references.title: The emergence of linguistic structure: An overview of the iterated learning model
references.venue: Simulating the evolution of language,
references.authors: Simon Kirby Hannah Cornish Kenny Smith
references.title: Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language
references.venue: In Proc. of the National Academy of Sciences of the United States of America (PNAS). National Academy of Sciences,
references.authors: Simon Kirby Monica Tamariz Hannah Cornish Kenny Smith
references.title: Compression and communication in the cultural evolution of linguistic structure
references.venue: Cognition, 141:87–102,
references.authors: Ryan Kiros Yukun Zhu Russ R Salakhutdinov Richard Zemel Raquel Urtasun Antonio Torralba Sanja Fidler
references.title: Skip-thought vectors
references.venue: In Proc. of Advances in Neural Information Processing Systems (NIPS),
references.authors: Stephen Kokoska Daniel Zwillinger
references.title: CRC standard probability and statistics tables and formulae
references.venue: Crc Press,
references.authors: Satwik Kottur José Moura Stefan Lee Dhruv Batra
references.title: Natural language does not emerge ‘naturally’ in multi-agent dialog
references.venue: In Proc. of Empirical Methods in Natural Language Processing (EMNLP),
references.authors: Angeliki Lazaridou Marco Baroni
references.title: Emergent multi-agent communication in the deep learning era
references.venue: arXiv preprint arXiv:2006.02419,
references.authors: Angeliki Lazaridou Karl Moritz Hermann Karl Tuyls Stephen Clark
references.title: Emergence of linguistic communication from referential games with symbolic and pixel input
references.venue: In Proc. of International Conference on Learning Representations (ICLR),
references.authors: Vladimir I Levenshtein
references.title: Binary codes capable of correcting deletions, insertions, and reversals
references.venue: In Soviet physics doklady,
references.authors: David K. Lewis
references.title: Convention: A Philosophical Study
references.venue: Wiley-Blackwell,
references.authors: Fushan Li Michael Bowling
references.title: Ease-of-teaching and language structure from emergent communication
references.venue: In Proc. of Advances in Neural Information Processing Systems (NeurIPS),
references.authors: Yuchen Lu Soumye Singhal Florian Strub Aaron Courville Olivier Pietquin
references.title: Countering language drift with seeded iterated learning
references.venue: In Proc. of International Conference on Machine Learning (ICML),
references.authors: Karen McComb Stuart Semple
references.title: Coevolution of vocal communication and sociality in primates
references.venue: Biology letters,
references.authors: Irit Meir Assaf Israel Wendy Sandler Carol A Padden Mark Aronoff
references.title: The influence of community on language structure: evidence from two young sign languages
references.venue: Linguistic Variation,
references.authors: Igor Mordatch Pieter Abbeel
references.title: Emergence of grounded compositional language in multi-agent populations
references.venue: In Proc. of Conference on Artificial Intelligence (AAAI),
references.authors: Daniel Nettle
references.title: Social scale and structural complexity in human languages
references.venue: Philosophical Transactions of the Royal Society B: Biological Sciences,
references.authors: Limor Raviv Antje Meyer Shiri Lev-Ari
references.title: Larger communities create more systematic languages
references.venue: Proc. of the Royal Society B: Biological Sciences,
references.authors: Limor Raviv Antje Meyer Shiri Lev-Ari
references.title: Compositional structure can emerge without generational transmission
references.venue: Cognition, 182:151–164,
references.authors: Limor Raviv Antje Meyer Shiri Lev-Ari
references.title: The role of social network structure in the emergence of linguistic structure
references.venue: Cognitive Science,
references.authors: Florencia Reali Nick Chater Morten H. Christiansen
references.title: Simpler grammar, larger vocabulary: How population size affects language
references.venue: Proc. of the Royal Society B: Biological Sciences,
references.authors: Yi Ren Shangmin Guo Matthieu Labeau Shay B. Cohen Simon Kirby
references.title: Compositional languages emerge in a neural iterated learning model
references.venue: In Proc. of International Conference on Learning Representations (ICLR),
references.authors: Cinjon Resnick Abhinav Gupta Jakob Foerster Andrew M. Dai Kyunghyun Cho
references.title: Capacity, bandwidth, and compositionality in emergent language learning
references.venue: In Proc. of Autonomous Agents and Multiagent Systems (AAMAS),
references.authors: Mathieu Rita Rahma Chaabouni Emmanuel Dupoux
references.title: LazImpa”: Lazy and impatient neural agents learn to communicate efficiently
references.venue: In Proc. of the Conference on Computational Natural Language Learning (CoNLL),
references.authors: Wendy Sandler Irit Meir Carol Padden Mark Aronoff
references.title: The emergence of grammar: Systematic structure in a new language
references.venue: Proceedings of the National Academy of Sciences,
references.authors: Kenny Smith Simon Kirby Henry Brighton
references.title: Iterated learning: A framework for the emergence of language
references.venue: Artificial life,
references.authors: Luc Steels
references.title: The synthetic modeling of language origins
references.venue: Evolution of communication,
references.authors: Richard S Sutton David A McAllester Satinder P Singh Yishay Mansour
references.title: Policy gradient methods for reinforcement learning with function approximation
references.venue: In Proc. of Advances in Neural Information Processing Systems (NIPS),
references.authors: Mónica Tamariz Simon Kirby
references.title: Culture: copying, compression, and conventionality
references.venue: Cognitive science,
references.authors: Olivier Tieleman Angeliki Lazaridou Shibl Mourad Charles Blundell Doina Precup
references.title: Shaping representations through communication: community size effect in artificial learning systems
references.venue: Visually Grounded Interaction and Language (ViGIL) Workshop,
references.authors: Pedregosa Paul van
references.title: Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
references.venue: Nature Methods,
references.authors: Elliott Wagner
references.title: Communication and structured correlation
references.venue: Erkenntnis, 71(3):377–393,
references.authors: Kyle Wagner James A Reggia Juan Uriagereka Gerald S Wilkinson
references.title: Progress in the simulation of emergent communication and language
references.venue: Adaptive Behavior,
references.authors: Alison Wray George W. Grace
references.title: The consequences of talking to strangers: Evolutionary corollaries of sociocultural influences on linguistic form
references.venue: Lingua, 117(3):543–578,
references.authors: Łukasz Kuciński P. Kołodziej Piotr Milos
references.title: Emergence of compositional language in communication through noisy channel. In Language in Reinforcement Learning (LaReL)
references.venue: ICML Workshop,
references.authors: Graesser
references.title: 2019) show that community of agents start coordinating their language when at least three agents are present in the community. They later assume that increasing the community size may impact
references.authors: Bouchacourt Baroni
references.title: 2019) while analyzing the influence of symmetric agents in the emergence of a common language. With different research objectives, Cogswell et al. (2019) exhibit a slight compositionality enhancement when increasing population size without the need of language transmission. Analogously, Tieleman et al. (2019) explicitly study community size and display a small but consistent gain of abstraction and structure within speakers
references.authors: Eventually Fitzgerald
references.title: 2019) suggest that populations improve regularization and generalization compared to single speaker-listener pairs but underlines that there is not a clear correlation between community size and learning properties
references.authors: Denamganaı Walker efficiency (Chaabouni et al Rita
references.title: 2020) or stability (Kharitonov et al., 2020
references.venue: (Kirby et al.,
references.authors: Graesser
references.title: 2019) show that community of agents start coordinating their language when at least three agents are present in the community. They later assume that increasing the community size may impact
references.authors: Raviv
references.title: 2019a) went one step further by arguing that the community size is predictive, i.e. a causal factor, of language structure and diversity
references.authors: Raviv
references.title: 2019a) went one step further by arguing that the community size is predictive, of language structure and diversity
sections.heading: 1 INTRODUCTION
sections.text: Language emergence has been explored in linguistics and artificial intelligence for two main reasons (Lazaridou & Baroni, 2020). On the one hand, artificially reproducing language emergence may help to understand the evolution of human languages (Steels, 1997; Briscoe, 2002; Wagner et al., 2003). On the other hand, language is known to be structured, and compositional (Bickerton, 2007), and imitating such properties would enhance machine learning representations. As a result, there exists a constant back and forth between cognitive sciences, linguistics, and artificial intelligence to retrieve the core ingredients of language emergence (Kirby et al., 2008). In this paper, we explore how the size of a population may impact the structure of emerging languages by using neural reinforcement learning methods.
  Especially, we explore the following socio-linguistic hypothesis: larger communities create more systematic languages (Raviv et al., 2019a; 2020). This hypothesis has been supported by a number of ethnographic (Gary Lupyan, 2010) and socio-linguistics (Raviv et al., 2020) observations as well as behavioral studies mimicking language emergence in a controlled setup (Raviv et al., 2019a). Few neural language emergence papers have explored how community size impacts language structure so far, but the available evidence is mitigated at best. Tieleman et al. (2019) observed a small but consistent regularization effect when pairing auto-encoders within a population. Similarly, Cogswell et al. (2019) observed slight improvements in language compositionality with a large population but only reported them in few experimental settings. Finally, Graesser et al. (2019) studied the impact of contact-agents for different population sizes, but they did not observe a correlation between the population size and the convergence speed, success rate, or mutual agent intelligibility.
  The following question arises: why does community size not improve language properties in recent emergent communication literature, although it is a key structuring factor in broader linguistics literature? We argue that recent emergent communication models are limited as they ignore individual learning capacities by working only with homogeneous populations. Consequently, they miss coupling effects emerging from agents’ asymmetries. As a result, we hypothesize that community size effects could occur as soon as local heterogeneities are introduced into populations.
  In this work, we explore the effects of population size with neural agents in the well-known Lewis referential game (Lewis, 1969). In this game, a speaker describes a hidden object to a listener, which must then reconstruct object properties. Both agents thus need to co-develop a communication protocol to solve the task. The population-based variant of this game randomly pairs one speaker and one listener from their respective communities. The goal is to observe whether increasing the number of agents enhances the communication protocol qualities, e.g. success rate, compositionality, generalization etc. (Kottur et al., 2017; Chaabouni et al., 2020; Lazaridou et al., 2018).
  Firstly, we reproduce Lewis reconstruction setting and confirm the experimental difference: when increasing the number of agents, we do not observe improvements over various emergent language metrics. We thus question the current paradigm to model population in the language emergence literature. In particular, all agents are trained uniformly, i.e., their learning speed, capacity, sampling are identical (Tieleman et al., 2019; Cogswell et al., 2019; Fitzgerald, 2019). Secondly, we evaluate the impact of two potential sources of model heterogeneities: (i) agents learning speed, (ii) agents capacities. In both cases, we observe a similar pattern: the absolute value of speakerlistener speed and capacity is not important, yet their relative value is crucial. We hence shed light on the strong correlation between language structures and agents relative training facilities. Thirdly, we push this reasoning further by distributing learning speeds across the population thus creating heterogeneous populations. We there observe an improvement of language scores combined with a variance reduction when increasing population sizes. In other words, larger communities of neural agents start developing more stable and structured languages when being heterogeneously designed. This observation brings a first stone toward solving the empirical and computational contradiction.
  Our experiments partially removed the apparent contradiction between the socio-linguistic observations and the recent emergent communication literature. They illustrate how crucial population training dynamics are in shaping emergent languages and how population heterogeneity may have been underestimated in the recent emergent communication literature. All in all, our contributions are three-fold: (i) we empirically show that the community size is not a structuring factor in language emergence by or in itself in the classic homogeneous Lewis setting; (ii) we give evidences that speaker-listener relative dynamics strongly affects language properties; (iii) we provide the first computational cues to remove the apparent difference between socio-linguistic literature and recent neural emergent communication works.
sections.heading: 2 RELATED WORK
sections.text: Population size in sociolinguistics. Population size is a core parameter defining the social environment an agent interacts with. Its impact on language structures has largely been studied on humans (Nettle, 2012; Bromham et al., 2015; Reali et al., 2018; Raviv et al., 2019a) and animals (Blumstein & Armitage, 1997; McComb & Semple, 2005; Wilkinson, 2013). By analyzing 2000 languages (Dryer & Haspelmath, 2013), a clear correlation was drawn between population size and diverse language features, e.g. larger communities tend to develop simpler grammars (Gary Lupyan, 2010; Meir et al., 2012; Reali et al., 2018). As part of their research on the influence of network structures on language emergence (Raviv et al., 2019a;b; 2020), Raviv et al. (2019a) went one step further by arguing that the community size is predictive of language structure and diversity. To do so, they split 150 people into different groups of given community size to isolate confounding factors. While people played a speaker-listener Lewis game, they observe that the greater the community size, the simpler and more consistent the generated language. Here, we intend to test this assumption in the context of neural language emergence. All things considered, we adopt a setting close to Raviv et al. (2019a)’s when computationally modeling human population.
  Populations in experimental emergent communication. Experimental language emergence has mainly been studied with two methods: behavioral studies (Kegl, 1994; Sandler et al., 2005) and simulations (neural and non neural) (Wagner et al., 2003; Lazaridou & Baroni, 2020). From behavioral studies and non neural simulations, two main approaches have emerged in the past 20 years: experimental semiotics (Galantucci & Garrod, 2011; Garrod et al., 2007) and iterated learning (Kirby & Hurford, 2002; Kirby et al., 2008; Beckner et al., 2017). According to experimental semiotics studies, languages are mainly subject to an expressivity pressure; they argue that messages should be highly informative to allow communication within a group (Fay & Ellison, 2013). According to Iterated Learning paradigm, structures emerge from a compressibility pressure; they argue that memory limitations compel messages to become simpler to be easily learned (Tamariz &
  Kirby, 2015), which is also referred to as transmission bottleneck (Smith et al., 2003). Kirby et al. (2015) then combined those two approaches and show that languages emerge as a trade-off between expressivity and compressivity during cultural evolution. Recently, similar ideas have been modeled in emergent communication frameworks involving neural agents (Ren et al., 2020; Lu et al., 2020). Because seminal non-neural simulations used specific optimization methods and models across studies (Wagner et al., 2003), it is hard to generalize a global picture. Indeed, Wagner et al. (2003) keep listing standalone experiments. Modern neural agents manage to simplify and standardize agents’ modeling. It thus enables explore the determinants of different structural properties of human languages within a unique framework, paving the way for holistic models of emergence communication (Lazaridou & Baroni, 2020). Our paper is related to this last set of neural works.
  Recent works in emergent communication have been debating the prerequisites to the emergence of language universals such as compositionality (Li & Bowling, 2019; Ren et al., 2020; Lazaridou et al., 2018; Mordatch & Abbeel, 2018; Resnick et al., 2020; Kottur et al., 2017; Choi et al., 2018; Łukasz Kuciński et al., 2020), generalisation (Baroni, 2020; Chaabouni et al., 2020; Hupkes et al., 2020; Denamganaı̈ & Walker, 2020), efficiency (Chaabouni et al., 2019; Rita et al., 2020) or stability (Kharitonov et al., 2020; Lu et al., 2020). Among them, a few works explored how different population structures may impact properties of emergent languages. Inspired by iterated learning methods (Kirby et al., 2008), Ren et al. (2020); Li & Bowling (2019); Lu et al. (2020) look at language evolution across multiple generations of agent pairs, i.e. population is spread over time. However, we consider here a population where multiple agent-pairs coexist simultaneously within a single generation. There, Graesser et al. (2019) show that community of agents start coordinating their language when at least three agents are present in the community. They later assume that increasing the community size may impact the emergent shared language, but did not observe it in their initial experiments. A similar hypothesis was also made by Bouchacourt & Baroni (2019) while analyzing the influence of symmetric agents in the emergence of a common language. With different research objectives, Cogswell et al. (2019) exhibit a slight compositionality enhancement when increasing population size without the need of language transmission. Analogously, Tieleman et al. (2019) explicitly study community size and display a small but consistent gain of abstraction and structure within speakers’ latent representations by increasing population size. Eventually, Fitzgerald (2019) suggest that populations improve regularization and generalization compared to single speaker-listener pairs but underlines that there is not a clear correlation between community size and learning properties. In this paper, we aim at analyzing how population size affects those discussed properties. We align with emergent communication literature and show that naively increasing community size does not consistently improve language properties. We then challenge the homogeneity assumption made in all population designs.
sections.heading: 3 METHOD
sections.text: We here describe the different components of language emergence in a population-based Lewis Game, namely, game rules, notations, training dynamics, and evaluation metrics. Finally, we define how we alter population dynamics by asymmetrizing agents and injecting heterogeneities.
sections.heading: 3.1 RECONSTRUCTION GAME.
sections.text: Game Rules: We study emergent communication in the context of the Lewis referential games (Lewis, 1969). There, a speaker observes all the attributes of an object. The speaker then outputs a descriptive message, which a second agent, the listener, receives. The listener must accurately reconstruct each value of each attribute to solve the task. Both agents are finally rewarded in light of the reconstruction accuracy. Note that another variant of this game requires the listener to retrieve the correct object within a list of distractors, but both settings are inherently similar.
  Game Formalism: The observed object v ∈ VK is characterized by |K| attributes where each attribute may take |V| values. We encode the observed object by a concatenation of one-hot representations of the attributes vk ∈ V of the object v for each attribute k ∈ K. For each new run, the set of objects is split into a training set X and test set. The intermediate message m ∈ WT is a sequence of T tokens, m = (mt)T−1t=0 where each token is taken from a vocabularyW of dimension |W|, finishing by a hard-coded end-of-sentence token EoS. The speaker and listener are two neural agents respectively parametrized by θ and φ. The speaker follows a recurrent policy πθ: given an input object v, it samples for all t a tokenmt with probability πθ(mt|m<t, v). We denote πθ(m|v) the
  probability distribution of the entire message given an input object v. The listener outputs for each k attributes a probability distributions over the values V: πkφ(vk|m). At training time, the speaker message is generated by sampling the policy m ∼ πθ(·|v). At test time we use the greedy message m̂t = arg maxm̄ πθ(m̄|m̂<t, v). Game Objective: As in (Chaabouni et al., 2020), we define the listener training goal to be the average of the multi-classification log-likelihood loss per attribute:
  Lφ = − 1 |X |.|K| ∑ v∈X ∑ m∈WT ∑ k∈|K| πθ(m|v) · log ( πkφ(vk|m) ) . (1)
  In our setting, we want the speaker to optimize the same objective. To do so, we define the speaker game reward as the negative loss of the listener,
  rt(v,m<t) =
  { 1 |K| ∑ k∈|K| log ( πkφ(vk|m) ) if t = T,
  0 otherwise. (2)
  Following the gradient policy theorem (Sutton et al., 2000), we maximize the speaker reward by minimizing the following objective over θ:
  Lθ = − 1 |X | ∑ v∈X ∑ t∈T log πkθ (mt|x,m<t) · rt(v,m<t), (3)
  where m is sampled according to the speaker policy πθ.
sections.heading: 3.2 POPULATION-BASED RECONSTRUCTION GAME
sections.text: We first create a population of N speakers and N listeners, so we have a total number of agents of 2N . Following (Tieleman et al., 2019), at each step, we uniformly sample one speaker and one listener and pair them together. We then proceed as in the classic one pair Lewis game: both agents play the game with a batch of inputs and receive an optimization step minimizing (Equation 3 & 1). This operation is repeated until convergence, i.e., all speaker-listener pairs have stable test losses. While standard, we note that this training procedure relies on strong latent assumptions: (i) each speaker (resp. listener) is uniformly sampled, i.e., there is no preponderant agent within the population, (ii) the communication graph is fully connected and uniform, i.e., all speakers may be paired with all listeners with the same probability, (iii) agents cannot be differentiated, i.e., agents have no information about the identity of their partners, (iv) speakers and listeners are all similar, i.e., there is no difference in the agent definitions nor in the optimization process. Overall, those hypotheses create a homogeneous training setting. In practice, the agents only differ by their initialization and optimization updates, e.g., stochastic agent pairing, game generations, and message sampling.
sections.heading: 3.3 ASSESSING EMERGENT LANGUAGE IN LEWIS GAMES.
sections.text: We here introduce various metrics to assess emergent languages structure and quality. To do so, we first need to introduce two distances in the input/object space and in the message space. We define the distance between two objects v and v′ as the proportion of common attributes computed by Dobj(v, v′) = 1|K| ∑ k 1{vk = v′k}. For the distance between two messages m and m′ we use the edit-distance (Levenshtein et al., 1966) and note it Dmes(m,m′).
  Speakers synchronization: Within populations, we measure how close speakers’ languages are by introducing a distance between the two languages. Given two set of speaker weights θ1 and θ2 and their respective language Lθ1 and Lθ2 ,
  DL(Lθ1 ,Lθ2) := Ev∈X ,m1∼πθ1 (.|v),m2∼πθ2 (.|v)[Dmes(m1,m2)]. (4)
  It computes the average distance between two speakers’ messages over all the dataset. When considering an entire population of N speakers, we can then compute the synchronization score rsync by averaging the distance between all pairs of speakers: rsync = 2N(N−1) ∑ i 6=j DL(Lθi ,Lθj ).
  Entropy: To measure language coherence, we study the entropy of the speaker languageHθ:
  Hθ := Ev,m∼πθ(·|v) [ − log(πθ(m|v)) ] = Ev∈X [ L−1∑ t=0 h(πθ(mt|m<t, v)) ] , (5)
  where h : w → −w log(w). In our setting, we seek to minimize entropy as it reduces language complexity and denotes language stability (Kharitonov et al., 2020). For instance, when Hθ is minimal, the speaker uses a unique message to refer to an object. When Hθ is high, the speaker generates synonyms to refer to the same object. While some entropy is beneficial with noisy communication channels, they do not improve robustness in our case; it only makes the listener’s task harder. Finally, to ease reading, we display the negative entropy (Neg-Entropy), so all metrics increases correspond to language improvement.
  Topographic Similarity: We use topographic similarity as a quantitative measure of compositionality (Brighton & Kirby, 2006; Lazaridou et al., 2018). Topographic similarity captures how well similarities between objects are transcribed by similarities in the message space by computing the Spearman correlation (Kokoska & Zwillinger, 2000; Virtanen et al., 2020) between pairwise object distance (Dobj(v1, v2)) and the corresponding message distances (Dmes(mv1 ,mv2)).
  Generalization: Test accuracy quantifies how well agents generalize to unseen objects. It measures the average reconstruction success on the test set where we define a success when the greedy prediction perfectly matches the input object. The greedy prediction v̂ is the one which maximizes the likelihood computed by the listener given the greedy message m̂: v̂ = arg maxv ∏ k π k φ(vk|m̂) with m̂t = arg maxm̄ πθ(m̄|m̂<t, v) the greedy message.
sections.heading: 3.4 ALTERING POPULATION DYNAMICS
sections.text: We present how diversity can be introduced within a population either by asymmetrizing speakerlistener in minimal size population (N = 2), or distributing heterogeneity within larger populations.
  Diversity factor: We aim to vary specific agent features toward creating diversity in the population and target one core components: the training speed. We control agents’ training speed by changing the number of gradient updates. Formally, we introduce the probability p of an agent to be optimized after each iteration of the game. Therefore, the lower p, the slower agent training. This approach has two positive features: (i) it neither alters the communication graph nor the agent sampling, preserving other homogeneity hypotheses in the population, (ii) it is more stable than modifying the learning rate as it avoids large destructive updates.
  Control parameters of local asymmetry: We provide a control parameter to characterize diversity at the scale of a minimal size population (N = 2). Noticeably, N = 2 is the closer setting to a single-pair of agents where we can compute speakers synchronization. To analyze the effect of speaker-listener training speed asymmetry, we introduce the relative training speed ρspeed := pS/pL where pS (resp. pL) is speaker’s learning speed (resp. listener’s learning speed).
  Distributing Population Heterogeneity: To create heterogeneous populations, we characterize every single speaker and listener with individual properties. We then sampled them once at the beginning of the training. Formally, when altering training speed, we sample pi ∼ Log-N (ηp, σp) for every i agent in the population, where Log-N is a log normal distribution.
sections.heading: 4 EXPERIMENTAL SETTING
sections.text: Game Properties: In the main paper, we use |K| = 4 attributes with |V| = 4 values. We also report scores with |K| = 2 attributes and |V | = 10 values in the Appendix F.1 to illustrate that our observation still holds in a different setting. Finally, we use a vocabulary size of |W| = 20 and a message length T = 10. For each run, we generate objects in VK uniformly at random, and we split them into a train and a test sets, which respectively contains 80% and 20% of the objects.
  Neural architectures: The speaker first encodes the input object v in a one hot encoding and then processes it with a linear layer. It uses it to initialize a single layer LSTM (Hochreiter & Schmidhuber, 1997) with layernorm (Kiros et al., 2015) and a (default) hidden size of 128. Finally, the LSTM output is fed to a linear layer of dimension |W| followed by a softmax activation. The listener is composed of a look-up table of dimension 128 followed by a LSTM with layernorm and a
  (default) hidden size of 128. Then, for each attribute k ∈ {1, ...,K}, we apply a linear projection of dimension |V| followed by a softmax activation to the last LSTM output. Optimization: For both agents, we use a Adam optimizer with a learning rate of 5e10−3, β1 = 0.9 and β2 = 0.999 and a training batch size of 1024 when optimizing for their respective loss. For the speaker, we set the entropy coefficient of 0.02. Finally, we re-normalize the reward by using r̄(x,w<t) =
  r(x,w<t)−ηrt σrt to reduce the gradient variance (Sutton et al., 2000), where ηrt and σ r t are
  respectively the average and the standard deviation of the reward within the batch at each time step. In practice, they are respectively equal to zero and one for all the timesteps but the last.
  We use the EGG toolkit (Kharitonov et al., 2019) as a starting framework. The code is available at HIDDEN. All experiments are run over six seeds. In all figures, bars show one standard deviation.
sections.heading: 5 RESULTS AND DISCUSSION
sections.text: This section first reproduces the impact of the population size on language properties in the homogeneous Lewis setting. We then study speaker-listener asymmetry by altering networks training speed, and highlight the importance of relative training speed in shaping language. We finally use relative training speed as a non-confounding heterogeneity factor when varying the community-size, and thus tackle the initial contradiction.
sections.heading: 5.1 COMMUNITY SIZE IS NOT ALONE A LANGUAGE STRUCTURING FACTOR
sections.text: Language properties are not enhanced by community size. In Figure 1, we observe that increasing population size does not improve language properties: speaker synchronization and compositionality remain almost constant, and entropy and generalization are even deteriorating. As shown in Appendix E, Spearman correlations for those metrics are respectively -0.44, -0.52, -0.15, -0.14 (with p-values< 0.05) meaning that there is no positive correlation between those metrics and population size. In addition, we do not observe any gain of language stability by increasing population size . By stable languages, we mean languages whose properties have low variation across seeds for a given population size. Here, increasing community size does not reduce the standard deviation of the language metrics across seeds. Finally, we note that the speaker synchronization is high despite using large communication channel, as also observed by (Graesser et al., 2019). Overall, we confirm that community size does not improve the language properties of neural agents.
  The potential pitfall of homogeneity. We hypothesize that the absence of positive correlation between population size and language properties may be explained by the simplicity of population modeling. The homogeneity assumption seems restrictive as it implies two main limitations: (i) although agents are not identically initialized, they have the same training dynamics in expectation, (ii) all agents have the same optimization process, whereas real communities are inherently diverse. Consequently, we formulate the subsequent hypothesis: Community size may be a structuring factor as soon as heterogeneity is introduced in populations design.
sections.heading: 5.2 LANGUAGE PROPERTIES ARE CONTROLLED BY SPEAKER-LISTENER ASYMMETRIES
sections.text: Before generating heterogeneous populations, we explore the impact of learning speed heterogeneities at the scale of a minimal size population (N = 2) in order not to add confounding factors and better intuit population dynamics. As described in Section 3.4, we here modify the learning speed for a minimal size population. Note that we compute this ratio through multiple values of pS/pL, where p ∈ {0.01, 0.02, 0.04, 0.1, 0.2, 0.5, 1.}.
  Relative differences between speaker and listener significantly affect language properties. In Figure 2, we display the evolution of language properties with ρspeed. We observe that asymmetrizing speaker-listener learning speed significantly influences language properties. When the speakers have a large update probability ps, we note a global improvement of language properties: (i) speakers synchronization rate is very high, (ii) language neg-entropy is reaching almost 0, (iii) compositionality is significantly improved and (iv) test accuracy is close its optimal value. On the contrary, when listeners have a large update probability compared to the speakers, we note the opposite trend. In Appendix C, we show that the trends are translated along the x-axis when varying setup parameters. Therefore, the ratio at which metrics start becoming optimal are dependent of the setup.
  As a sanity check, we then verify that altering p does not fundamentally change the language properties per itself. We thus sweep over the gradient update probability p while setting a minimal population size of N = 2, and compute the Spearman correlation in Table 1. We observe that hyperparameter p has no statistically significant correlations with most of the language scores, i.e. their Spearman coefficients are below 0.4 with p-values above 0.05. Therefore, the impact of this correlation is small enough in our experiments to be neglected. In sum, it is not the magnitude of the parameters that is critical to shape language, but the relative ratio between them.
  Understanding learning speed asymmetries. We here try to provide some pieces of intuitions to understand ρspeed by looking at the extreme cases. In the limit ρspeed 1, i.e. ps pl , the listener is almost optimal wrt. the speaker at each speaker update. It implies that listener’s loss is optimally minimized for each message refering to a single input. Then, as soon as speakers develop a language where each message refers to a single input, the game is over as speaker’s rewards are immediately maximal. It means that many languages can emerge from this limit case, including almost degenerated languages.
  In the limit ρspeed 1, i.e. ps pl, the speaker is almost optimal wrt. the listener at each listener update. The speaker thus targets the messages providing the highest rewards as in a stationary RL task. We can safely assume that the set of messages providing the highest rewards for each input is small. It then explains why the speaker converges to a low entropy language. In addition, when N = 2, both speakers solve the same communication task on same quasi-stationary environment and obtain similar rewards. They are then more likely to align on a similar language as they observe the . This use-case also suggests that when speakers are “fast enough”, a common interlocutor is the only requirement for the emergence of a common language. Finally, we push further the assumption by interpreting the gain of compositionality through “ease-of-teaching” (Li & Bowling, 2019). The authors show that compositional codes are easier to teach for listeners. We then hypothesize here that ’faster’ speakers develop codes easier to learn for ’slow’ listeners and thus higher quality languages.
  In Appendix D.1, we show that trends noticed in Figure 2 can be reproduced varying the ratio between speaker’s and listener’s capacities. It suggests that other parameters influencing the relative speaker-listener co-adaptation affect language properties. We provide first hints on a potential indirect influence of this capacity ratio on agents’ relative training speed. More generally, we suspect that many parameters may be latently related to training speed, e.g. community-graph (Graesser et al., 2019), newborn agents (Ren et al., 2020; Cogswell et al., 2019), or broadcasting by using different speaker-listener ratios as illustrated in Appendix D.2, but we leave it for future analysis.
sections.heading: 5.3 COMMUNITY SIZE STRUCTURES LANGUAGES OF HETEROGENEOUS POPULATIONS
sections.text: Previous sections have identified that learning speed is a powerful controlling factor and could alter language structure when asymmetrizing speaker-listener pairs. However, these experiments remain limited for (i) we have not yet observed larger population-size tend to generate more stable and structured language, (ii) this asymmetry remains artificial and specific to our Lewis game setting. Therefore, we here leverage this training speed factor within a population, characterizing individual agents with different update probabilities. We hence sample agent update probability with p ∼ Log-N (ηp, σp) with ηp = −1 and σp = 1 if not specified otherwise. Larger heterogeneous population leads to higher quality language. In Figure 3, we observe that emergent language metrics are now positively correlated with population size. When increasing the population size from 2 to 20 speaker synchronization and neg-entropy go from low to almost optimal values, compositionality increases by 22% in average and generalization remains stable. As detailed in Appendix E, speaker synchronization, neg-entropy and compositionality have a significant correlation with Spearman coefficients equal to 0.55, 0.24, and 0.21 (p-values< 0.05). Those are the first hints toward our initial goal: when individuals are heterogeneous, a large population size correlates with higher quality and stable language properties. In Appendix F.2, we report the results with distribution changes. When the distribution is more concentrated, trends are less significant and we note that metrics do not have the same sensitivity to heterogeneities: a minimal dispersion of the distribution is required to observe significant effects. When changing the distribution without changing variance (β distribution instead of log-normal distribution), trends are not notably affected.
  The overall gain over language metrics can be interpreted in light of Section 5.2. Language score evolution depends on the relative speaker-listener learning speeds. In heterogeneous populations, the probability of sampling extreme learning speed p and thus enforcing speaker-listener asymmetries increases with N . One explanation would be that extreme agents, especially slow listeners, behave as kinetic bottlenecks, forcing the speakers to structure their languages. Thus, language emergent properties would be particularly determined by fast speakers and/or slow listeners.
  Larger heterogeneous population leads to more stable language. Similarly, the population size correlates with the variance of the language scores across seeds. Hence, when increasing the population size from 2 to 20; the standard deviations of speakers synchronization, neg-entropy, compositionality are divided by 10, 15 and 2; yet generalization is less affected. One hypothesis for this reduction is that smaller communities are more likely to be disparate because of the sampling. For instance, if we characterize a population with the empirical average of the training speeds η̂p, it has high variance with small N . On the other side, when Ngrows, η̂p gets closer to the average learning speed, limN→∞ η̂p = ηp, and its variance gets smaller. In other words, larger heterogeneous communities have more stable languages because of Law of large numbers in our models.
  Increasing diversity enforces population size benefits. In Figure 4, we challenge further the impact of heterogeneity by varying the standard deviation σp of the learning speed distribution. For each value of σp, we run the game with populations of size N = 2 and N = 10 and compare the relative evolution of the scores between the two population sizes. We note that the higher σp, the larger the gain of synchronization, neg-entropy, generalization when increasingN . The relative gain of neg-entropy is almost equal to 100% when σp > 1 while speaker synchronization gain is close to 200% for large values of σp. For generalization, the 10% relative gain is noteworthy as it corresponds to reaching close to 100% test accuracy. The more discrepancy exists within the population
  (large σp), the more beneficial is to have large populations. Furthermore, speaker synchronization, neg-entropy and generalization standard deviations reduce up to almost 80% when having σp > 2. In sum, this result corroborates that larger populations lead to more stable language.
  On the overall performances of heterogeneous populations. On Figure 3, we see that language properties of homogeneous populations with ρspeed = 1 decrease when population size is increasing. On the contrary, metrics get higher for heterogeneous populations (generalization stays almost constant). Thus, when modeling large populations, there is a gain distributing heterogeneities within populations. However, metrics remain below the best pair observed for N = 2 when varying ρspeed. It means that enlarging populations does not lead to an absolute better language than the best one we can obtain with a minimal size population. Though, when population size increases, heterogeneous populations metrics get closer to the best pair scores. Therefore, population size tends to synchronize on a language whose properties are close to language of the best single pair.
sections.heading: 6 CONCLUSION
sections.text: One objective of language emergence simulations is to identify simple (non-biological) rules that characterize language evolution (Steels, 1997; Briscoe, 2002; Wagner et al., 2003; Kirby, 2001). However, those models require a delicate balance between simplicity to ease analysis and complexity to remain realistic. In this paper, we argue that the current population-based models may actually be too simplistic as they assume homogeneous communities. We show that this simplification is a potential root cause of the experimental difference between neural observations and socio-linguistic literature. Namely, larger populations should lead to more stable and structured languages, which is not observed in recent emergent models. Yet, as soon as we add diversity within populations, this contradiction partially vanishes. We advocate for better integrating population diversity and dynamics in emergent literature if we want computational models to be useful. In this journey, we also observe that the relative training speed of the agents may be an underestimated factor in shaping languages. Worse, other agent properties may be confounded with it, such as network capacity.
  This work also opens many questions. First, our observations were performed in the Lewis game so far, and it would be interesting to observe the impact of diversity in more complex settings. Second, while heterogeneity is a structuring factor in population, the core factors are yet to be investigated, e.g., how to correctly model diversity, are there some causal components? Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial experimental difference such as varying the communication network topology (Wagner, 2009), the proportion of contact agents (Wray & Grace, 2007; Clyne, 1992; Graesser et al., 2019) or the proportion of second learners (Ellis, 2008). Fourth, although heterogeneous populations recover a socio-linguistics result, the average scores remain below the best emergent protocol. We leave for future work how heterogeneous populations may be leveraged to structure further the language (e.g. more complex tasks, larger population). Finally, training speed is a natural controlling parameter for computational models, but it is unclear how it may relate to human behavior. Overall, we hope that this paper provides new insights toward modeling populations in language emergence, and it is part of this constant back and forth between cognitive sciences, linguistics, and artificial intelligence.
sections.heading: A RANDOM BASELINES
sections.text: We report the values of Speakers synchronization, neg-entropy, topographic similarity and test accuracy for randomly-initialized and untrainded agents:
  Table 2: Average Speakers synchronization, neg-entropy, topographic similarity and test accuracy for randomly-initialized and untrainded agents. We report the standard deviation across 10 seeds
  Experiment Sp. sync Neg-Entropy Compo. Gene.
  Random agents 0.09 ±0.03 -28.4 ±0.23 0.04 ±0.02 0.24 ±0.02
sections.heading: B NON-CONFOUDING PARAMETERS
sections.text: 10 1 100 p
  0.4
  0.5
  0.6
  0.7
  0.8
  0.9
  1.0 Speakers synchronization
  10 1 100 p
  1.0
  0.8
  0.6
  0.4
  0.2
  0.0 Neg-entropy
  10 1 100 p
  0.4
  0.5
  0.6
  0.7
  0.8 Topographic similarity
  10 1 100 p
  0.75
  0.80
  0.85
  0.90
  0.95
  1.00 Test accuracy
  Figure 5: Variations of language metrics with increasing values of gradient update probability p.
  In Figure 5, we show the variations of language scores with increasing values of p and h, while being identical for speaker and listener. In Table 1, we shows the Spearman correlation between the metrics and the control parameters. We do not observe significant correlation. In addition, both the relative and absolute variations of the parameters are negligible compared to experiments displayed in Figure 2.
  C VARYING SETUP PARAMETERS WHEN STUDYING SPEAKER-LISTENER ASYMMETRIES
  In this Section, we show how trends displayed in Figure evolve when performing some change of parameters in the setup. As shown in Figure 6, when modifying agents’ hidden sizes (speaker’s hidden size is equal to 64 and listener’s hidden size is equal to 512 while they are both equal to 128 in Figure 2), the trends are the same. However, we notice a translation of the curves. It means that the ratio of learning speed at which metrics start becoming optimal is controlled by the parameters of the setup. Figure 6 highlights that the trends are sensitive to agents’ capacity.
sections.heading: D OTHER SPEAKER-LISTENER ASYMMETRIES INFLUENCING EMERGENT LANGUAGE PROPERTIES
sections.text: In complement of Section 5.2, we show in this Appendix that trends observed in Figure 2 can be reproduced with other speaker-listener asymmetries. In particular, we study the ratio between speaker’s capacity and listener’s capacity in a minimal size population (N = 2) in Appendix D.1 and the ratio between the number of speakers and listeners in Appendix D.2.
sections.heading: D.1 AGENTS’ CAPACITY
sections.text: We here reproduce the trends observed in Figure 2 with another speaker-listener asymmetry. Formally, we introduce ρcapacity := hS/hL where hS (resp. hL) is speaker’s hidden size (resp. listener’s hidden size). For the following experiments, we compute this ratio though multiple values of hS/hL for h ∈ {32, 64, 128, 256, 512, 1024}.
  The absolute capacity of agents has no significant impact on language properties. As done in Section 5.2, we first perform a sanity check and verify that altering hS and hL while keeping hS = hL =: h does not fundamentally change the language properties per itself. We sweep over h while setting a minimal population size ofN = 2, and compute the Spearman correlation in Table 3. We observe that h have no statistically significant correlations with most speaker synchronization, compositionality generalization (Spearman correlation inferior to 0.3 or p-values above 0.05). The only noteworthy correlation is between h and the Neg-entropy with a Spearman correlation of 0.89 and a p-value< 0.05. However, as shown in Figure 7 the variations are two orders of magnitude lower than the neg-entropy variations in Figure 8. Therefore, the impact of this correlation is small enough in our experiments to be neglected.
  The relative speaker-listener capacity significantly affects language properties In Figure 7, we see that when varying ρcapacity , we get trends similar to those observed in Figure 2. In Table 5, we compute the Spearman correlation between the metrics and ρcapacity and notice that there is a significant correlation between ρcapacity and studied metrics.
  Network capacity may be confounded with training speed. While the effect of training speed may be partially intuited in Section 5.2, the impact of capacity is more tedious to analyze. As we have already observed that training speed is a crucial parameter, we thus test whether a correlation
  between network capacity and learning speed can be established. In Figure 9 (resp. Figure 10), we pair a fixed pretrained listener (resp. speaker) with newly initialized speakers (resp. listeners) of increasing capacities and see how fast those new speakers (resp. listeners) reach convergence. We observe that all trained agents converge to the same accuracy. However, we also note that they have different convergence speeds. Speakers (resp. listeners) of capacity 512, 128, 32 respectively have to see 100, 250, and 900 batches (resp. 50, 100 and 250 batches) to reach train accuracy of 95%. In short: the larger the network, the faster the training speed. We conclude that network capacity may be a confounding factor of training speed in our setting and that the high degree of similarity in Figure 2 may be partially explained by the indirect impact of capacity on learning speed.
sections.heading: D.2 ALTERING THE SPEAKERS AND LISTENERS RATIO
sections.text: In complement to ρspeed in Section 5.2 and ρcapacity in Appendix D.2, we extends results of to another population factor and intuit how it relates to learning speed. We introduce the ratio between the number of speakers and the number of listeners within a population ρagents := NlistenerNspeaker . Note that this ratio cannot be applied to a single speaker-listener pair. Yet, it introduces an asymme-
  try between the community of speakers and the community of listeners. We show the evolution of language metrics in Figure 11. There, we observe than unbalancing Ns and Nl has a strong impact on language scores. Speaker synchronization reach upper and lower bound depending on the ratio, Neg-entropy either is very low or close to 0, and compositionality and generalization increase. ρagents is thus an additional control factor of language properties. Extreme cases coincide with the training speed interpretation of Section 5.2. Indeed, at each iteration of the game, each speaker (resp. listener) has a probability 1/Nspeakers (resp. 1/Nlisteners) to be sampled. Thus, speakers have Nlisteners/Nspeakers more of less learning steps than listeners. Consequently, when ρagents 1, individual speakers (resp. listeners) have way more optimization steps than individual listeners (resp. speakers) and we fairly hypothesize that speakers (resp. listeners) train faster than listeners (resp. speakers). This ratio may therefore be a confounding factor of a training speed mechanism. Note that, interestingly, ρagents can also be related to different sampling procedures, or broadcasting phenomenon.
sections.heading: E ADDITIONAL CORRELATIONS
sections.heading: F COMPLEMENTARY SETTING
sections.heading: F.1 DIFFERENT INPUT SPACE
sections.text: In this section, we reproduce our core experiments in a different Lewis game setting, with |K| = 2 attributes containing each |V | = 10 values.
sections.heading: F.2 DIFFERENT HETEROGENEITY DISTRIBUTION
sections.text: In this section, we vary the distribution of agent update probability p to evaluate how the observed trends are sensitive to the heterogeneity distribution.
  Log-normal distribution with lower standard deviation We first test how language properties are affected by a variance reduction. In Figure 14 agent update probability p are sampled according to a log-normal distribution Log-N (ηp, σp) with ηp = −1 and σp = 0.5. Compared to Figure 3, standard deviation has been divided by 2 meaning a more concentrated heterogeneity distribution. We see that speakers synchronization and neg-entropy increases are slower and that compositionality increase is now very low. Generalization remains almost unchanged. This plot suggests that the distribution of heterogeneities must be minimally wide to observe a positive correlation with population size and that all metrics do not have the same sensitivity to heterogeneities.
  Distribution change We now change test language properties are affected by a distribution change. In Figure 15, agent update probability p are sampled according to a Beta distribution β(1, 2). Variances of β(1, 2) and Log-N (−1, 0.5) (Figure 14) have the same order of magnitude. As we can see, trends are similar for Figure 14 and Figure 15. Trends are thus not notably affected by changing the log-normal distribution by a β distribution of similar variance.
sections.heading: G CHANGES
sections.text: Here is the list of changes made after reviewers’ comments. When a change has been made, we show how the paper was before/after the change.
sections.heading: G.1 CONCERNING GENERAL COMMENTS
sections.heading: G.1.1 CHANGE 1.1 (REFERRING TO SECTION 2)
sections.heading: Before:
sections.text: Populations in simulated emergent communication. Recent works in emergent communication have been debating the prerequisites to the emergence of language universals such as compositionality (Li & Bowling, 2019; Ren et al., 2020; Lazaridou et al., 2018; Mordatch & Abbeel, 2018; Resnick et al., 2020; Kottur et al., 2017; Choi et al., 2018; Łukasz Kuciński et al., 2020), generalisation (Baroni, 2020; Chaabouni et al., 2020; Hupkes et al., 2020; Denamganaı̈ & Walker, 2020), efficiency (Chaabouni et al., 2019; Rita et al., 2020) or stability (Kharitonov et al., 2020; Lu et al., 2020). Among them, a few works explored how different population structures may impact properties of emergent languages. Inspired by iterated learning methods (Kirby et al., 2008), Ren et al. (2020); Li & Bowling (2019); Lu et al. (2020) look at language evolution across multiple generations of agent pairs, i.e. population is spread over time. However, we consider here a population where multiple agent-pairs coexist simultaneously within a single generation. There, Graesser et al. (2019) show that community of agents start coordinating their language when at least three agents are present in the community. They later assume that increasing the community size may impact the emergent shared language, but did not observe it in their initial experiments. A similar hypothesis was also made by Bouchacourt & Baroni (2019) while analyzing the influence of symmetric agents in the emergence of a common language. With different research objectives, Cogswell et al. (2019) exhibit a slight compositionality enhancement when increasing population size without the need of language transmission. Analogously, Tieleman et al. (2019) explicitly study community size and display a small but consistent gain of abstraction and structure within speakers’ latent representations by increasing population size. Eventually, Fitzgerald (2019) suggest that populations improve regularization and generalization compared to single speaker-listener pairs but underlines that there is not a clear correlation between community size and learning properties. In this paper, we aim at analyzing how population size affects those discussed properties. We align with emergent communication literature and show that naively increasing community size does not consistently improve language properties. We then challenge the homogeneity assumption made in all population designs.
sections.heading: After:
sections.text: Populations in experimental emergent communication. Experimental language emergence has mainly been studied with two methods: behavioral studies (Kegl, 1994; Sandler et al., 2005) and simulations (neural and non neural) (Wagner et al., 2003; Lazaridou & Baroni, 2020). From behavioral studies and non neural simulations, two main approaches have emerged in the past 20 years: experimental semiotics (Galantucci & Garrod, 2011; Garrod et al., 2007) and iterated learning (Kirby & Hurford, 2002; Kirby et al., 2008; Beckner et al., 2017). According to experimental semiotics studies, languages are mainly subject to an expressivity pressure; they argue that messages should be highly informative to allow communication within a group (Fay & Ellison, 2013). According to Iterated Learning paradigm, structures emerge from a compressibility pressure; they argue that memory limitations compel messages to become simpler to be easily learned (Tamariz & Kirby, 2015), which is also referred to as transmission bottleneck (Smith et al., 2003). Kirby et al. (2015) then combined those two approaches and show that languages emerge as a trade-off between expressivity and compressivity during cultural evolution. Recently, similar ideas have been modeled in emergent communication frameworks involving neural agents (Ren et al., 2020; Lu et al., 2020). Because seminal non-neural simulations used specific optimization methods and models
  across studies (Wagner et al., 2003), it is hard to generalize a global picture. Indeed, Wagner et al. (2003) keep listing standalone experiments. Modern neural agents manage to simplify and standardize agents’ modelling. It thus enables explore the determinants of different structural properties of human languages within a unique framework, paving the way for holistic models of emergence communication (Lazaridou & Baroni, 2020). Our paper is related to this last set of neural works.
  Recent works in emergent communication have been debating the prerequisites to the emergence of language universals such as compositionality (Li & Bowling, 2019; Ren et al., 2020; Lazaridou et al., 2018; Mordatch & Abbeel, 2018; Resnick et al., 2020; Kottur et al., 2017; Choi et al., 2018; Łukasz Kuciński et al., 2020), generalisation (Baroni, 2020; Chaabouni et al., 2020; Hupkes et al., 2020; Denamganaı̈ & Walker, 2020), efficiency (Chaabouni et al., 2019; Rita et al., 2020) or stability (Kharitonov et al., 2020; Lu et al., 2020). Among them, a few works explored how different population structures may impact properties of emergent languages. Inspired by iterated learning methods (Kirby et al., 2008), Ren et al. (2020); Li & Bowling (2019); Lu et al. (2020) look at language evolution across multiple generations of agent pairs, i.e. population is spread over time. However, we consider here a population where multiple agent-pairs coexist simultaneously within a single generation. There, Graesser et al. (2019) show that community of agents start coordinating their language when at least three agents are present in the community. They later assume that increasing the community size may impact the emergent shared language, but did not observe it in their initial experiments. A similar hypothesis was also made by Bouchacourt & Baroni (2019) while analyzing the influence of symmetric agents in the emergence of a common language. With different research objectives, Cogswell et al. (2019) exhibit a slight compositionality enhancement when increasing population size without the need of language transmission. Analogously, Tieleman et al. (2019) explicitly study community size and display a small but consistent gain of abstraction and structure within speakers’ latent representations by increasing population size. Eventually, Fitzgerald (2019) suggest that populations improve regularization and generalization compared to single speaker-listener pairs but underlines that there is not a clear correlation between community size and learning properties. In this paper, we aim at analyzing how population size affects those discussed properties. We align with emergent communication literature and show that naively increasing community size does not consistently improve language properties. We then challenge the homogeneity assumption made in all population designs.
sections.heading: G.1.2 CHANGE 1.2 (REFERRING TO SECTION 2)
sections.heading: Before:
sections.text: Raviv et al. (2019a) went one step further by arguing that the community size is predictive, i.e. a causal factor, of language structure and diversity.
sections.heading: After:
sections.text: As part of their research on the influence of network structures on language emergence (Raviv et al., 2019a;b; 2020), Raviv et al. (2019a) went one step further by arguing that the community size is predictive of language structure and diversity.
sections.heading: G.1.3 CHANGE 1.3 (REFERRING TO SECTION 6)
sections.heading: Before:
sections.text: Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial contradiction.
sections.heading: After:
sections.text: Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial experimental difference such as varying the communication network topology (Wagner, 2009), the proportion of contact agents (Wray & Grace, 2007; Clyne, 1992; Graesser et al., 2019) or the proportion of second learners (Ellis, 2008).
sections.heading: G.1.4 CHANGE 1.4 (REFERRING TO SECTION 5.3)
sections.text: Before:
  Previous sections have identified that learning speed is a powerful controlling factor and could alter language structure when asymmetrizing speaker-listener pairs. However, these experiments remain limited for (i) we have not yet observed larger population-size tend to generate more systematic and structured language, (ii) this asymmetry remains artificial and specific to our Lewis game setting. Therefore, we here leverage this training speed factor within a population, characterizing individual agents with different update probabilities. We hence sample agent update probability with p ∼ Log-N (ηp, σp) with ηp = −1 and σp = 0.5 if not specified otherwise. Larger heterogeneous population leads to higher quality language. In Figure 3, we observe that emergent language metrics are now positively correlated with population size. When increasing the population size from 2 to 20; the speaker synchronization, neg-entropy, compositionality, and generalization increase by 41%, 64%, 8%, and 2% on average. As detailed in Appendix E, compositionality and generalization have a small positive but statistically significant correlation with Spearman coefficients above 0.3 (p-values< 0.05), and we do not observe a language score contraction as initially noted with homogeneous population in Figure 1. Those are the first hints toward our initial goal: when individuals are heterogeneous, a large population size correlates with higher quality and stable language properties.
sections.heading: After:
sections.text: Previous sections have identified that learning speed is a powerful controlling factor and could alter language structure when asymmetrizing speaker-listener pairs. However, these experiments remain limited for (i) we have not yet observed larger population-size tend to generate more systematic and structured language, (ii) this asymmetry remains artificial and specific to our Lewis game setting. Therefore, we here leverage this training speed factor within a population, characterizing individual agents with different update probabilities. We hence sample agent update probability with p ∼ Log-N (ηp, σp) with ηp = −1 and σp = 1 if not specified otherwise. Larger heterogeneous population leads to higher quality language. In Figure 3, we observe that emergent language metrics are now positively correlated with population size. When increasing the population size from 2 to 20 the speaker synchronization and neg-entropy go from low to almost optimal values, compositionality increases by 22% in average and generalization remains stable. As detailed in Appendix E, speaker synchronization, neg-entropy and compositionality have a significant correlation with Spearman coefficients equal to 0.55, 0.24, and 0.21 (p-values< 0.05). Those are the first hints toward our initial goal: when individuals are heterogeneous, a large population size correlates with higher quality and stable language properties. In Appendix F.2, we report the results with distribution changes. When the distribution is more concentrated, trends are less significant and we note that metrics do not have the same sensitivity to heterogeneities: a minimal dispersion of the distribution is required to observe significant effects. When changing the distribution without changing variance (β distribution instead of log-normal distribution), trends are not notably affected.
sections.heading: G.1.5 CHANGE 1.5 (REFERRING TO SECTION 6)
sections.text: Change 1
sections.heading: Before:
sections.text: Yet, as soon as we add diversity within populations, this contradiction vanishes.
sections.heading: After:
sections.text: Yet, as soon as we add diversity within populations, this contradiction partially vanishes.
  Change 2
sections.heading: Before:
sections.text: Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial contradiction.
sections.heading: After:
sections.text: Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial contradiction such as varying the communication network topology (Wagner, 2009), the proportion of contact agents (Wray & Grace, 2007; Clyne, 1992; Graesser et al., 2019) or the proportion of second learners (Ellis, 2008).
sections.heading: G.2 CONCERNING REVIEWER ZXEP COMMENTS
sections.heading: G.2.1 CHANGE 2.1 (REFERRING TO SECTION 5.2)
sections.heading: Before
sections.text: We observe that asymmetrizing speaker-listener learning speed and capacity significantly influences language properties. When the speakers have a large update probability ps or are over-parametrized compared to the listeners, we note a global improvement of language properties: (i) speakers synchronization rate is very high, (ii) language neg-entropy is reaching almost 0, (iii) compositionality is significantly improved and (iv) test accuracy is close its optimal value. On the contrary, when listeners have a large update probability or are over-parametrized compared to the speakers, we note the opposite trend.
sections.heading: After
sections.text: We observe that asymmetrizing speaker-listener learning speed and capacity significantly influences language properties. When the speakers have a large update probability ps or are over-parametrized compared to the listeners, we note a global improvement of language properties: (i) speakers synchronization rate is very high, (ii) language neg-entropy is reaching almost 0, (iii) compositionality is significantly improved and (iv) test accuracy is close its optimal value. On the contrary, when listeners have a large update probability or are over-parametrized compared to the speakers, we note the opposite trend. In Appendix C, we show that the trends are translated along the x-axis when varying setup parameters. Therefore, the ratios at which metrics start becoming optimal are dependent of the setup.
sections.heading: G.2.2 CHANGE 2.2 (REFERRING TO SECTION 5.2)
sections.heading: Before
sections.text: In short: the larger the network, the faster the training speed. We conclude that network capacity is a confounding factor of training speed in our setting.
sections.heading: After
sections.text: In short: the larger the network, the faster the training speed. We conclude that network capacity may be a confounding factor of training speed in our setting and that the high degree of similarity in Figure 2 may be partially explained by the indirect impact of capacity on learning speed.
sections.heading: G.3 CONCERNING REVIEWER UKKL COMMENTS
sections.heading: G.3.1 CHANGE 3.1 (REFERRING TO SECTION 1)
sections.heading: Before
sections.text: In Figure 1, we observe that increasing population size does not improve language properties: speaker synchronization and compositionality remain almost constant, and entropy and generalization are even deteriorating.
sections.heading: After
sections.text: In Figure 1, we observe that increasing population size does not improve language properties: speaker synchronization and compositionality remain almost constant, and entropy and generalization are even deteriorating. As shown in Appendix E, Spearman correlations for those metrics are respectively -0.44, -0.52, -0.15, -0.14 (with p-values < 0.05) meaning that there is no positive correlation between those metrics and population size.”
sections.heading: G.4 CONCERNING REVIEWER 5N9H COMMENTS
sections.text: Changes following Reviewer 5n9H comments are either discussed in Appendix or directly changed in the main text (typography).
sections.heading: G.5 CONCERNING REVIEWER GRQ9 COMMENTS
sections.heading: G.5.1 CHANGE 5.1 (REFERRING TO SECTION 2)
sections.heading: Before
sections.text: Raviv et al. (2019a) went one step further by arguing that the community size is predictive, i.e. a causal factor, of language structure and diversity.
sections.heading: After
sections.text: Raviv et al. (2019a) went one step further by arguing that the community size is predictive, of language structure and diversity.
sections.heading: G.5.2 CHANGE 5.2 (REFERRING TO SECTION 5.3)
sections.heading: Before
sections.text: Trends are particularly significant for neg-entropy and speaker synchronization.
sections.heading: After
sections.text: The relative gain of neg-entropy is almost equal to 100% when σp > 1 while speaker synchronization gain is close to 200% for large values of σp.
title: ON THE ROLE OF POPULATION HETEROGENEITY IN EMERGENT COMMUNICATION