Natural language allows us to refer to novel composite concepts by combining expressions denoting their parts according to systematic rules, a property known as compositionality . In this paper, we study whether the language emerging in deep multi-agent simulations possesses a similar ability to refer to novel primitive combinations, and whether it accomplishes this feat by strategies akin to human-language compositionality. Equipped with new ways to measure compositionality in emergent languages inspired by disentanglement in representation learning, we establish three main results. First, given sufficiently large input spaces, the emergent language will naturally develop the ability to refer to novel composite concepts. Second, there is no correlation between the degree of compositionality of an emergent language and its ability to generalize. Third, while compositionality is not necessary for generalization, it provides an advantage in terms of language transmission: The more compositional a language is, the more easily it will be picked up by new learners, even when the latter differ in architecture from the original agents. We conclude that compositionality does not arise from simple generalization pressure, but if an emergent language does chance upon it, it will be more likely to survive and thrive. Most concepts we need to express are composite in some way. Language gives us the prodigious ability to assemble messages referring to novel composite concepts by systematically combining expressions denoting their parts. As interest raises in developing deep neural agents evolving a communication code to better accomplish cooperative tasks, the question arises of how the emergent code can be  Contributed equally. endowed with the same desirable compositionality property (Kottur et al., 2017; Lazaridou et al., 2018; Mordatch and Abbeel, 2018; Cogswell et al., 2019; Li and Bowling, 2019). This in turn requires measures of how compositional an emergent language is (Andreas, 2019). Compositionality is a core notion in linguistics (Partee, 2004), but linguists' definitions assume full knowledge of primitive expressions and their combination rules, which we lack when analyzing emergent languages (Nefdt, 2020). Also, these definitions are categorical, whereas to compare emergent languages we need to quantify degrees of compositionality. Some researchers equate compositionality with the ability to correctly refer to unseen composite inputs (e.g., Kottur et al., 2017; Cogswell et al., 2019). This approach measures the generalization ability of a language, but it does not provide any insights on how this ability comes about. Indeed, one of our main results below is that emergent languages can attain perfect generalization without abiding to intuitive notions of compositionality. Topographic similarity has become the standard way to quantify the compositionality of emergent languages (e.g., Brighton and Kirby, 2006; Lazaridou et al., 2018; Li and Bowling, 2019). This metric measures whether the distance between two meanings correlates with the distance between the messages expressing them. While more informative than generalization, topographic similarity is still rather agnostic about the nature of composition. For example, when using, as is standard practice, Levenshtein distance to measure message distance, an emergent language transparently concatenating symbols in a fixed order and one mixing deletion and insertion operations on free-ordered symbols can have the same topographic similarity. We introduce here two more opinionated measures of compositionality that capture some intuitive properties of what we would expect to happen in a compositional emergent language. One possibility we consider is that order-independent juxtapositions of primitive forms could denote the corresponding union of meanings, as in English noun conjunctions: cats and dogs , dogs and cats . The second still relies on juxtaposition, but exploits order to denote different classes of meanings, as in English adjective-noun phrases: red triangle , blue square . Both strategies result in disentangled messages, where each primitive symbol (or sym-bol+position pair) univocally refers to a distinct primitive meaning independently of context. We consequently take inspiration from work on disentanglement in representation learning (Suter et al., 2019) to craft measures that quantify whether an emergent language follows one of the proposed composition strategies. Equipped with these metrics, we proceed to ask the following questions. First, are neural agents able to generalize to unseen input combinations in a simple communication game? We find that generalizing languages reliably emerge when the input domain is sufficiently large. This somewhat expected result is important nevertheless, as failure-to-generalize claims in the recent literature are often based on very small input spaces. Second, we unveil a complex interplay between compositionality and generalization. On the one hand, there is no correlation between our compositionality metrics and the ability to generalize, as emergent languages successfully refer to novel composite concepts in inscrutablly entangled ways. (Order-dependent) compositionality, however, if not necessary, turns out to be a sufficient condition for generalization. Finally, more compositional languages are easier to learn for new agents, including agents that are architecturally different from the ones that evolved the language. This suggests that, while composition might not be a natural outcome of the need to generalize, it is a highly desirable one, as compositional languages will more easily be adopted by a large community of different agents. We return to the implications of our findings in the discussion. We designed a variant of Lewis' signaling game (Lewis, 1969). The game proceeds as follows: 1. Sender network receives one input i and chooses a sequence of symbols from its vocabulary V = { s 1 , s 2 ..., s c voc } of size c voc to construct a message m of fixed length c len . 2. Receiver network consumes m and outputs  i . 3. Agents are successful if  i = i , that is, Receiver reconstructs Sender's input. Each input i of the reconstruction game is comprised of i att attributes, each with i val possible values. We let i att range from 2 to 4 and i val from 4 to 100 . We represent each attribute as a i val one-hot vector. An input i is given by the concatenation of its attributes. For a given ( i att , i val ), the number of input samples | I | = i i att val . This environment, which can be seen as an extension of that of Kottur et al. (2017), is one of the simplest possible settings to study the emergence of reference to composite concepts (here, combinations of multiple attributes). Attributes can be seen as describing object properties such as color and shape, with their values specifying those properties for particular objects ( red , round ). Alternatively, they could be seen as slots in an abstract semantic tree (e.g., agent and action), with the values specifying their fillers (e.g., dog , barking ). In the name of maximally simplifying the setup and easing interpretability, unlike Kottur et al. (2017), we consider a single-step game. We moreover focus on input reconstruction instead of discrimination of a target input among distractors as the latter option adds furtherx complications: for example, languages in that setup have been shown to be sensitive to the number and distribution of the distractors (Lazari-dou et al., 2018). For a fixed | I | , we endow Sender with large enough channel capacity | C | = c c len voc ( c voc  { 5 , 10 , 50 , 100 } and c len  { 3 , 4 , 6 , 8 } ) to express the whole input space (i.e., | C |  | I | ). Unless explicitly mentioned, we run 10 different initializations per setting. See Appendix 8.1 for details about the range of tested settings. The game is implemented in EGG (Kharitonov et al., 2019). 1 2.2 Agent architecture Both agents are implemented as single-layer GRU cells (Cho et al., 2014) with hidden states of size 500. 2 Sender encodes i in a message m of fixed 1 Code can be found at https://github.com/ facebookresearch/EGG/tree/master/egg/zoo/compo_vs_generalization . length c len as follows. First, a linear layer maps the input vector into the initial hidden state of Sender. Next, the message is generated symbol-by-symbol by sampling from a Categorical distribution over the vocabulary c voc , parameterized by a linear mapping from Sender's hidden state. The generated symbols are fed back to the cell. At test time, instead of sampling, symbols are selected greedily. Receiver consumes the entire message m . Further, we pass its hidden state through a linear layer and consider the resulting vector as a concatenation of i att probability vectors over i val values each. As a loss, we use the average cross-entropy between these distributions and Sender's input. Popular approaches for training with discrete communication include Gumbel-Softmax (Mad-dison et al., 2016; Jang et al., 2016), REINFORCE (Williams, 1992), and a hybrid in which the Receiver gradients are calculated via back-propagation and those of Sender via REINFORCE (Schulman et al., 2015). We use the latter, as recent work (e.g., Chaabouni et al., 2019) found it to converge more robustly. We apply standard tricks to improve convergence: (a) running mean baseline to reduce the variance of the gradient estimates (Williams, 1992), and (b) a term in the loss that favors higher entropy of Sender's output, thus promoting exploration. The obtained gradients are passed to the Adam optimizer (Kingma and Ba, 2014) with learning rate 0 . 001 . Topographic similarity (topsim) (Brighton and Kirby, 2006) is commonly used in language emergence studies as a quantitative proxy for compositionality (e.g., Lazaridou et al., 2018; Li and Bowling, 2019). Given a distance function in the input space (in our case, attribute value overlap, as attributes are unordered, and values categorical) and a distance function in message space (in our case, following standard practice, minimum edit distance between messages), topsim is the (Spearman) correlation between pairwise input distances and the corresponding message distances. The measure can detect a tendency for messages with similar meanings to be similar in form, but it is relatively results with LSTMs, that were slower to converge. agnostic about the type of similarity (as long as it is captured by minimum edit distance). We complement topsim with two measures that probe for more specific types of compositionality, that we believe capture what deep-agent emergent-language researchers seek for, when interested in compositional languages. In most scenarios currently considered in this line of research, the composite inputs agents must refer to are sets or sequences of primitive elements: for example, the values of a set of attributes, as in our experiment. In this restricted setup, a compositional language is a language where symbols independently referring to primitive input elements can be juxtaposed to jointly refer to the input ensembles. Consider a language with a symbol r referring to input element color:red and another symbol l referring to weight:light , where r and l can be juxtaposed (possibly, in accordance with the syntactic rules of the language) to refer to the input set { color:red , weight:light } . This language is intuitively compositional. On the other hand, a language where both r and l refer to these two input elements, but only when used together, whereas other symbol combinations would refer to color:red and weight:light in other contexts, is intuitively not compositional. Natural languages support forms of compositionality beyond the simple juxtaposition of context-independent symbols to denote ensembles of input elements we are considering here (e.g., constructions that denote the application of functions to arguments). However, we believe that the proposed intuition is adequate for the current state of affairs in language emergence research. The view of compositionality we just sketched is closely related to the idea of disentanglement in representation learning. Disentangled representations are expected to enable a consequent model to generalize on new domains and tasks (Bengio et al., 2013). Even if this claim has been challenged (Bozkurt et al., 2019; Locatello et al., 2019), several interesting metrics have been proposed to quantify disentanglement, as reviewed in Suter et al. (2019). We build in particular upon the Information Gap disentanglement measure of Chen et al. (2018), evaluating how well representations capture independence in the input sets. metric measures whether symbols in specific positions tend to univocally refer to the values of a specific attribute. This order-dependent strategy is commonly encountered in natural language structures (and it is a pre-condition for sophisticated syntactic structures to emerge). Consider English adjective-noun phrases with a fully intersective interpretation, such as yellow triangle . Here, the words in the first slot will refer to adjectival meanings, those in the second to nominal meanings. In our simple environment, it might be the case that the first symbol is used to discriminate among values of an attribute, and the second to discriminate among values of another attribute. Let's denote s j the j th symbol of a message and a j 1 the attribute that has the highest mutual information with s j : a j 1 = arg max a I ( s j ; a ) . In turn, a j 2 is the second highest informative attribute, a j 2 = arg max a (cid:54) = a j 1 I ( s j ; a ) . Denoting H ( s j ) the entropy of j -th position (used as a normalizing term), we define posdis as: posdis = 1 /c len c len (cid:88) j =1 I ( s j ; a j 1 )  I ( s j ; a j 2 ) H ( s j ) (1) We ignore positions with zero entropy. Eq. 1 captures the intuition that, for a language to be compositional given our inputs, each position of the message should only be informative about a single attribute. However, unlike the related measure proposed by Resnick et al. (2019), it does not require knowing which set of positions encodes a particular attribute, which makes it computationally simpler (only linear in c len ). Posdis assumes that a language uses positional information to disambiguate symbols. However, we can easily imagine a language where symbols univocally refer to distinct input elements independently of where they occur, making order irrelevant. 3 Hence, we also introduce bag-of-symbols disentanglement (bosdis) . The latter maintains the requirement for symbols to univocally refer to distinct meanings, but captures the intuition of a permutation-invariant language, where only symbol counts are informative. Denoting by n j a counter of the j -th symbol in a message, bosdis is given by: bosdis = 1 /c voc c voc (cid:88) j =1 I ( n j ; a j 1 )  I ( n j ; a j 2 ) H ( n j ) (2) In all experiments, the proposed measures topsim , posdis and bosdis are calculated on the train set. 3 This is not unlike what happens in order-insensitive constructions such as English conjunctions: dogs and cats , cats and dogs . In Appendix 8.2, we illustrate how the three metrics behave differently on three miniature languages. Across the languages of all converging runs in our simulations, their Spearman correlations are: topsim / posdis : 0 . 08 ; topsim / bosdis : 0 . 38 ; posdis / bosdis : 0 . 31 . These correlations, while not extremely high, are statistically significant ( p < 0 . 01 ), which is reassuring as all metrics attempt to capture compositionality. It is also in line with reasonable expectations that the most opinionated posdis measure is the one that behaves most differently from topsim . In our setup, generalization can be straightforwardly measured by splitting all possible distinct inputs so that the test set only contains inputs with attribute combinations that were not observed at training. Generalization is then simply quantified by test accuracy. In intuitive terms, at training time the agents are exposed to blue triangles and red circles , but blue circles only appear at test time. This requires Sender to generate new messages, and Receiver to correctly infer their meaning. If a blue circle is accurately reconstructed, then agents do generalize. For all the considered settings, we split the possible distinct inputs into 90% train and 10% test items. This implies that the absolute training/test set sizes increase with input dimension (this issue is further discussed in Appendix 8.4). Finally, we only evaluate generalization for runs that successfully converged, where convergence is operationalized as > 99 . 9% training-set accuracy. Fig. 1 shows that emergent languages are able to almost perfectly generalize to unseen combinations as long as input size | I | is sufficiently large (input size/test accuracy Spearman  = 0 . 86 , p  0 ). The figure also shows that the way in which a large input space is obtained (manipulating i att or i val ) does not matter (no significant accuracy difference between the bracketed runs, according to a set of t-tests with p > 0 . 01 ). Moreover, the correlation is robust to varying agents' capacity (Appendix 8.3; see Resnick et al. (2019) for a thorough study of how agent capacity impacts generalization and compositionality). Importantly, the effect is not simply a product of larger input sizes coming with 25 ( 2 , 5 ) 100 ( 2 , 10 ) 125 ( 3 , 5 ) 256 ( 4 , 4 ) 256 ( 2 , 16 ) 625 ( 4 , 5 ) 625 ( 2 , 25 ) 1000 ( 3 , 10 ) 2500 ( 2 , 50 ) 10000 ( 4 , 10 ) 10000 ( 2 , 100 ) input size 0.0 0.2 0.4 0.6 0.8 1.0 t e s t a cc u r a c y Figure 1: Average accuracy on unseen combinations as a function of input size of successful runs. larger training corpora, as we replicate it in Appendix 8.4 while keeping the number of distinct training examples fixed, but varying input combinatorial variety . What matters is that, in the training data, specific attribute values tend to occur with a large range of values from other attributes, providing a cue about the composite nature of the input. That languages capable to generalize will only emerge when the input is varied enough might seem obvious, and it has been shown before in mathematical simulations of language emergence (Nowak et al., 2000), as well as in studies of deep network inductive biases (Zhao et al., 2018). However, our result suggests an important caveat when interpreting experiments based on small input environments that report failures in the generalization abilities of deep networks (e.g., Kottur et al., 2017; Lake and Baroni, 2018). Before assuming that special architectures or training methods are needed for generalization to emerge, such experiments should be repeated with much larger/varied input spaces, where it is harder for agents to develop ad-hoc strategies overfitting the training data and failing to generalize. We also considered the relation between channel capacity | C | and language emergence. Note that | C |  | I | is a prerequisite for successful communication, and a perfectly compositional language could already generalize at the lower | C | = | I | bound. Indeed, limiting channel capacity has been proposed as an important constraint for the emergence of compositionality (Nowak and Krakauer, 1999). However, we find that, when | I | is sufficiently large to support generalization, our deep agents need | C | > | I | in order to even converge at training time. The minimum | C | / | I | ratio across all converging runs for each configuration with | I |  625 (the settings where we witness generalizing languages) is on average 5.9 (s.d.: 4.4). Concretely, this implies that none of our successful languages is as compact as a minimal fully-compositional solution would afford. Appendix 8.5 reports experiments focusing, more specifically, on the relation between channel capacity and generalization, showing that it is essential for | C | to be above a large threshold to reach near-perfect accuracy, and further increasing | C | beyond that does not hamper generalization. Having established that emergent languages can generalize to new composite concepts, we test whether languages that generalize better are also more compositional. Since bosdis and topsim correlate with | C | (Appendix 8.6), we compute Spearman correlations between test accuracy and compositionality metrics across all converging runs of each ( i att , i val , c len , c voc ) configuration separately. Surprisingly, in just 4 out of 141 distinct settings the correlation is significant ( p < 0 . 01 ) for at least 1 measure. 4 We further analyze the ( i att = 2 , i val = 100 , c len = 3 , c voc = 100 ) setting, as it has a large number of generalizing runs, and it is representative of the general absence of correlation we also observe elsewhere. Fig. 2 confirms that even non-compositional languages (w.r.t. any definition of compositionality) can generalize well. Indeed, for very high test accuracy ( > 98% ), we witness a large spread of posdis (between 0 . 02 and 0 . 72 ), bosdis (between 0 . 03 and 0 . 4 ) and topsim (between 0 . 11 and 0 . 64 ). In other words, deep agents are able to communicate about new attribute combinations while using non-compositional languages. We note moreover that even the most compositional languages according to any metric are far from the theoretical maximum ( = 1 for all metrics). We observe however that the top-left quadrants of Fig. 2 panels are empty. In other words, it never happens that a highly compositional language has low accuracy. To verify this more thoroughly, for each compositionality measure  , we select those languages, among all converging runs in all con-4 3 , 3 and 1 (different) significant settings for topsim , posdis and bosdis , respectively. figurations , that have  > 0 . 5 , and compute the proportion of them that reaches high test accuracy ( > 0 . 80 ). We find that this ratio equates 0 . 90 , 0 . 50 , and 0 . 11 for posdis , bosdis , and topsim respectively. That is, while compositionality is not a necessary condition for generalization, it appears that the strongest form of compositionality, namely posdis , is at least sufficient for generalization. This provides some evidence that compositionality is still a desirable feature, as further discussed in Section 6. We gain further insights on what it means to generalize without full compositionality by taking a deeper look at the language shown in red in Fig. 2, that has near-perfect generalization accuracy ( > 99%), and whose posdis score (0.70), while near the relative best, is still far from the theoretical maximum (we focus on posdis since it is the easiest compositional strategy to qualitatively char-acterize). As its behavior is partially interpretable, this mediumposdis  language offered us clearer insights than more strongly entangled cases. We partially analyze one of the latter in Appendix 8.7. Note that, with ( i att = 2 , i val = 100 ), a ( c len = 2 , c voc = 100 ) channel should suffice for a perfectly positionally disentangled strategy. Why does the analyzed language use ( c len = 3 ) instead? Looking at its mutual information profile (Appendix Table 5), we observe that positions 2 and 3 ( pos2 and pos3 ) are respectively denoting attributes 2 and 1 ( att2 and att1 ): pos3 has high mutual information with att1 and low mutual information with att2 ; the opposite holds for pos2 . The remaining position, pos1 , could then be simply redundant with respect to the others, or encode noise ignored by Receiver. However, this is not quite the case, as the language settled instead for a form of leaky disentanglement. The two disentangled positions do most of the job, but the third, more entangled one, is still necessary for perfect communication. To see this, consider the ablations in Table 1. Look first at the top block, where the trained Receiver of the relevant run is fed messages with the symbol in one original position preserved, the others shuffled. Confirming that communication is largely happening by disentangled means, preserving pos2 alone suffices to have Receiver guessing a large majority of att2 values, and keeping pos3 unchanged is enough to guess almost 90% of att1 values correctly. Conversely, preserving pos1 alone causes a complete drop in accuracy for both attributes. However, neither pos2 nor pos3 are sufficient on their own to perfectly predict the corresponding attributes. Indeed, the results in the bottom block of the table (one symbol shuffled while the others stay in their original position) confirm that pos1 carries useful complementary information: when fixing the latter and either one of the other positions , we achieve 100% accuracy for the relevant attribute ( att2 for pos1 + pos2 and att1 for pos1 + pos3 ), respectively. In sum, pos2 and pos3 largely specialized as predictors of att2 and att1 , respectively. However, they both have a margin of ambiguity (in pos2 and pos3 there are 96 and 98 symbols effectively used, respectively, whereas a perfect 1-to-1 strategy would require 100). When the symbols in these positions do not suffice, pos1 , that can refer to both attributes, serves a disambiguating role. We quantified this complementary function as follows. We define the cue validity of s p (symbol in position p ) w.r.t an attribute a as CV ( s p , a ) = max  a P ( a | s p ) , where  a iterates over all possible values of a . CV ( s pos 1 , att 2) is significantly higher in those (train/test) messages where CV ( s pos 2 , att 2) is below average. Similarly, CV ( s pos 1 , att 1) is significantly higher in messages where CV ( s pos 3 , att 1) is below average ( p  0 in both cases). We might add that, while there is a huge difference between our simple emergent codes and natural languages, the latter are not perfectly disentangled either, as they feature extensive lexical ambiguity, typically resolved in a phrasal context (Piantadosi et al., 2012). The need to generalize to new composite inputs does not appear to constitute a sufficient pressure to develop a compositional language. Given that compositionality is ubiquitous in natural language, we conjecture that it has other beneficial properties, making it advantageous once agents chanced upon it. Compositional codes are certainly easier to read out by humans (as shown by our own difficulty in qualitatively analyzing highly entangled languages), and we might hypothesize that this ease-of-decoding is shared by computational agents. A long tradition of subject studies and computational simulations has shown that the need to transmit a language across multiple generations or to populations of new learners results in the language being more compositional (e.g., Kirby, 2001; Kirby et al., 2015; Verhoef et al., 2016; Cornish et al., 2017; Cogswell et al., 2019; Guo et al., 2019; Li and Bowling, 2019). Our next experiments are closely related to this earlier work, but we adopt the opposite perspective. Instead of asking whether the pressure to transmit a language will make it more compositional, we test whether languages that have already emerged as compositional, being easier to decode, are more readily transmitted to new learners. 5 Specifically, we run 30 games in the largest input setting ( i att = 2 , i val = 100 ), varying the channel parameters. We select the pairs of agents that achieved a high level of generalization accuracy (  0.80). Next, following the paradigm of Li and Bowling (2019), we freeze Sender, and train a new 5 Li and Bowling (2019) established this for hand-crafted languages; we extend the result to spontaneously emerging ones. Receiver from scratch. We repeat this process 3 times per game, initializing new Receivers with different random seeds. Once the newly formed pair of agents is successful on the training set, we measure its test accuracy. We also report speed of learning, measured by area under the epochs vs. training accuracy curve. We experiment with three Receiver architectures. The first two, GRU (500) and GRU (50), are GRUs with hidden layer sizes of 500 (identical to the original Receiver) and 50, respectively. The third is a two-layer Feed-Forward Network (FFN) with a ReLu non-linearity and hidden size 500. The latter Receiver takes the flattened one-hot representation of the message as its input. This setup allows probing ease of language transmission across models of different complexity. We leave the study of language propagation across multiple generations of speakers to future work. Results in the same setting studied in Section 5 are presented in Table 2 (experiments with other setups are in Appendix 8.8). Both learning speed and generalization accuracy of new Receivers are strongly positively correlated with degree of compositionality . The observed correlations reach values almost as high as 0 . 90 for learning speed and 0 . 80 for generalization, supporting our hypothesis that, when emergent languages are compositional, they are simpler to understand for new agents, including smaller ones (GRU (50)), and those with a different architecture (FFN). The natural emergence of generalization There has been much discussion on the generalization capabilities of neural networks, particularly in linguistic tasks where humans rely on compositionality (e.g., Fodor and Lepore, 2002; Marcus, 2003; van der Velde et al., 2004; Brakel and Frank, 2009; Kottur et al., 2017; Lake and Baroni, 2018; Andreas, 2019; Hupkes et al., 2019; Resnick et al., 2019). In our setting, the emergence of generalization is very strongly correlated with variety of the input environment. While this result should be replicated in different conditions, it suggests that it is dangerous to study the generalization abilities of neural networks in thought experiment setups where they are only exposed to a small pool of carefully-crafted examples. Before concluding that garden-variety neural networks do not generalize, the simple strategy of exposing them to a richer input should always be tried. Indeed, even studies of the origin of human language conjecture that the latter did not develop sophisticated generalization mechanisms until pressures from an increasingly complex environment forced it to evolve in that direction (Bickerton, 2014; Hurford, 2014). Generalization without compositionality Our most important result is that there is virtually no correlation between whether emergent languages are able to generalize to novel composite inputs and the presence of compositionality in their messages (Andreas (2019) noted in passing the emergence of non-compositional generalizing languages, but did not explore this phenomenon systematically). Supporting generalization to new composite inputs is seen as one of the core purposes of compositionality in natural language (e.g., Pagin and Westerstahl, 2010). While there is no doubt that compositional languages do support generalization, we also found other systems spontaneously arising that generalize without being compositional, at least according to our intuitive measures of compositionality. This has implications for the ongoing debate on the origins of compositionality in natural language, (e.g., Townsend et al., 2018, and references there), as it suggests that the need to generalize alone might not constitute a sufficient pressure to develop a fully compositional language. Our result might also speak to those linguists who are exploring the non-fully-compositional corners of natural language (e.g., Goldberg, 2019). A thorough investigation of neural network codes that can generalize while being partially entangled might shed light on similar phenomena in human languages. Finally, and perhaps most importantly, recent interest in compositionality among AI researchers stems from the assumption that compositionality is crucial to achieve good generalization through language (e.g., Lake and Baroni, 2018; Lazaridou et al., 2018; Baan et al., 2019). Our results suggest that the pursuit of generalization might be separated from that of compositionality, a point also recently made by Kharitonov and Baroni (2020) through hand-crafted simulations. What is compositionality good for? We observed that positional disentanglement, while not necessary, is sufficient for generalization. If agents develop a compositional language, they are then very likely to be able to use it correctly to refer to novel inputs. This supports the intuition that compositional languages are easier to fully understand. Indeed, when training new agents on emerged languages that generalize, it is much more likely that the new agents will learn them fast and thoroughly (i.e., they will be able to understand expressions referring to novel inputs) if the languages are already compositional according to our measures. That language transmission increases pressure for structured representations is an established fact (e.g., Kirby et al., 2015; Cornish et al., 2017). Here, we reversed the arrow of causality and showed that, if compositionality emerges (due to chance during initial language development), it will make a language easier to transmit to new agents. Compositionality might act like a dominant genetic feature: it might arise by a random mutation but, once present, it will survive and thrive, as it guarantees that languages possessing it will generalize and will be easier to learn. From an AI perspective, this suggests that trying to enforce compositionality during language emergence will increase the odds of developing languages that are quickly usable by wide communities of artificial agents, that might be endowed with different architectures. From the linguistic perspective, our results suggest an alternative view of the relation between compositionality and language transmissionone in which the former might arise by chance or due to other factors, but then makes the resulting language much easier to be spread. Compositionality and disentanglement Language is a way to represent meaning through discrete symbols. It is thus worth exploring the link between the area of language emergence and that of representation learning (Bengio et al., 2013). We took this route, borrowing ideas from research on disentangled representations to craft our compositionality measures. We focused in particular on the intuition that, if emergent languages must denote ensembles of primitive input elements, they are compositional when they use symbols to univocally denote input elements independently of each other. While the new measures we proposed are not highly correlated with topographic similarity, in most of our experiments they did not behave significantly differently from the latter. On the one hand, given that topographic similarity is an established way to quantify compositionality, this serves as a sanity check on the new measures. On the other, we are disappointed that we did not find more significant differences between the three measures. Interestingly one of the ways in which they did differ is that, when a language is positionally disentangled, (and, to a lesser extent, bag-of-symbols disentangled), it is very likely that the language will be able to generalizea guarantee we don't have from less informative topographic similarity. The representation learning literature is not only proposing disentanglement measures, but also ways to encourage emergence of disentanglement in learned representations. As we argued that compositionality has, after all, desirable properties, future work could adapt methods for learning disentangled representations (e.g., Higgins et al., 2017; Kim and Mnih, 2018) to let (more) compositional languages emerge. We thank the reviewers for feedback that helped us to make the paper clearer.
objective objective objective abstain abstain abstain abstain abstain objective abstain abstain abstain method method abstain abstain result abstain abstain abstain abstain abstain method abstain abstain abstain method abstain result abstain abstain objective abstain abstain abstain abstain method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain method method method method abstain abstain abstain other abstain abstain abstain abstain abstain abstain method method abstain method method abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain other method abstain abstain abstain abstain abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain method method abstain abstain result abstain abstain abstain other