Neural language models (LMs) such as GPT-2 estimate the probability distribution over the next word by a softmax over the vocabulary. The softmax layer produces the distribution based on the dot products of a single hidden state and the embeddings of words in the vocabulary. However, we discover that this single hidden state cannot produce all probability distributions regardless of the LM size or training data size because the single hidden state embedding cannot be close to the embeddings of all the possible next words simultaneously when there are other interfering word embeddings between them. In this work, we demonstrate the importance of this limitation both theoretically and practically. Our work not only deepens our understanding of softmax bottleneck and mixture of softmax (MoS) but also inspires us to propose multi-facet softmax (MFS) to address the limitations of MoS. Extensive empirical analyses confirm our find-ings and show that against MoS, the proposed MFS achieves two-fold improvements in the perplexity of GPT-2 and BERT. Recently, researchers have found that transformer-based language models (LMs), such as GPT-2, can predict the next/masked word distribution better as their sizes grow (Radford et al., 2019; Brown et al., 2020; Kaplan et al., 2020). Compared to greedily outputting the most probable next word, sampling the next word from the predicted distribution allows a LM to generate more diverse and high-quality text sequences (Holtzman et al., 2020). By autoregressively sampling the next word according to its predicted probability, large LMs can be used to assist creative writing (Akoury et al., 2020), reduce the cost of building datasets (West et al., 2021; Liu et al., 2022), generate codes (Li et al., 2022), solve math problems (Cobbe et al., 2021), etc. As a result, one natural question arises: Do modern language modeling architectures still have restrictions in their ability to represent the appropriate distribution over next words or masked words? In this paper, we discover that, when predicting the next word probabilities given an ambiguous context, GPT-2 is often incapable of assigning the highest probabilities to the appropriate non-synonym candidates. For example, given the input prompt After debating whether to bow to the woman or the king first, the jester decided on the [MASK] , we would expect the distribution over the [MASK] fillers to put high probabilities on woman or king or their synonyms. However, GPT-2 might incorrectly assign the second-highest probability to  queen  as in Figure 1. In the final softmax layer of GPT-2, the log probabilities of the woman and king are computed based on the dot product between a single hidden state embedding and the global word embeddings of the woman and king , respectively. To have the highest but similar dot products for the two options, the transformer encoder in GPT-2 wants to output the hidden state that is close to the average of the woman embedding and the king embedding. However, the words queen , king , woman , and man tend to form a parallelogram in the embedding space (Mikolov et al., 2013; Ethayarajh et al., 2019; Wang et al., 2019) 1 , which means the man and queen also have a similar average. Therefore, GPT-2 is forced to also output man or queen when it wants to output woman or king . The problem not only happens to GPT-2 or the words whose embeddings form a parallelogram shape. Even though the hidden state embeddings of LMs are contextualized, the embedding of each 1 Section 2.1 provides more background knowledge about the parallelogram shape and the softmax bottleneck. word in the softmax layer is global and static during the inference time. Globally dissimilar words could all become the suitable next word in a context while other interfering words might be between them, which makes the ideal next word embedding distribution have multiple modes and cannot be modeled by the single embedding representation. In this work, we propose theorems showing that given any LM using the output softmax layer, when there are more than N word embeddings in a N  1 dimensional subspace/hyperplane (e.g., four embeddings in a two-dimensional plane), we can always find a set of possible next words (e.g., woman and king ) such that there are some other interfering words between them (e.g., man or queen ). That is, the multimodal next word distribution must exist if a few word embeddings are linearly dependent. Recently, mixture of softmax (MoS) (Yang et al., 2018) regains attention as one of the few effective architecture modifications for transformer LM (Narang et al., 2021; Anonymous, 2021). In the meanwhile, Parthiban et al. (2021) show that the softmax bottleneck (Yang et al., 2018) theory is not sufficient to explain the improvement of MoS. As a remedy, our theorems not only provide geometrical intuitions of why and when the multiple embedding representation such as MoS would do better but also suggest that the softmax bottleneck might not be completely solved even if we adopt a very large hidden state size. For example, no matter how large the hidden state size is, as long as queen king = woman man in the embedding space, the LMs cannot output a pair of words in the longer diagonal of the parallelogram as the top two output words. After better understanding why mixture of softmax (MoS) works well, we propose two enhancements over MoS. The first enhancement considers the hidden states of multiple positions and multiple transformer layers when determining the probability in each softmax; the second enhancement uses different contextualized embeddings to compute the probabilities of different subsets of words in the vocabulary. The resulting method, multi-facet softmax (MFS), significantly outperforms the MoS and the softmax layer in GPT-2 on the perplexity for predicting the next word, especially in ambiguous context and non-English text in OpenWebText (Rad-ford et al., 2019). Finally, we also show that MFS could improve the performance of GPT-2 on ProtoQA (Boratko et al., 2020), a commonsense question answering dataset where each question has multiple acceptable answers. and empirical contributions as follows. Theory : We show the softmax layer using a single embedding is sometimes not able to output an appropriate rank of probabilities on a set of words with linearly dependent embeddings. Method : Addressing two weaknesses in MoS (Yang et al., 2018), we propose multi-facet softmax (MFS), a new alternative to the output softmax layer. MFS can replace the softmax in pre-trained LMs to better handle ambiguous contexts without re-training the LMs from scratch. Analysis : Our comprehensive empirical analyses discover and explain several phenomena, such as a) why using multiple embeddings is usually better than the single embedding with the non linearity, b) why the improvement is larger in ambiguous contexts, less common languages, or GPT-2 compared to BERT, and c) why a LM often confuses with similar words. In this section, we first review the softmax layer of GPT-2 formally and explain why queen king = woman man still tends to hold in contextualized LMs. Next, we present our theoretical analyses, which generalize the woman and king example by showing that the candidate words in a low dimensional subspace would induce the impossibility of ranking some candidates on top of other candidates. The LMs typically use a softmax layer to predict PS ( x | c t ) , the probability of the next word x given the context at the t th position c t : where h c t is the t th hidden state in the context c , and w x is the output word embedding for the word x (i.e., the linear weights that project the hidden state to the logit of the word x ). 2 Yang et al. (2018) point out that the log probability distribution over all the words in the vocabulary V is log ( PS ( x | c t )) | x  V = h Tc t w x  log (cid:0)(cid:80) x (cid:48) exp( h Tc t w x (cid:48) ) (cid:1) | x  V . The distribution is a linear projection from the hidden state h c t with dimension D , so the degree of freedom in the distribution is only D (i.e., there cannot be more than D linearly independent log distributions). We call this restriction softmax bottleneck theory. 2 Notice that some LMs such as BERT add a bias term for each word before the softmax layer. For simplicity, our theoretical analyses focus on the LMs without the bias term such as GPT-2. During training, the ideal output word embedding w x should be close to the hidden states of the contexts h c t that co-occur with the word x while far away from the other hidden states. This objective is similar to the objective function of Word2Vec (Mikolov et al., 2013) except that the context embeddings are contextualized (Kong et al., 2020; Li et al., 2020). If a context c t has a higher chance to co-occur with queen compared to king , the context also has a higher chance to co-occur with woman compared to man to a similar degree. This is the main reason that makes queen king = woman man in the Word2Vec space (Ethayarajh et al., 2019). Therefore, the same linear relations tend to hold in the output word embedding space of GPT-2 as well (Wang et al., 2019). In addition to words satisfying the analogy relations, the following theorems imply that any linear dependency among the words causes the difficulties of LM in ranking the words in an arbitrary order according to their logits (i.e., dot products between the hidden state and the word embedding). For example, woman + king = queen + man makes a LM unable to assign the highest positive logits to woman and king and output them as the top two words in Figure 1. Theorem 1. If the nonzero output embeddings of N words in a set W are linearly dependent and on one side of a hyperplane through the origin, the single embedding representation cannot produce positive logits for a subset of the words in W that are higher than all the logits of the other words in W . Here, we provide an intuitive justification: if N embeddings are in a subspace whose dimension is smaller than N  1 (e.g., three points in a one-dimensional line), the N embeddings are going to be linearly dependent and some set of words cannot have the top dot products due to the limited degree of freedom in the subspace. In Appendix D, we formally prove the theorem by identifying the sets of words that cannot be ranked top by the single embedding representation. In practice, linear dependency holds approximately instead of exactly. For example, woman = queen + man king +  . In this practical condition, the following theorem states that the logits of the 8050 Input Hidden States (#I) Sec. 3.2 dot product interfering words (i.e., man and queen ) cannot be much smaller than the logits of the candidate words (i.e., woman and king ). Theorem 2. Let the output word embeddings in the set W = { w i (cid:54) = 0 | i = 1 ...N } satisfy w 1 = a 2 w 2 + ... + a N w N +  , where the constant a 2 , ..., a N are neither all zero nor all negative and ||  || < (cid:15) . Then, there must be a nontrivial partition P = { G, S } of W such that there is no hidden state || h ||  r and a threshold   r(cid:15) that make min w g  G h T w g  (1+  )  and max w s  S h T w s <  , where  = 2 1+ (cid:80) i =2 ...N | a i | . In the king-woman example, (1+  ) = (1+ 24 ) = 1 . 5 . Assuming ||  || < (cid:15) = 0 . 01 and || h ||  r = 20 , we can get h T   0 . 01  20 = 0 . 2 . Then, we cannot find a hidden state h such that h T w king  1 . 5  0 . 01  20 = 0 . 3 and h T w woman  0 . 3 but h T w queen < 0 . 2 and h T w man < 0 . 2 because h T w king + h T w woman = h T w queen + h T w man + h T  . The formal proof of Theorem 2 can be found in Appendix D and Appendix B.1 estimates (cid:15) in several language models. Even though, theoretically speaking, outputting woman and king as the top two words is possible due to the appearance of  , LMs may not successfully learn to output the optimal h and the optimal hidden state for these four words could lead to the wrong probabilities of the other words. Consequently, LMs sometimes still rank queen or man higher than woman or king in practice. Using multiple embeddings is a natural solution for modeling a multimodal distribution (Bishop, 1994). For instance, we can use three embeddings to capture the high probability on the woman and king but low probability on the man and queen in Figure 1. Inspired by our geometric analysis on the limitation of the single embedding, we improve the state-of-the-art multiple embedding solution, mixture of softmax (MoS) (Yang et al., 2018) by two enhancements: multiple input hidden states and multiple partitions on the vocabulary. Yang et al. (2018) propose mixture of softmax (MoS) to allow a LSTM-based (Hochreiter and Schmidhuber, 1997) LM to produce more linearly independent log probability distributions of the output words given different contexts. As in Figure 2 (c), the MoS first uses multiple linear layers L fk to project a hidden state h c t into multiple facet embeddings f c t ,k = L fk ( h c t ) . 3 The multiple facets f c t ,k and softmaxes would lead to multiple probability distributions, and output probability is the weighted average of the distributions: P MoS ( x | c t ) = K (cid:88) k =1  c t ,k exp( f T c t ,k w x ) (cid:80) x (cid:48) exp( f Tc t ,k w x (cid:48) ) . The prior weights  c t ,k = exp( L k ( h ct )) (cid:80) k (cid:48) exp( L k (cid:48) ( h ct )) , where L k is another linear projection for dynamically generating the weights and the projection goes through a softmax to ensure (cid:80) Kk =1  c t ,k = 1 . To model the multimodal distribution, the facets (i.e., the embeddings for different softmaxes) should be able to move freely. For example, in Figure 1, we have three facets but only have two modes, so the two embeddings are very close to the word king . However, when we want to output three dissimilar top words such as the king , woman , and knight , one of the facets should be moved to be near to the embedding of the knight . Therefore, we want our solution to satisfy two properties: a) the linear transformation matrix in L fk should have a full rank to avoid limiting the degree of freedom in each facet, and b) the relative location of the facets should be context-dependent. MoS cannot satisfy both properties. If the first one is satisfied, the input hidden state is uniquely determined by a facet (e.g., h c t = ( L f 1 )  1 ( f c t , 1 ) ). Then, there exists a global transformation between two facets (e.g., f c t , 2 = L f 2 (cid:16) ( L f 1 )  1 ( f c t , 1 ) (cid:17) ), which violates the second property. That is, assuming LM can move every facet freely (i.e., the facet's degree of freedom is the same as the dimension of the hidden state), LM cannot make the first two facets close to woman and king in one context but make the two facets close to woman and knight in another context. In other words, since the facet embeddings are the projection of a single hidden state, the total degree of freedom in all facet embeddings cannot exceed the dimension of the hidden state. Our solution to this issue is using more input hidden states to construct the facets. As the orange box in Figure 2, we first concatenate a W  H block of input hidden states into  i =0 ...W  1 ,m =0 ...H  1 h M  m c t  i , where M  m is the transformer layer index and t  i is the index of the i th to the last word in the context. The W  H is fixed as 3  3 in this paper. We make its dimension the same as the original hidden state h Mc t using a linear layer L h plus a GELU activation function (Hendrycks and Gimpel, 2016). Then, we concatenate it with the original hidden state to form a new input hidden state q c t = h Mc t  GELU (cid:16) L h (  i,m h M  m c t  i ) (cid:17) . The new input hidden state is passed through the linear transformation L fk to compute the facets f c t ,k = L fk ( q c t ) and our prior weights  c t ,k = exp ( L k ( q ct )) (cid:80) k (cid:48) exp ( L k (cid:48) ( q ct )) . Since the dimension of q c t is larger than the dimension of f c t ,k , the inverse function ( L fk )  1 no longer exists. The next word distribution could have many modes. However, using many softmaxes significantly increases our computational burden because we need to compute the dot product between each facet and all the word embeddings in our vocabulary. Inspired by our analysis, we propose to split all the words in the vocabulary into multiple partitions 4 and use different facets for different partitions. For example, if we can put any word from { queen , man , woman , king } into one partition and the rest of the words into another partition, we no longer have queen king = woman man in either of the partitions. In this method, each word only belongs to one partition, so we only need to compute one dot product for each word. Thus, the extra computational cost only comes from the extra linear projections for preparing the facets. In many contexts c t , the distribution of the next word has only a single mode and the global similarity between words may be useful. Using the multiple partitions alone might lose the similarity information between words in different partitions. Therefore, we propose to only replace the first softmax layer in MoS with the multiple partition method to learn the global similarity of words in different partitions using the other softmaxes. The architecture is illustrated in Figure 2 (d). Formally, we compute the probability using PMP ( x | c t ) =  c t , 1 exp(( f j x c t , 1 ) T w x ) (cid:80) x (cid:48) exp(( f j x (cid:48) c t , 1 ) T w x (cid:48) ) + K (cid:88) k =2  c t ,k exp( f Tc t ,k w x ) (cid:80) x (cid:48) exp( f Tc t ,k w x (cid:48) ) , (4) where j x is the partition index that the word x belongs to and f j x c t , 1 is the facet for the j x th partition. 4 In this work, we simply put the J  n + j th word into j th partition (e.g., when the number of partitions J = 4 , the first partition includes the words with indexes 0 , 4 , 8 , ... ). This simple global partitioning method reduces the chance of putting all the interfering words and candidates in the same partition, while minimizing the extra computational cost in our PyTorch implementation because PyTorch supports strided index slicing without copying the variable. We evaluate different LM architectures by comparing their capability of predicting the next word in Wikipedia 2021 and a subset of OpenWebText (Radford et al., 2019). In addition to perplexity, we also compare their mean reciprocal ranks (MRR) in Appendix C.1. The size of the training, validation, and testing set are 96%, 2%, and 2% of the whole corpus, respectively. After loading the pre-trained GPT-2 models, we train the GPT-2 Small for 1 epoch and GPT-2 Medium for 0.4 epochs. We also test our methods on BERT in Appendix B.2. Please see Appendix G for more details of our experiment setup. We set different numbers of softmaxes, input hidden states, and partitions in our MFS framework to construct our baselines. The configuration of different baselines could be seen in Table 1. Softmax (GPT-2) : Using a single softmax, input hidden state, and partition as in Figure 2 (a) and Equation 1. The baseline is the same as the original GPT-2 except that we add one more linear layer that converts the hidden state h Mc t to the facet embedding f c t , 1 as in other methods. SigSoftmax (Kanai et al., 2018) : The same as Softmax except when predicting the next word, Kanai et al. (2018) add some non-linearity into the softmax layer by multiplying the exponent and sigmoid of the logits. Softmax + Multi-input : Letting Softmax access multiple input hidden states as in Figure 2 (b) and Equation 3. The method is similar to Tenney et al. (2019); Fan et al. (2020), and Tay et al. (2021). MoS (Yang et al., 2018) : MoS (3) is the mixture of softmax with 3 facets/softmaxes, whose probability comes from Equation 2. We also run the MoS with 4 softmaxes in GPT-2 Small and call the model MoS (4) . DOC (Takase et al., 2018) : Similar to our enhancement using multiple input hidden states, direct output connection (DOC) makes each of their facets coming from a different input hidden state. Other configurations include Softmax + Multi-partition , which adds four partitions into the softmax, MFS w/o Multi-partition , which uses only one partition in MFS and could also be viewed as MoS + Multi-input , and MFS w/o Multi-input , which uses only one input hidden state to generate all facets. Table 1 shows that applying MFS to GPT-2 Small achieves more than 15% of the perplexity improvement between GPT-2 Small and GPT-2 Medium , while only increasing 5% of their size differences. Except for Softmax + Multi-partition , adding multiple input hidden states or partitions in different configurations significantly boost the performances. In Appendix B.3, we further show that the improvement of MFS over Softmax could even become 3-5 times larger in the top 5-10% of the most ambiguous contexts compared to the rest of the contexts, which suggests that some improvements indeed come from successfully modeling multimodal distribution. MFS usually doubles the perplexity improvements between MoS (3) and Softmax but the running time of MFS remains similar to MoS (3) because MFS only needs a few more linear layers, which is more efficient than adding one more softmax as in MoS (4) . DOC is worse than MoS (3) . This may be due to a starvation problem: the facet from the last hidden state h Mc t has the prior probability close to 1 and receives most of the gradients. Finally, compared with Softmax , the mixed results in SigSoftmax suggest that adding non-linearity into the softmax layer without modeling the multimodal distribution might not always improve the models (Parthiban et al., 2021). OpenWebText is mostly composed of English text, but some non-English text in the corpus allows us to compare the capability of different models in a multi-lingual setting. Table 2 shows that multiple embeddings improve the perplexity of the non-English text more than the perplexity of the English text. We hypothesize that the distribution of the next non-English word is more likely to be multi-mode because GPT-2 learns the global token embeddings mostly in the English contexts, which could make the embeddings of similar tokens in non-English contexts far away. In Table 3, we present three contexts from the validation set of different datasets and compare the top three predictions of MFS and Softmax on GPT-2 Small . In OpenWebText and Wikipedia 2021, we can see that Softmax misses the correct answer in its top three predictions. We synthesize a dataset using templates (Ribeiro et al., 2020) to verify whether the softmax layer in the original GPT-2 really has difficulty in learning to output the bimodal distribution in Figure 1 and whether the multiple embedding methods could overcome the problem. First, we collect the four words with semantic analogy relations in Google analogy dataset (Mikolov et al., 2013). Next, we insert two out of the four words into our manually written templates to form the contexts and the templates we used could be found in Appendix G.3. For example, given the context  I went to Paris and Germany before, and I love one of the places more, which is , the GPT-2 learns to predict either Paris or Germany . The two words can be either the diagonal words (e.g., king and woman ) or the edge word (e.g., king and queen ) in the parallelogram. Finally, we create a dataset with 122k training contexts, 250k validation contexts, and 122k testing contexts, where the word pairs in the testing set are unseen in the training set to see whether the model could learn to output the bimodal distribution in a general way. 5 5 The setting is realistic because any related words could become the next word in some ambiguous contexts and all We load the models pre-trained on OpenWebText and continue fine-tuning the models on the last word of each sentence for 10 epochs. We report the testing performances of the best model selected by the validation loss. Since the sets of the word pairs in the training and testing set are disjoint, updating the output word embedding would make GPT-2 solve the task by memorizing/overfitting the training set quickly and lead to much worse testing performances. Thus, we freeze the output word embedding during the training. We visualize the predictions of the Paris-Germany example in the last column of Table 3. We can see two of the softmaxes are close to Paris and the remaining one is close to German , while Softmax overestimates the probability of Paris and ranks France higher than the German . The result verifies that the correct probability distribution of the words in some ambiguous context is hard to learn using Softmax . Quantitatively, Table 4 indicates that when the possible next words are the diagonal words, the Softmax model performs much worse compared to other multiple embedding alternatives. In the edge word dataset, the multiple embedding solutions are still better but have a much smaller gap. MFS w/o Multi-partition slightly improves MoS . We hypothesize the reason is that multiple input hidden states could help the facets to be moved more freely. Finally, multiple partitions seem to cause slight overfitting in this bimodal distribution prediction task. ProtoQA (Boratko et al., 2020) is a question-answering dataset built for evaluating the commonsense reasoning ability of language models. Each question in ProtoQA is ambiguous and leads to a distribution of possible answers. For instance, the answer to  Name something that people usually do before they leave for work?  is  Shower 0.43, Breakfast 0.30, ... . The paper discovers that by reformulating the question-answering task as a context (e.g.,  One thing people usually do before they leave for work is ... ), GPT-2 could generate the possible answers by sampling the next words from its word prediction distribution. The dataset gives us a chance to directly compare the quality of the distributions generated by different LMs in Table 5. After pretraining GPT-2 Medium on the OpenWebText, we fine-tune them using the training data in ProtoQA for 2 epochs. We repeat the fine-tuning 5 times and compare their average perplexity in our validation set. Next, we generate 150 sentences starting from each context and compare the generated answers with the ground truth distribution. For each fine-tuned model, we repeat the generation evaluation 3 times and report the average accuracy of the resulting 15 trials. We can see that the multiple softmaxes, input hidden states, and partitions usually improve the quality of prediction distribution, and the proposed MFS , which combines all modifications, achieves the best performances. Yang et al. (2018) propose the concept of softmax bottleneck , which points out that the dot product in the softmax layer restricts the representation power of outputting arbitrary conditional probabilities. It also proposes MoS to break the softmax bottleneck in an RNN-based LM. Kanai et al. (2018) and Ganea et al. (2019) add nonlinearities into the softmax layer to break the bottleneck more effi-8055 ciently, but the approaches gain less improvement compared to MoS . A limitation of the aforementioned previous work is that they do not tell us which kinds of sentences would be affected by the bottleneck more and whether the order of the top few next words would be affected, which are the main research questions of our work. Contrary to the previous belief that a large hidden state dimension would eliminate the softmax bottleneck, our theorems suggest that some words in a low dimensional subspace could still make the single embedding in the softmax layer become a bottleneck of arbitrarily ranking the output words. Furthermore, our geometric analyses provide an intuitive explanation about why breaking the bottleneck using multiple embeddings leads to better performances compared to only adding the non-linearity. Demeter et al. (2020) also analyze the structural weakness of the softmax layer from a geometric perspective. They discover that the words with high prior frequencies could stop the LMs from assigning the high probabilities to rare words, which can be viewed as a special case of our theory (See Appendix E). For instance, our work shows that the softmax layer could still prevent the LMs from outputting some top words even if all the possible next words have the same prior frequency. Our theory is deeply connected to the mathematical work that counts the number of possible rankings of points in an embedding space (Cover, 1967; Good and Tideman, 1977). Compared to the studies, our work focuses more on analyzing the multimodal distribution in the word embedding space and its implication to language models. An alternative to model the multimodal distribution is to use multiple embeddings to represent each output word (Athiwaratkun and Wilson, 2017; Miao et al., 2019). Compared to MoS or our approach that use multiple embeddings to represent each hidden state of the context, their method requires many extra parameters to store different senses of each output word. Another type of related model (Shazeer et al., 2017; Fedus et al., 2021) dynamically routes the signals to different experts (i.e., feed-forward networks) and Zhang et al. (2022); Mittal et al. (2022) use multiple embeddings in the attention layers. The methods are similar to MoS and our approach, but they add the multiple embeddings inside each layer of the transformer encoder while the proposed MFS is an alternative to the output softmax layer. When the ideal distribution in the output word embedding space has multiple modes, GPT-2 cannot learn to correctly rank the words in all the modes as the top next words. This shows that the single embedding in the softmax layer, which is used nearly universally by current LMs, constitutes a performance upper bound of predicting the next/masked word. To address the systematic failure caused by these structural weaknesses, we propose multi-facet softmax (MFS). In our experiments, we confirm that the MFS significantly outperforms the standard softmax layer and alleviates the softmax bottleneck in the transformer-based LMs such as GPT-2 better than mixture of softmax (MoS). We thank Michael Boratko, Jay Yoon Lee, Sabrina J. Mielke, Steve Cheng-Xian Li, and the anonymous reviewers for their constructive feedback. This work was supported in part by the Center for Data Science and the Center for Intelligent Information Retrieval, in part by the Chan Zuckerberg Initiative under the project Scientific Knowledge Base Construction, in part by the IBM Research AI through the AI Horizons Network, in part using high performance computing equipment obtained under a grant from the Collaborative R&D Fund managed by the Massachusetts Technology Collaborative, in part by the National Science Foundation (NSF) grant numbers DMR-1534431, IIS-1763618, and IIS-1955567, and in part by the Office of Naval Research (ONR) via Contract No. N660011924032 under Subaward No. 123875727 from the University of Southern California. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsor. This work studies a general limitation of LMs and proposes solutions. The proposed theory can help us to understand that some types of hallucinations, mistakes, or biases of LMs could come from softmax bottleneck and their incapability of modeling the correct distribution. For example, there are 60% of male characters and 40% of female characters in our training corpus. The language generation model might be forced to assign more than 60% 8056 probability to male characters as being much more likely to output king than woman in Figure 1. Recently, Narang et al. (2021); Anonymous (2021) show that MoS is one of the few architecture modifications of transformer-based LM that can provide consistent improvements in downstream applications. Hyung Won Chung, Thibault Fvry, Henry Tsai, Melvin Johnson, and Sebastian Ruder. 2021. Rethinking embedding coupling in pre-trained language models. In ICLR . Thomas M Cover. 1967. The number of linearly inducible orderings of points in d-space. SIAM Journal on Applied Mathematics , 15(2):434439. Angela Fan, Thibaut Lavril, Edouard Grave, Armand Joulin, and Sainbayar Sukhbaatar. 2020. Addressing some limitations of transformers with feedback memory. arXiv preprint arXiv:2002.09402 . William Fedus, Barret Zoph, and Noam Shazeer. 2021. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961 . In Proceedings of the 36th International 8057 Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , volume 97 of Proceedings of Machine Learning Research , pages 20732082. Our work provides a fundamental reason why the multiple embedding representation is better, which could inspire more future studies that propose a better multiple-embedding architecture to improve LMs (e.g., multi-lingual BERT) or downstream applications. As examples, we list several possible future directions in Appendix H. Finally, a better LM could lead to both positive and negative societal impacts, but they are not the focus of this paper. Generally speaking, this paper deepens our understanding of the weaknesses of modern LMs and we believe the knowledge can help us to design a better LM that increases the positive impacts and reduces the negative impacts in the future.
abstain abstain method objective objective objective abstain abstain abstain abstain method objective abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain objective abstain objective abstain abstain result objective result abstain objective abstain objective objective objective objective result other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain other method method abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method method abstain abstain other other other abstain method method other abstain abstain method method other method other objective abstain abstain objective result other other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain method