We provide an NLP framework to uncover four linguistic dimensions of political polarization in social media: topic choice, framing, affect and illocutionary force. We quantify these aspects with existing lexical methods, and propose clustering of tweet embeddings as a means to identify salient topics for analysis across events; human evaluations show that our approach generates more cohesive topics than traditional LDA-based models. We apply our methods to study 4.4M tweets on 21 mass shootings. We provide evidence that the discussion of these events is highly polarized politically and that this polarization is primarily driven by partisan differences in framing rather than topic choice. We identify framing devices, such as grounding and the contrasting use of the terms terrorist and crazy, that contribute to polarization. Results pertaining to topic choice, affect and illocutionary force suggest that Republicans focus more on the shooter and event-specific facts (news) while Democrats focus more on the victims and call for policy changes. Our work contributes to a deeper understanding of the way group divisions manifest in language and to computational methods for studying them. 1 1 Introduction Elites, political parties, and the media in the US are increasingly polarized (Layman et al., 2010; Prior, 2013; Gentzkow et al., forthcoming), and the propagation of partisan frames can influence public opinion (Chong and Druckman, 2007) and party identification (Fiorina and Abrams, 2008). Americans increasingly get their news from internet-based sources (Mitchell et al., 2016), and political information-sharing is highly ideologically segregated on platforms like Twitter (Conover et al., 2011; Halberstam and Knight, 1 All data and code is available at: https://github. com/ddemszky/framing-twitter 2016) and Facebook (Bakshy et al., 2015). Prior NLP work has shown, e.g., that polarized messages are more likely to be shared (Zafar et al., 2016) and that certain topics are more polarizing (Balasubramanyan et al., 2012); however, we lack a more broad understanding of the many ways that polarization can be instantiated linguistically. This work builds a more comprehensive framework for studying linguistic aspects of polarization in social media, by looking at topic choice, framing, affect, and illocutionary force. We explore these aspects of polarization by studying a sample of more than 4.4M tweets about 21 mass shooting events, analyzing polarization within and across events. Framing and polarization in the context of mass shootings is well-studied, though much of the literature studies the role of media (Chyi and McCombs, 2004; Schildkraut and Elsass, 2016) and politicians (Johnson et al., 2017). Several works find that frames have changed over time and between such events (Muschert and Carr, 2006; Schildkraut and Muschert, 2014), and that frames influence opinions on gun policies (HaiderMarkel and Joslyn, 2001). Prior NLP work in this area has considered how to extract factual information on gun violence from news (Pavlick et al., 2016) as well as quantify stance and public opinion on Twitter (Benton et al., 2016) and across the web (Ayers et al., 2016); here we advance NLP approaches to the public discourse surrounding gun violence by introducing methods to analyze other linguistic manifestations of polarization. We are particularly interested in the role of the shooter's race in shaping polarized responses to these events. Implicit or explicit racial biases can be central in people's understanding of social problems (Drakulich, 2015); in the mass shooting context, race is a factor in an event's newsworthiness (Schildkraut et al., 2018) and is often mentioned prominently in media coverage, particularly when the shooter is non-white (Mingus and Zopf, 2010; Park et al., 2012). Duxbury et al. (2018) find that media representations of white shooters disproportionately divert blame by framing them as mentally ill while representations of non-white shooters are more frequently criminal-ized, highlighting histories of violent behavior. The important question remains as to how polarized ideologies surrounding race take shape on forums such as Twitter. Therefore, in all of the analyses throughout this paper we consider the race of the shooter as a potential factor. We note that in the 21 shooting events we study, shootings in schools and places of worship are overwhelmingly carried out by white perpetrators, so we cannot fully disentangle the effect of race from other factors. Data collection. We compiled a list of mass shootings between 2015 and 2018 from the Gun Violence Archive. 2 For each, we identified a list of keywords representative of their location (see Appendix A). Given Twitter search API's limitations on past tweets, we retrieved data from a Stanford lab's archived intake of the Twitter firehose. 3 For each event, we built a list of relevant tweets for the two weeks following the event. A tweet is relevant if it contained at least one of the event's location-based representative keywords and at least one lemma from the following list: shoot, gun, kill, attack, massacre, victim. We filtered out retweets and tweets from users who have since been deactivated. We kept those 21 events with more than 10,000 tweets remaining. For more details see Appendix A. Partisan assignment. We estimate the party af-filiation of users in the dataset from the political accounts they follow using a method similar to that of Volkova et al. (2014), which takes advantage of homophily in the following behavior of users on twitter (Halberstam and Knight, 2016). We compile a list of Twitter handles of US Congress members in 2017, the 2016 presidential and vice 2 https://www.gunviolencearchive.org 3 With the exception of the two most recent shootings in Pittsburgh and Thousand Oaks, for which we collected tweets real time via the Twitter API. presidential candidates, and other party-affiliated pages. 4 We label a user as a Democrat if they followed more Democratic than Republican politicians in November 2017, and as a Republican if the reverse is true. For each event, 5172% of users can be assigned partisanship in this way; to validate our method we compare state averages of these inferred partisan labels to state two-party vote shares, finding a high correlation (Figure 1). 5 3 Quantifying Overall Polarization We begin by quantifying polarization (equiva-lently, partisanship) between the language of users labeled Democrats and Republicans after mass shooting events. We establish that there is substantial polarization, and that the polarization increases over time within most events. Pre-processing. We first build a vocabulary for each event as follows. Each vocabulary contains unigrams and bigrams that occur in a given event's tweets at least 50 times, counted after stemming via NLTK's SnowballStemmer and stopword removal. 6 We refer to these unigrams and bigrams collectively as tokens . 4 See Appendix B.1 for the complete list. 5 We performed the sanity check for all partisan users with a valid US state as part of their geo-location (  350k users). 6 Stopword list is provided in Appendix A.1 Measure of partisanship. We apply the leave-out estimator of phrase partisanship from Gentzkow et al. (forthcoming). Partisanship is defined as the expected posterior probability that an observer with a neutral prior would assign to a tweeter's true party after observing a single token drawn at random from the tweets produced by the tweeter. If there is no difference in token usage between the two parties, then this probability is . 5 , i.e. we cannot guess the user's party any better after observing a token. The leave-out estimator consistently estimates partisanship under the assumption that a user's tokens are drawn from a multinomial logit model. The estimator is robust to corpus size. The leave-out estimate of partisanship  LO between Democrats i 2 D and Republicans i 2 R is  LO = 1 2  1 | D | X i 2 D q i     i + 1 | R | X i 2 R q i  (1     i )  where q i = c i /m i is the vector of empirical token frequencies for tweeter i , with c i being the vector of token counts for tweeter i and m i the sum of token counts for tweeter i ; and    i = ( q D \ i  ( q D \ i + q R \ i )) is a vector of empirical posterior probabilities, excluding speaker i and any token that is not used by at least two speakers. Here we let  denote element-wise division and q G = P i 2 G c i / P i 2 G m i denote the empirical token frequency of tweeters in group G . The estimator thus captures two intuitive components of partisanship: between-group difference (poste-rior probability for each feature), and within-group similarity (dot-product between the feature vector of each speaker and that of their group). User-level measures. As the above leave-out estimator represents the average of user-level polarization values, we take the user-level dot product ( q i     i ) as an estimate of the polarization of user i 's language. We consider the correlation of this value and the number of politicians a user follows in total and from their preferred party. Overall polarization. As Figure 2 shows, the discussion of each event 7 is highly polarized: values range from . 517 to . 547 . For comparison, this 7 For all the experiments using the leave-out estimator, we exclude Fort Lauderdale for which we only have tweets for the first day after the event; polarization is most dominant a few days after each event, making it incomparable. For reference, the leave-out estimate for Fort Lauderdale is . 51 . measure, for most events, is similar to or higher than the polarization in the US congress (  . 53 in recent years) (Gentzkow et al., forthcoming). While we observe a slight increase in polarization over the past three years, this increase is not statistically significant ( p  . 26 ). Post-event polarization. To see how polarization changes at the event level, we computed the leave-out estimate for each of the first 10 days following the events (see Figure 3). An event-day level regression of partisanship on days since the event suggests a slight increase in post-event polarization across events (slope = . 002 , p < 0 . 05 ). Fitting separate regressions, we find that the five events with the steepest increase in polarization are Burlington (slope = . 03 , p < 0 . 05 ), Orlando (slope = . 006 , p < 0 . 001 ), Las Vegas (slope = . 003 , p < 0 . 001 ), Chattanooga (slope = . 003 , p < 0 . 05 ) and Roseburg (slope = . 003 , p < 0 . 05 )  note that except for Las Vegas, these shootings are all committed by a person of color. Are the changes in the leave-out score due to different users tweeting at different times or due to the same users becoming more or less political? We found that while on average only  10% of users tweeted on multiple days ( SD = 5% ) across the events, these users contribute  28% of the tweets ( SD = 15% ). After removing these users from the leave-out estimation, we found that the temporal patterns remain with the same statistical significance, providing one piece of evidence 0.50 0.55 0.60 0.65 2.5 5.0 7.5 10.0 Day after event Lea v e  ou t e s t i m a t e Burlington Chattanooga Orlando Roseburg Vegas all other Figure 3: Leave-out estimate post-event. that changes in polarization are not due to changes within users who tweet on multiple days. User-level polarization. We estimated a linear regression of the leave-out score on the total number of followed politicians and the number from the user's preferred party, with controls for event indicators. The estimates imply that, fixing the total number of followed politicians, one more followed politician from one's preferred party is associated with an increase of . 009 SD in the leave-out. Fixing the number of followed politicians from the user's preferred party, one more followed politician is associated with a decrease of . 02 SD in the leave-out. Topic choice can be a tool for agenda-setting by establishing what an author or institution deems worthy of discussion (McCombs, 2002), and works in NLP have used topic modeling as an approach to measure this effect (Tsur et al., 2015; Field et al., 2018). The strategy of highlighting particular aspects within topics as a means of framing (Entman, 2007) has also been quantified in the NLP literature (Boydstun et al., 2013; Card et al., 2015; Naderi and Hirst, 2017). Previous work largely focuses on the relation between topic and framing in the news media; we study social media, proposing methods to identify general, non-event-specific topics and to quantify betweenand within-topic polarization. Topic assignment. Our goal is to induce topics that are salient in our narrow domain and comparable across events. This presents a challenge for traditional topic modeling approaches, since the discourse surrounding these events is inherently tied to concrete aspects of the events that tend to covary with topic usage, like location, setting, and demographics of the shooter and victims. We build on the ability of vector space models to represent higher-level semantics to develop our own embedding-based topic assignment approach, comparing it with two traditional LDA-based methods: MALLET 8 and the Biterm Topic Model (BTM) (Yan et al., 2013); BTM was developed specifically for tweets. For all of these methods, we first randomly sample 10k tweets from each event forming our subset S of all tweets T ; then, we create a vocabulary V of word stems that occur at least ten times in at least three events within S (  2000 word stems) and remove all stems from T are not part of V . Sampling is crucial for encouraging event-independent topics given the large disparity among event-level tweet counts (the largest event, Orlando, has 225  more tweets than the smallest event, Burlington). For the embedding-based approach, we: 1. Train GloVe embeddings (Pennington et al., 2014) on V based on 11-50k random samples of tweets from each event. 9 2. Create sentence embeddings e t , 8 t 2 T using Arora et al. (2017)'s method, by computing the weighted average v t of the embeddings of stems within t and removing v t 's projection onto the first principal component of the matrix the rows of which are v t , 8 t 2 S . Stem weights are set to be inversely proportional to their frequencies in S . 3. Jointly cluster the embeddings e t , 8 t 2 S via k-means using cosine distance and assign all tweet embeddings e t , 8 t 2 T to the centroids to which they are closest. We also trained MALLET and BTM on S and used the resulting models to infer topics for all tweets in T , assigning each tweet to its highest probability topic. Henceforth, we use d to mean cosine distance for k-means and probabilities for MALLET and BTM. A manual inspection found that about 25% of the tweets are either difficult to assign to any topic or they represent multiple topics equally. To filter out such tweets, for each tweet we looked at the ratio of d to its closest and second closest topic and removed tweets that have ratios higher than the 75 th percentile (calculated at the model-level). 10 8 http://mallet.cs.umass.edu/topics.php 9 This sample is different from S as it includes more tweets to increase data size, which is important for training the embeddings, where the slightly disproportional representation of events is less problematic. 10 This procedure filters out 11 26% tweets (M= 22% , Figure 4: Topic model evaluations, collapsed across k = 6  10 . Table 1: Our eight topics (with their average proportions proportions across events) and nearest-neighbor stem embeddings to the cluster centroids. Topic names were manually assigned based on inspecting the tweets. To compare the models, we ran two MTurk experiments: a word intrusion task (Chang et al., 2009) and our own, analogically defined tweet intrusion task, with the number of topics k ranging between 6 10 . Turkers were presented with either a set of 6 words (for word intrusion) or a set of 4 tweets (for tweet intrusion), all except one of which was close (in terms of d ) to a randomly cho-sen topic and one that was far from that topic but close to another topic. Then, Turkers were asked to pick the odd one out among the set of words / tweets. More details in Appendix D. We find that our model outperforms the LDA-based methods with respect to both tasks, particularly tweet intrusion  see Figure 4. This suggests that our model both provides more cohesive topics at the word level and more cohesive groupings by topic assignment. The choice of k does not yield a significant difference among model-level accuracies. However, since k = 8 slightly outperforms other k -s in tweet intrusion, we use it for further analysis. See Table 1 for nearest neighbor stems to each topic and Appendix C.2 for example tweets. SD= 4% ) across events, for our model, for eight topics. from Section 3.1 provides a measure of partisanship. The information in a tweet, and thus partisanship, can be decomposed into which topic is discussed, and how it's discussed. To measure within-topic partisanship for a particular event, i.e. how a user discusses a given topic, we re-apply the leave-out estimator. For each topic, we calculate the partisanship using only tweets categorized to that topic. Then, overall within-topic partisanship for the event is the weighted mean of these values, with weights given by the proportion of tweets categorized to each topic within each event. Between-topic partisanship is defined as the expected posterior that an observer with a neutral prior would assign to a user's true party after learning only the topic  but not the words  of a user's tweet. We estimate this value by replacing each tweet with its assigned topic and applying the leave-out estimator to this data. Figure 5 shows that for most events within-topic is higher than between-topic partisanship, suggesting that while topic choice does play a role in phrase partisanship (its values are meaningfully higher than . 5 ), within-topic phrase usage is significantly more polarized. Linear estimates of the relationship between within and between topic partisanship and time show that while within-topic polarization has increased over time, between-topic polarization has remained stable. This finding supports the idea that topic choice and topic-level framing are distinct phenomena. Partisanship also differs by topic, and within days after a given event. Figure 6 shows polarization within topics for 9 days after Las Ve-Figure 6: Las Vegas within-topic polarization in the days after the event. The bar charts show the proportion of each topic in the data at a given time. gas. We find that solidarity has the lowest and shooter's identity & ideology the highest polarization throughout; polarization in most topics increases over time and news has the steepest increase. Similar patterns are present after Orlando (Figure 17 in Appendix I). Measuring polarization of topics for other events over time is noisy, given the sparsity of the data, but overall within-topic polarization is consistent: the most polarized topics on average across events are shooter's identity & ideology ( . 55 ) and laws & policy ( . 54 ), where people are apparently polarized about both why an event happened and what to do about it. Fact-and sympathy-based topics display less polarization: news ( . 51 ), victims & location ( . 52 ), solidarity ( . 52 ) and remembrance ( . 52 ). As shown in Figure 7, investigation , news , and shooter's identity & ideology are more likely to be discussed by Republicans and laws & policy and solidarity more likely to be discussed by Democrats across events. 11 Topics preferred by Republicans seem to relate more to the shooter than to the victims, while topics preferred by Democrats seem to relate more closely to the victims. The shooter's race appears to play a role in topic preference: if the shooter is white, Democrats become more likely to focus on shooter's identity & ideology and laws & policy and Republicans on news and investigation than if the shooter is a person of color. 11 p -values are calculated using a one sample t-test, comparing with zero: shooter's identity & ideology ( p < 0 . 05 ), investigation ( p < 0 . 001 ), laws & policy ( p < 0 . 1 ), news ( p < 0 . 05 ), solidarity ( p < 0 . 001 ). and types of grounding that are used as partisan framing devices, contributing to polarization. Partisan tokens. We estimate the partisanship of tokens via their event-level log odds ratio of Democrats relative to Republicans (based on the vocabularies we create in Section 3.1). We compare these estimates across events. 12 Grounding. We study whether there is polarization in which prior tragic events are referenced in the context of a particular mass shooting. We compile a list of keywords representing major events of mass violence in the US in the past two decades and kept those that were mentioned at least 100 times by Democrat or Republican users. For all tweets for each event in our dataset, we counted the mentions of past context events . For example, in the following tweet posted after Las Vegas: Dozens of preventable deaths should not be the cost of living in America. Stand up to the #NRA. #LasVegasShooting #SandyHook #Charleston, Sandy Hook and Charleston are the context events. Finally, we calculated the partisan log odds ratio of each context event. We focus on the partisanship of the term terrorist and crazy, which exhibit differential pat-12 pat-12 To compare the partisanship of tokens at the eventand topic-level, we also z -score the log odds ratios (Monroe et al., 2008) within events and topics; the most partisan tokens are reported in Appendix F. The reason why we do not z -score for the between-event comparison is because data verbosity disproportionately affects the range of the values' magnitudes. Note that the signs of values  which we focus on for the cross-event comparison  are not affected by z -scoring. terns across events based on the shooter's race. 13 Terrorist is always more likely to be used by Democrats than Republicans in events where the shooter is white, and the opposite is true when the shooter is a person of color (Figure 8); crazy is more likely used by Republicans if the shooter is white than if they are a person of color and the opposite is true (although the pattern is weaker) when a shooter is white. These findings support related work (Perea, 1997; Delgado and Stefancic, 2017) discussing binary conceptualization of race in the US, and its influence in determining whether a shooter's mental health or aspects of their identity are discussed. However, the fact that the influence of race flips completely for Democrats and Republicans is a striking result that calls for further exploration. The partisanship of contextual grounding also corroborates our finding that the shooter's race in-fluences how people conceptualize a certain event. Our results in Figure 9 suggest a few key take-aways: the two most frequently employed context events are both highly partisan (Sandy Hook for Democrats and 9/11 for Republicans); shootings at schools and places of worship are more likely to be brought up by Democrats; Democrats are more likely to reference events with white shooters, while Republicans are more likely to reference those with shooters who are people of color. Affect is intimately tied to ideological reasoning (Redlawsk, 2002; Taber et al., 2009), and so emotional expression represents another semantic layer relevant to polarization (Iyengar et al., 2012; Suhay, 2015). Others have shown that emotion words can help detect political ideology on Twitter (Preotiuc-Pietro et al., 2017) and that emo-13 Note that these words in fact have the largest difference (negative and positive, respectively) if we calculate the differences between the mean z -scores  grouped by the shooter's race  for all tokens in our joint vocabulary. tive political tweets are more likely to be shared (Stieglitz and Dang-Xuan, 2012). Here, we employ a lexicon-based approach to measure valence (positive and negative) and five basic emotion categories (disgust, fear, trust, anger, and sadness). Since word-affect associations are highly domain dependent, we tailored an existing affect lexicon, the NRC Emotion Lexicon (Mohammad and Turney, 2013), to our domain via label propagation (Hamilton et al., 2016). Specifically, we stem all the words in the lexicon and select 8-10 representative stems per emotion category that have an association with that emotion in the context of mass shootings. For each emotion category, we compute pairwise cosine distances between the GloVe embedding of each in-vocabulary stem and the representative stems for that emotion, and include the 30 stems with the lowest mean cosine distances. The resulting lexicons can be found in Appendix E. We use these lexicons to measure the partisanship of each affect category. For each event and each party we aggregate stem frequencies per emotion category. We then calculate the partisan log odds ratio of each category for each event. The log odds ratio of each affect category is shown in Figure 10. These findings suggest that positive sentiment, sadness and trust are more likely to be expressed by Democrats across events, while fear and disgust are more likely to be expressed by Republicans, particularly when the shooter is a person of color. Anger, trust and negative sentiment is similarly likely to be expressed by both parties. 14 Our results about fear and disgust accord with existing literature on emotion and political ideology: conservatives score higher than liberals on subjective measures of fear (e.g. Jost et al., 2017; Federico et al., 2009; Hibbing et al., 2014) and disgust sensitivity is also associated with political conservativism (e.g. Inbar et al., 2009, 2012). Modality is a lexical category concerned with necessity and possibility (Kratzer, 2002; Fintel, 14 p -values are calculated using a one sample t-test, comparing to zero: anger ( p  0 . 43 ), disgust ( p  0 . 06 ), fear ( p < 0 . 001 ), negative ( p  . 2 ), positive ( p < 0 . 001 ), sadness ( p < 0 . 02 ), trust ( p  0 . 07 ). 2006). In the aftermath of a tragic event, people seek solutions, a process that often involves re-flecting on what should have happened or should happen now or in the future (e.g. to prevent such events). We hypothesize that the use of modals in our data gives insight into the kind of (illocution-ary) acts (Austin, 1962) the users are performing via their tweets, such as calling for action, assigning blame, expressing emotions, and stating facts. We work with all forms of the four most frequent necessity modals in our data  should, must, have to and need to . For each, we quantify its partisanship via its partisan log odds ratio. We also annotate a random sample of 200 tweets containing modals to see whether they are indeed used in contexts that imply calls for change / action (e.g. We must have gun control!') and / or to express the user's mental state about the event, such as despair or disbelief (e.g. Why do people have to die?'). Table 2 shows a random sample of tweets containing some form of either should, must, have to , or This roller coaster debate MUST STOP! Sensible gun ownership is one thing but assault weapons massacre innocent lives. The savagery of gore at #Parkland was beyond belief & must be the last. In times of tragedy shouldn't we all come together?! Prayers for those harmed in the #PlannedParenthood shooting. Communities need to step up and address white on white crime like the Las Vegas massacre. White men are out of control. he BLM protest shooting, planned parenthood, now cali... domestic terrorism will crumble this country, SANE PPLHAVE TOFIGHT BACK Shooting cops is horrible, cannot be condoned. But must be understood these incidents are outgrowth of decades of police abuses. #BatonRouge 1. Islamic terrorists are at war with us 2. Gun free zones = kill zones 3. Americans should be allowed to defend themselves #Chattanooga Las Vegas shooting Walmart shooting and now 25 people killed in Texas over 90 people killed Mexico should build that wall to keep the US out CNN reporting 20 dead, 42 injured in Orlando night club shooting. Just awful. The US must act to control guns or this carnage will continue. need to . More collocations, as well as their partisanship, can be found in Appendix H. These examples, as well as our annotation, support the hypothesis that these modals are primarily used to call for action. Of the 200 modal uses,  78% express calls for change/action,  40% express the user's mental state. 15 We also compute the representation p mx of each modal m in each topic x 2 X via ( f m x / P x 0 2 X f mx 0 ) / ( f x / P x 0 2 X f x 0 ) , where f x is the number of tweets from topic x , and f mx the number of those also containing m . We find that that modals are over-represented in the laws & policy topic (see Figure 11). This evidence suggests that calls for policy change  especially gun control, based on annotated samples  are a dominant subset of calls for action. 15 Other uses are primarily epistemic ones (e.g. The suspect must be mentally ill'). must (mean:  . 3 , p < 0 . 001 ), should (mean:  . 18 , p < 0 . 01 ), need to (mean:  . 18 , p < 0 . 01 )  where Democrat and Republican log odds are negative and positive, respectively. 16 A two-tailed t-test shows that only should exhibits statistically significant difference based on the shooter's race ( p < 0 . 03 ), as it is even more likely to be used by Democrats when the shooter is white. To understand whether assigning blame in this domain is a partisan propensity, we also study uses of should have . 17 The log odds of should have (mean:  . 22 , p < 0 . 05 ) show that it is similarly likely to be used by Democrats as should ( p  0 . 8 from two-tailed t-test). Interestingly the log odds ratio of should have , unlike that of should , does not differ significantly based on the shooter's race ( p  0 . 8 from two-tailed t-test). Moreover, we did not find a significant difference in the partisanship of should have nor any other modal based on the administration (Obama or Trump) a shooting took place under, suggesting that Democrats are more likely call for change and assign blame even if their preferred party is in power. We show that inspecting polarization on social media from various angles can shed light on salient phenomena pertinent to group divisions. Applying the leave-out estimator of phrase partisanship to data on mass shootings, we find that reactions to these events are highly polarized politically. To disentangle topic choice and topic-level framing  two phenomena that contribute to polarization  we introduce a tweet-clustering approach. By sampling, requiring words in the vocabulary to appear in multiple events and relying 16 p -values are from one sample t-test, comparing to 0 . on the abstraction of a vector space model, we generate cohesive topic representations that are robust to disparities among event-level vocabularies and tweet counts. Human evaluation shows that our method outperforms LDA-based approaches. Our induced topics suggest that Republicans preferentially discuss topics about the shooter's identity and ideology, investigation and news, while Democrats preferentially discuss solidarity and policy-related topics. We also find that the setting and the shooter's race interact with polarization. For example, Democrats are more likely to contextualize any mass shooting among school shootings and call white shooters terrorists than are Republicans, who in turn are more likely to liken any shooting to other violent events perpetrated by people of color  whom they are more likely to call terrorist than are Democrats. Moreover, Democrats are more likely to frame the shooter as mentally ill when they are a person of color and Republicans when they are white. We also demonstrate that looking at affect and illocutionary force can help us understand users' polarized responses to these tragic events: Republicans are more likely to express fear and disgust than are Democrats, while Democrats are more likely to express sadness and positive sentiment, to make calls for action and assign blame. Polarization is a multi-faceted phenomenon: in this paper we present a set of measures to study these different facets through the lens of language. We show that these measures provide convergent evidence, creating a clearer picture of the complex ideological division permeating public life. Acknowledgements. We thank Jure Leskovec and Adrijan Bradaschia for data, and Cleo Con-doravdi, Chris Potts, Linda Ouyang, David Ritz-woller and Frank Yang for helpful feedback. We are grateful for the support of the Stanford Cy-ber Initiative, the Melvin and Joan Lane Stanford Graduate Fellowship (to D.D.), NSF GRF DGE-114747 (to N.G.), the Michelle and Kevin Douglas Stanford Interdisciplinary Graduate Fellowship (to R.V.), NSF grant CRII 1657155 (to J.Z.), the Stanford Institute for Economic Policy Research and the Knight Foundation (to M.G.) and the Brown University Population Studies and Training Center (to J.S.).
objective objective method method objective abstain objective abstain other result method abstain abstain abstain method abstain abstain abstain abstain method method abstain abstain abstain method method abstain method abstain abstain method other abstain method abstain abstain method abstain method abstain method abstain method abstain abstain method abstain abstain method abstain abstain other method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method abstain method abstain abstain abstain method method abstain method other abstain method abstain abstain method abstain abstain method abstain method abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain method method method method method method method abstain abstain abstain result abstain abstain abstain abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain result result result objective abstain method result abstain result abstain abstain objective method result abstain abstain other