Decisions on state-level policies have a deep effect on many aspects of our everyday life, such as health-care and education access. However, there is little understanding of how these policies and decisions are being formed in the legislative process. We take a data-driven approach by decoding the impact of legislation on relevant stakeholders (e.g., teachers in education bills) to understand legislators' decision-making process and votes. We build a new dataset for multiple US states that interconnects multiple sources of data including bills, stakeholders, legislators, and money donors. Next, we develop a textual graph-based model to embed and analyze state bills. Our model predicts winners/losers of bills and then utilizes them to better determine the legislative body's vote breakdown according to demographic/ideological criteria, e.g., gender. State-level legislation is the cornerstone of national policies and has long-lasting effects on residents of US states. Thus, decoding the processes that shape state bills is crucial yet involved. State legislatures vote on 23 times more bills than Federal legislatures, exceeding 120K bills per year (King, 2019). In addition, these state bills cover a broader range of local problems as each state possesses lawmaking power effective within its boundaries. E.g., the State of Washington Health Care Committee addresses health service issues including licensing and regulation of health care facilities and providers. Moreover, it regulates pharmacies, pharmaceutical drugs, state public health programs, and private/public insurance markets (House, 2021). We argue that recent NLP architectures can provide new insights into the state-level legislative efforts. In particular, contextualized graph and text embedding can better represent policies within and across states via a shared political context. However, most of the prior efforts are focused on analyzing congressional bills with traditional techniques, e.g., (Gerrish and Blei, 2011, 2012). A few state-level studies (Eidelman et al., 2018; Davoodi et al., 2020) took great steps in predicting the progression of state bills towards a vote on the floor and the breakdown of votes based on demographic metrics (e.g., gender). But their main downside is they evaluate policies in a limited context and do not capture cross-state patterns. Medical malpractice actions. Permits a patient to bring an action against a health care provider without submitting the complaint to the medical review board if: (1) the amount of the claim is not more than $15,000; (2) the cause of action is based on the removal of the wrong body part Cleavage Yea Nay Figure 1: A health bill leading to voting cleavage based on the party metric primarily due to its specific losers (red) and winners (green). Winners-Losers analysis . In this work, we take a new data-driven approach to analyzing state legislation. Our key insight is that each state bill inevitably produces some winners and losers to provide practical solutions to specific in-state and local problems. Thus, we argue that it is important to examine state bills in the larger context of their impact on different population segments as well as commercial and professional stakeholders. To help clarify this idea, consider the example in Figure 1. This state bill makes it easier for patients ( winners) to take legal actions against healthcare providers ( losers) . This analysis of winners and losers (WLs) can foster transparency in legislative efforts in each state, while interconnecting different states through common stakeholders and revealing cross-state patterns. In addition, the context of WLs can enable a new category of NLP models for predicting the roll-call behavior of legislators. based on the ideological and demographic identities of legislators (Section 2). Each of such metrics (e.g., party, gender, district, ideology) splits legislators into groups. Measuring lack of consensus within and across these groups, which has political and social benefits, can be done using two classification tasks (Section 5): For a given metric, we say a bill is competitive (Figure 2) if the majority vote of legislators from a group (e.g., Democrat, male, urban, liberal) is different from that of the opposite group (e.g., Republican, female, rural, conserva-tive). Similarly, a bill is inverse-competitive if there is a tie in votes of members of the same group (e.g., liberals). For instance, the health bill in Figure 1 resulted in a party competitive vote. Another example is a state bill on abortion that requires... physician performing an abortion to admitting privileges at a hospital in the county resulted in a gender competitive vote. We show the context of winners/losers of these bills could hint at such cleavages prior to voting (Sections 4, 6). Framework overview. To achieve this goal, we address multiple NLP challenges in our proposed framework: (1) Data: The legislative process in US states does not track the stakeholders of bills and the impact of bills on them. Thus, we design a reliable crowd-sourcing pipeline to extract and analyze winners and losers of state bills from their text and form a new annotated dataset. (2) Modeling: To automate the WL analysis, next, we provide a nationwide graph abstraction to model the state legislative process, as well as a joint text and graph embedding architecture for predicting winners and losers. Our model captures the interplay of different entities, e.g., bills, stakeholders, legislators, and money donors, while maintaining dependencies between their textual attributes. We leverage RGCN (Schlichtkrull et al., 2018), a relational graph convolutional network, to represent diverse relations. We also adopt the RoBERTa transformer (Liu et al., 2019) after performing domain-adaptive pretraining on political texts using the MLM (Masked Language Model) task. (3) Application: Finally, we showcase the ability of our WL analysis and prediction model in decoding the voting behavior of state legislators. In summary, we make three technical contributions :  We provide the first definition and realization of winners/losers analysis for state bills using the latest NLP advances. (Sections 2, 3, 4). We developed a new joint graph and text embedding model that both predicts win-ners/losers of bills and legislators' votes. In particular, it incorporates the winners/losers inference into the vote prediction task, to evaluate bills in a broader context (Section 5). We operationalized the winners/losers analysis for several legislative topics (e.g., health) and created a new dataset. The extensive evaluation shows our approach delivers a higher F1, than existing models (Sections 3, 6) 2 Related Works Our work is inspired by some promising studies: Roll-call classification. Eidelman et al. 2018 associate the bill's text with partisan information of its sponsors to predict the likelihood of a mem-ber of the U.S. Congress voting in support of a bill. Similarly, Gerrish and Blei 2011 embed the combined topic and text of Congress bills in the ideological space and develop ideal point models for inferring votes. Peng et al., 2016; Kornilova et al., 2018; Kraft et al., 2016; Patil et al., 2019; Karimi et al., 2019; Pujari and Goldwasser, 2021 augment this model using data on social networks, thus generating better embeddings. Bill text classification. Instead of leveraging bill text in models to describe the behavior of each legislator, Yano et al. 2012 include the bill's text in a model that directly predicts whether a bill comes out from a standing committee. Particularly, they develop features based on the urgency of the problem being solved by the bill and the set of legislators co-sponsoring the bill. Eidelman et al. 2018 conduct a similar study on US states. Winners-losers analysis. Analyzing the impact of bills on its stakeholders is a well-studied topic in the political science literature. Gamm and Kousser, 2010 reveal state legislators are more likely to write bills aimed at a particular local stakeholder when the legislative body is dominated by one party. Similarly, Bagashka and Clark, 2016 show state legislators are motivated to introduce particularistic bills designed to help a specific geographical area within 271 their district. Pennock, 1979 analyzes legislation based on its generalized and particularized impact on different interest groups. By leveraging recent NLP advances (e.g., contextualized language models, graph embedding, crowdsourcing), our work extends these studies and provides the first automated framework for the stakeholders analysis on state bills. Voting cleavages. Research has covered multiple ways that the demographic background of legislators can affect roll-call voting. Frederick 2010 demonstrates gender affects the roll-call vote in the Senate by changing the influence of partisanship for GOP women. Broach 1972 describes that urban-rural voting cleavages happen in less partisan states and on bills that separate urban and rural interests. Similar to us, Davoodi et al. 2020 build a textual graph to predict such cleavages. While our focus is on a different problem, stakeholders analysis, we outperform this prior study by representing bills in a broader context containing their stakeholders. Graph embedding in NLP. Our work uses Graph convolutional networks (GCNs), which have been applied to various NLP tasks, e.g., Semantic role labeling (SRL) (Marcheggiani and Titov, 2017) and relation classification in clinical narratives (Li et al., 2018). In these tasks, GCNs encode the syntactic structure of sentences. Similarly, Deffer-rard et al., 2016; Peng et al., 2018; Henaff et al., 2015 use graph neural networks (GNNs) to represent a network of documents based on their references. Similar to our work but for a different problem and objective, Sawhney et al., 2020 analyze speech-level stance of members of the parliament, by performing node classification on graph attention networks (GATs), and Pujari and Goldwasser, 2021 analyze social media content generated by politicians using a graph transformer model. We first provide an overview of key players in the state-level legislative process. Then, we model them using an efficient text-based graph abstraction (Figure 3), which will enable us to embed and evaluate state policies in a broad context and perform the stakeholder and roll-call analysis on them. Our model, unlike prior works, fully captures the interplay of main players in the lawmaking process: sists of two chambers 1 : the House and the Senate. The legislative process starts with legislators sponsoring a bill in a chamber. The idea of a bill can come from different sources. Next, the bill goes through multiple roll-call votes in the origin chamber, where it can fail at any stage. It is first referred to the proper committee by the chamber leader. Committee members, before casting their votes, may set up a public hearing with the sponsors and interested parties. If the bill passes out of the committee, it reaches the second reading, where the full chamber debates, amends, and votes on the bill. If the bill passes by a majority vote, it is scheduled for the third reading and final vote. A bill must go through a similar procedure in the other chamber before it is acted on by the governor. 2. Contributors . While legislators navigate through bills, external contributors influence their decisions. Individual and corporate money donors aim at developing changes in the outcome and theme of bills starting from the election times. Lobbyists launch campaigns to persuade legislators towards certain policies. Such efforts inevitably lead to new bills or amendments to existing laws. 3. Stakeholders . A state bill cannot benefit everyone and it produces beneficial or detrimental effects on its stakeholders. Identifying winners and losers of a bill from its text is crucial, which can hint at the fate of a bill. Particularly, legislators do not always write bills themselves. Corporations and interest groups (e.g., ALEC) sell fill-in-the-blank bills to legislators. Thus, we can see voting patterns on bills with the same winners and losers. 1 Nebraska's legislature is unique in the nation because it has a single-house system. To model these players and their interactions, we design a legislative graph with three important properties (Figure 3). First, since each of the players (e.g., stakeholders, legislators) has different textual attributes, our proposed graph supports heterogeneous textual nodes. Second, we form a nationwide graph to capture cross-state patterns (abla-tion study in Appendix A.2) by building common entities (e.g., stakeholders in Section 4). Finally, our abstraction supports multiple relations between each pair of entities (e.g., legislators voting and sponsoring a bill). With this overview, we present the nodes and relations that we will realize based on the real data: Node types . The nodes in the legislative graph contain a rich set of textual features: (1) Bill nodes embed title, abstract, and body of state bills. (2) Stakeholder nodes come with short texts on political interests and constituent entities of stakeholders of policies in bills (will be detailed shortly). (3) Legislator nodes contain diverse textual information on legislators, e.g., their biography, political interests, committee assignments, and demographic profile (e.g., party, gender, ideology, and district). (4) Contributors nodes have text-based attributes on money donors covering their specific/general business interests, party, and their type (individual or non-individual). Relation types . Based on the legislative process, legislator and bill nodes participate in Bill Sponsorship , 'No Vote , and Yes' Vote relations in the graph (See Appendix A.4 for handling abstain votes.) A stakeholder node forms Winner , Loser , or Neutral relations with a bill node, which we will extract it based on the bill text. Similarly, we form two types of relations between contributors and legislators: Positive Donation realized based on the real donation data, and Negative Donation , which we infer when a contributor shows a lack of interest in a demographic of legislators (e.g., never donates to women). We sample and connect such legislators and the contributor via a negative relation. Next, we describe how we build up the legislative graph, by collecting data on legislators, bills, and contributors. US states do not record the impact of bills on relevant stakeholders. Thus, we explain how to derive stakeholders from bill nodes, perform winners-losers analysis on them, and interconnect different US states by forming common stakeholder nodes. We highlight how our analysis can be used (1) to inform the public about the dynamic and direction of state policies, and (2) to determine legislators' roll-call behavior with different demographic and ideological profiles. Topic Education Health Law Agriculture # of bills 957 942 1140 758 Table 2: Bills sampled for the stakeholders analysis. Bills. From the LegiScan website (LegiScan, 2019), we collected data on bills introduced in Indiana, Oregon, and Wisconsin from 2011 through 2018 (details in Appendix 7). We developed a crawler that uses the LegiScan API to fetch legislative information on every bill, including: (1) bill metadata, e.g., the bill type, title, description, sponsors, and links to its texts; (2) vote metadata, e.g., legislator's roll-call vote; and (3) legislator metadata, e.g., party and district info. Then, our crawler converts bill texts in PDF format to text files. In total, we collected  35k bills and sampled 58% of them that had both roll-call data and full texts. Our focus is on the 2nd/3rd reading, in which the full chambers vote, so we selected 32% of the bills for building the legislative graph (Table 1). In LegiScan, each bill is associated with a main topic (e.g., health), used for referral to a proper committee. For the four most frequent topics (Table 2), we will define a group of generic stakeholders for the winners-losers analysis. Legislators. Our crawler also used Ballotpedia (Ballotpedia, 2019) to collect text information on each legislator's biography, political interests, and committee assignments. Also, it consumed other publicly available datasets to identify a legislator's demographic profile, e.g., ideology, gender, and district. The ideology scores for legislators (Shor and McCarty, 2011) were grouped into conservatives, moderates, and liberals. The district identifier was combined with GIS census data (Census, 2019) to identify each legislator as representing an urban or 273 State Gender Party Geography Ideology F M D R UR RU C M L IN 50 176 67 159 161 64 125 94 7 OR 47 103 83 67 133 17 28 61 61 WI 51 157 84 124 160 48 78 49 81 All 148 436 234 350 454 129 231 204 149 Table 3: Aggregated legislators' attributes UR: Urban, RU: Rural, C: Conservative, M: Moderate, L: Liberal. Topic Stakeholders W (%) L (%) E du c a t i o n Edu. companies & service providers 1.4 1 Educational institutions and schools 23.9 8.7 State education agencies 6.3 8.6 Teachers and education workers 13.2 1.3 Students 34.2 1.6 A g r i c u l t u re Agriculture and food-related companies 4.5 4.1 Agricultural and food producers 24.4 6.9 End consumers or retail customers 11.6 11.2 State agriculture and food agencies 14.5 1.4 Grocery stores or food providers 11.6 9.8 H e a l t h Healthcare facilities 16.7 7.7 Healthcare providers and professionals 6.8 3.3 Insurance providers and companies 11.4 10.5 Patients and insurance owners 16.7 6.3 Pharma and medical device companies 4.6 0.5 State healthcare agencies 11.7 4 L a w Law enforcement agencies and officers 15.7 24.7 Judges 11.5 9.4 Victims, offenders, suspects 9.9 11.2 Lawyers 9.8 7.7 Table 4: Stakeholders of different bill topics and their frequency distribution as winners (W) and losers (L). Contributors : FollowTheMoney (FollowThe-Money, 2019) records donations to state legislators and candidates. Our crawler collected the information of donors for each legislator in our dataset (See Table 1). This includes multiple textual attributes for each contributor: type (individual or non-individual), general party, and economic and business information. While the contributor data can be utilized in more sophisticated ways, we focused on major contributors by setting a minimum donation threshold and pruning donors who contributed to a single legislator; We set the fraction of negative donations (Section 3) to 30% of the positive ones extracted from the data. For each select bill topic, we (authors) randomly sampled 10% of bills and carefully analyzed their texts. We recorded entities discussed in the bill texts as well as the detrimental or beneficial impact of the suggested policies on them (regardless of the legislative outcome, i.e., passed in a vote or not). To interconnect different states and optimize the legislative graph, we deduplicated entities and clustered those whose interests align (e.g., surgeons, doctors, dentists, and etc.) into generic ones (e.g., healthcare providers). Table 4 shows the final list of stakeholders for the select topics. With detailed annotation guidelines, we leveraged Amazon MTurk for labeling  4k bill texts from these topics (Table 2), where 3-5 workers identified the effect of the suggested policies in each bill on the relevant stakeholders. As will be detailed in Appendix A.1, we ensured the accuracy of the labeled data is 90%+. Based on the outcomes of the previous two steps, we formed a legislative graph for our target states. We briefly provide two results from the winners-losers analysis on the graph to highlight its importance. First, we show the frequency distribution of the stakeholders as a winner vs. a loser for each topic in Table 4, which would inform the public about the dynamics and directions of state-level policies. E.g., under the education topic, students were the largest winners, while educational institutions were the major losers. For law bills, law enforcement agencies were the top losers given the recent nationwide focus on police use of force. Also, our winners-losers analysis captures the policy preferences of different ideological and demographic groups of legislators. For example, Democrats are more likely to support legislation benefiting teachers, compared to Republicans (GOP). This fact is also reflected in our models predicting voting cleavages in Section 6 (e.g., our naive model, WL-Correlation, outperforming other models in its category). Here, to motivate the need for building such models, we are interested in 274 measuring the rate of 'Yes' votes from each demographic and ideological group of legislators on bills of a given topic, where a stakeholder is a loser and a winner. E.g., on education bills benefiting a stakeholder (e.g., Students) as a winner, we compute, A = [ # of yes votes ] / [ total # of votes ] in the GOP legislators. Similarly, on educations bills, where this stakeholder is a loser, we calculate B = [ # of yes votes ] / [ total # of votes ] for GOP. We then report the difference, A-B, in Table 5, where a large positive value indicates the stakeholder is being advantaged by the respective group of legislators. E.g., we see GOP has significantly more Yes votes when students are winners, compared to Yes votes when students are losers. By running queries on the legislative graph containing all players (e.g., donors), we were able to see the voting behavior of GOP could be motivated by major donations to this party from corporations representing students (e.g., School Choice). The stakeholder analysis based on human data annotation is expensive and time-consuming. To automate the analysis and better leverage its results in other applications, we build up a contextualized embedding architecture and define two classification tasks on the legislative graph: 5.1 Classification Tasks on Legislative Graph Task 1: winners-losers prediction. Our first task is to predict the relation between a bill node and each relevant stakeholder node (based on its topic in Table 4). Such predicted relations will bring valuable insights into the bills, while also clarifying legislators' roll-call behavior (Section 6). Thus, we define the next task to showcase these benefits. Task 2: bill cleavage and survival prediction. For a bill, we predict if (1) it shows identifiable voting cleavages and (2) it can advance by getting a pass. We achieve these by predicting and aggregating roll-call relations (between legislators and bills) in the graph. In particular, we assign 9 labels to each bill: (1) Competitive labels : For voting cleavages, we split legislators into groups based on their demographic and ideological pro-files (party, gender, ideology, and the urban/rural nature of their district as defined in Section 4). For an attribute (e.g., gender), we say a voting round is competitive if the majority of legislators from one group (e.g., Women) and the majority of the opposite group (e.g., Men) cast different votes (Figure 2a). (2) Inverse-competitive labels : Similarly, for an attribute (e.g., gender), a voting round is inverse-competitive if we identify a partial or complete tie (Appendix A.4) in the vote of legislators of the same group (e.g., Men in Figure 2b). (3) Survival label : Finally, a bill passes its current voting round by getting a majority vote. At a high-level, we propose a unified model to jointly embed and classify both roll-call and winner-loser relations in the legislative graph (Fig-ure 4a): (a) We first train our model to predict relations between bill nodes and their stakeholders. One can use the result of this stage for further analysis of state policies (e.g., Section ). (b) Our key insight is that knowing winner-loser relations enhances the embedding of nodes in the legislative graph. Thus, we conduct inference on bills that lack such relations (if any) using the pretrained model from step (a) and add these predicted relations to the graph. (c) Next, continue training on the updated graph to fine-tune the model for the roll-call (vote) prediction task. Finally, we aggregate the predicted votes for the bill cleavages/survival analysis. In all these steps, our model generates and jointly optimizes both text and graph embeddings for each node, and consumes them to classify the two types of relations. Thanks to jointly optimizing 275 the tasks over the textual and graph information, our architecture outperforms existing models (Sec-tion 6). Hereafter, we detail the layers in our model using a bottom-up approach: 5.3 Contextualized Text Embedding Layers The lower half of our model generates a contextual embedding for textual attributes of nodes in the legislative graph. We leverage the RoBERTa architecture (Liu et al., 2019). For improved performance, one of our contributions is that we will pretrain RoBERTa on unlabeled bill texts using the MLM task (Section 6). In more detail, for each bill node, we feed three pieces of textual information to RoBERta: title, abstract, and body. RoBERTa does not support input sequences longer than 512 tokens. Thus, we take the representation of each of these components separately (the embedding of their [CLS] token) and do average pooling to output the final representation of the bill. Similarly, the text embedding of stakeholder , legislator , and contributor nodes are the average of the vectors representing their key textual attributes (Section 4). On top of the text embedding layer, we place a Relational Graph Convolutional Network (RGCN) to create a graph embedding for each node. The RGCN uses the text embedding of each node to initialize its graph representation. In parallel, we build a feed-forward neural network (FFNN), taking the text embeddings of nodes to a concatenation layer for our joint text-graph optimization. The (non-relational) GCN has multiple layers and each layer performs two operations: propagation and aggregation . In the propagation, nodes update their neighbors by sharing their features or hidden states. In the aggregation, each node adds up the messages coming to it to update its representation. In GCN, at layer l + 1 , the hidden representation of node i , h il +1 , with neighbours N i is: h il +1 =  (cid:32) (cid:88) j  N i 1 c i W l h lj (cid:33) (1) GCN uses the same weight matrix in each layer, W l , and normalization factor, c i = | N i | , for all relation types in a graph. We choose RGCN as it uses unique parameters for each relation type, thus better handling our multi-relational graph. In RGCN, the embedding of node i in layer ( l + 1) is: h il +1 =  (cid:32) W l 0 h li + (cid:88) r  R (cid:88) j  N ri 1 c i,r W lr h lj (cid:33) (2) A 3-layer RGCN turns out to be sufficient in our case to capture the 3rd order relations between contributors and stakeholder nodes. By combining the outputs of the RGCN and FFNN, we train a relation classification layer by using the DistMult scoring function (Schlichtkrull et al., 2018; Yang et al., 2014). For each relation ( s, r, d ) being predicted, this layer computes f ( s, r, d ) = e Ts W r e d . e s and e r are the joint text and graph embeddings of the source and destination nodes and w r is a diagonal relational weight matrix. Our loss function is L = LCLS + L Text + L Graph enabling us to jointly optimize the text and graph embeddings as well as the relation prediction. LCLS is the cross-entropy loss of the relation classification; L Graph and L Text are the L2 regularization of RGCN's and FFNN's weights, optimizing the graph and text representations, respectively. We first evaluate the efficiency of our legislative graph abstraction and text+graph embedding model in the winners-losers prediction. Then, we show the benefits of our combined inference of stakeholders and roll-calls in decoding state bills. Data Split and metric. We split the legislative graph (Formed in Section 4) based on bill nodes. We randomly select 20% of the bills for testing and keep the rest for training and validation. We study three settings in terms of the winners-losers (stakeholders) information in the graph: (a) Unknown winners-losers relations. (b) Known relations based on our human-labeled annotation. (c) Predicted: 30% of bills in the train graph come with such relations and we predict them for the rest of bills. In Appendix A.3, we will report the results of stateand time-based splits. Finally, given our data is highly skewed, we choose Macro F1 as the main metric over accuracy. Settings/parameters. We build our joint model (Figure 4) on top of PyTorch, DGL, and spaCy. We set the initial embedding dimension in RoBERTa and RGCN to 1024. The FFNN and RGCN take 276 the embeddings to a 256-dimensional space. We also used Adam optimizer, and for each observed relation (Table 1), we sampled a negative example. We devise robust baselines for both of our tasks: 1. Text-based models. We build a logistic regression (LR) classifier that takes the text embedding of a bill and predicts if it shows a certain cleavage or passes/fails. A similar classifier takes the text embeddings of a bill and a stakeholder to classify their relation. We evaluate three embedding architectures: (a) BoW , where unigram and bigram features (top 5K highest-scoring) are used to represent textual information. (b) RoBERTa (Liu et al., 2019). (c) Pretrained RoBERTa that we adapted its domain by applying MLM on 10K unlabeled state bills (39K sentences) (Gururangan et al., 2020). We study two additional variations of these models (only for winners-losers prediction due to limited space): Sponsors , where the bill sponsors are represented using a one-hot vector and concatenated to the bill text representation. Roll-Call , where we concatenate a vector containing cleav-age/survival info. of each bill to its text embedding. 2. Graph-based models : We build a relation classifier over edge embeddings, generated by three widely-used graph models, to predict roll-call and winner-loser relations (for the bill cleavages/survival task, we aggregate votes): (a) DeepWalk (Perozzi et al., 2014) that generates embeddings for nodes and edges by running Skip-Gram on random walks on the graph. (b) GCN (Kipf and Welling, 2016) is a basic 3-layer GCN model with random node features in its first layer. (c) RGCN (Schlichtkrull et al., 2018) is the relational GCN handling different relation types in the legislative graph. 3. Naive models . We evaluate three naive classi-fiers: (a) Majority : A baseline predicting the most frequent class in the train data. (b) Sponsor : An LR classifier that uses the one-hot embedding of bill sponsors to determine bill survival/cleavages (simi-larly winner/loser relations). (c) WL-Correlation (solely for the survival/cleavage task) predicts a legislator's vote on a test bill with known win-ners/losers based on his historical votes on train bills with the same winners/losers. We compare these models in predicting relations between bills and their relevant stakeholders (Table 6). winner-loser relations between bills and stakeholders. (1) In the vanilla text-based category , RoBERTa shows 2.9 higher F1 than BoW. Our pretrained RoBERTa generates more efficient contextual embedding for text information of bills and stakeholders (e.g., summary), and thus better determines the impact of a bill on its stakeholders. Including the sponsors' info in the pretrained RoBERTa leads to the best text model. (2) In the graph-based models , Deepwalk/GCN exhibits a sharp drop in F1, by ignoring the heterogeneity of relations in the graph and thus producing inefficient representations for them. RGCN overcomes this issue and approaches the best text model with F1 of 63.9. (3) Our joint text-graph model combines the strengths of the graph and text models and delivers 3.3 points higher F1. Next, we focus on the performance of different models in determining voting cleavages/survival, with Unknown , Known , Predicted winners-losers in the legislative graph. In Table 7, we report the results for the bill survival and party-based voting cleavages (results for the other cleavages in Appendix, Table 11). We can make a few observations: First, our stakeholder analysis helps all models to better decode state policies, when comparing the same model in the Unknown and Known winners-losers settings: (1) In the text-based models , prediction on the textual information of both bills and known winners-losers delivers a higher F1 than only on the text of bills (e.g., Pretrained RoBERTa model gets a 5.4% boost in F1 in predicting party competitive bills). Similarly, (2) In the graph-based models : RGCN overcomes 277 Models Pass/Fail demographic voting cleavages in Table 11. the limitations of Deepwalk in handling heterogeneous relation types (winner-loser vs. roll-call) and delivers consistent gains in the setting Known . (3) Our model has the best performance due to generating and optimizing a joint graph and text representation for legislators, bills, money donors, and stakeholders in the setting with known winners and losers. Second, by focusing on the models with the Predicted winners-losers information, we observe: (5) Our model still beats the other baselines, due to our unified model for roll-call and winner-loser training as well as our text-based legislative graph abstraction (Section 5). Of course, there is an expected drop in F1 across different models including ours, when we consume predicted winner-loser relations instead of human-labeled ones. This drop could be tolerable in most cases, thus not hindering the automation of our stakeholder analysis and leveraging its results in downstream vote analysis tasks (ethical considerations in Section 7). We took a new data-driven approach to analyze state legislation in the US. We showed that identifying the winners/losers of state bills can (1) inform the public on the directions of state policies, and (2) build a nationwide context for a better understanding of legislators' roll-call behaviors. Thus, we proposed a text-based graph abstraction to model the interplay of key players in the state legislative process, e.g., bills, stakeholders, legislators, and donors to legislators' campaigns. Next, to automate our analysis, we developed a shared text and graph embedding architecture to jointly predict winners/losers of bills and legislators' votes on them. We created a new dataset using different data sources and human annotation and evaluated the strength of our architecture against existing models. We hope this work will provide a starting point for further studies examining the impact of policy decisions on individuals and groups, an important step towards making the democratic process more transparent. Analyzing state legislation is a sensitive task, where unexpected results of research and deployed ML systems can create misguided beliefs on the government policies on important topics (e.g., health, education). Thus, we would like to discuss some ethical aspects related to our work in terms of data and model (considering potential scenarios suggested by Chandrabose et al., 2021): 1. Selection of data sources . While there can be different inherent imbalances in the state legislature (e.g., gender and party distribution), we were not able to identify that our data sources adding systematic political and social biases to our study, e.g., towards demographic populations of legislators. All our data sources (e.g., LegiScan and FollowTheMoney) are publicly available and have been used by the political science community over the years. LegiScan (LegiScan, 2019) is a nonpartisan and impartial legislative tracking and reporting service for state bills. FollowTheMoney (Fol-lowTheMoney, 2019) is a nonpartisan, nonprofit organization revealing the influence of campaign money on state-level elections and public policy in all US states. Finally, Ballotpedia (Ballotpedia, 2019) is a nonpartisan, nonprofit organization providing a brief introduction, biography, committee assignment, and general information on legislators across different years. Our study combined these data sources for analyzing state bills in a broad context, thus contributing to reduced data bias for all models evaluated in this paper. 2. Selection of states . In addition, to help mitigate the risk of data collection bias or topic preference that can be introduced through the choice of specific state legislatures, we randomly picked a red, a blue, and a purple state (indicating 278 a significant majority for Republicans, Democrats or more balanced state legislature, respectively). There were some restrictions in terms of collecting the data from the above sources (e.g., FollowTheMoney and Ballotpedia). These data sources and services often limit the number of API calls and queries for retrieving the data for educational institutions. Besides this, annotating the data through Amazon MTurk was expensive for us so we conducted our study on four highly discussed topics in state bills (i.e., health, education, agriculture, and law). We will explore ways of expanding our dataset to more states and topics over time. 3. Disguised winners and losers . In theory, the authors of state bills (e.g., interest groups selling fill-in-the-blank bills to legislators) may try to reframe bills (disguise winners or losers) to further their political aims. At the first glance, this could pose a challenge to our bill annotation, dataset, and stakeholder analysis. As described in Section 3, the state legislative process has a multi-stage reviewing process in two chambers (e.g., first reading, second reading, and third reading). Thus, we have observed that it is hard to hide the impact of bills on their relevant stakeholders from our qualified annotators, i.e., the authors and multiple vetted MTurk workers for each example, in practice. In addition, our work on MTurk maps the impact of policies suggested by bills to winners and losers. Thus, it already considers those stakeholders that are not mentioned in the text explicitly (More details in Appendix A.1). 4. Winners and losers analysis . The analysis, aligning demographic cleavages with winners and losers preferences, is done at an aggregate level based on the data we annotated. These preferences could be influenced by other factors beyond demographics. Deriving conclusions from this analysis could require longitudinal studies, capturing the change of these patterns over time, for example when analyzing policies intended to help correct inequities towards marginalized groups. Our goal is to provide a tool for domain experts that would point at nuanced, stakeholder specific, legislative preferences that can be studied further in order to determine their significance. 5. Handling abstain votes . There are abstain (absent and N/A) votes in our dataset. However, we did not include them in our study due to their extremely low frequency (for our proposed model and other baseline models). We leave this evaluation as a future work. 6. Handling other countries and languages. While our dataset is specific to the US, the the problem we studied, stakeholder analysis, can be generalized to legislation from other countries and in different languages. Although we have not evaluated such bills (due to lack of data sources), we expect such legislation to produce winners or losers to provide practical solutions to their local problems. In particular, our framework offers a multi-relational graph abstraction and prediction models to analyze stakeholders of bills (winners/losers) and the voting behavior of legislators. These techniques can support non-US national and state-level legislative processes. To accommodate other languages, one could adopt cross-lingual embedding models, e.g., XLM-R (Conneau et al., 2019) instead of RoBERTa, in our architecture. We would like to acknowledge the members of the PurdueNLP lab. We also thank the reviewers for their constructive feedback. The funding for the use of mTurk was part of the Purdue University Integrative Data Science Initiative: Data Science for Ethics, Society, and Policy Focus Area. This work was partially supported by an NSF CAREER award IIS-2048001.
abstain abstain method objective objective method abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain other abstain objective method abstain abstain abstain abstain abstain abstain method abstain abstain abstain result abstain objective objective method abstain abstain method abstain objective objective objective objective objective result abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain objective abstain abstain abstain abstain method result abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain method abstain abstain abstain abstain abstain abstain abstain method method abstain method abstain abstain abstain method abstain abstain method method abstain method abstain method abstain abstain other abstain other abstain abstain abstain abstain method abstain method abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain method abstain abstain abstain abstain abstain method abstain abstain method method abstain method abstain abstain abstain abstain method abstain method abstain method abstain abstain abstain other method other method abstain abstain method abstain abstain result abstain method method method abstain method method method method abstain method method abstain method abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result other abstain abstain abstain abstain result abstain result objective result objective objective objective method abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain method other other other other