Trust is implicit in many online text conversationsstriking up new friendships, or asking for tech support. But trust can be betrayed through deception. We study the language and dynamics of deception in the negotiation-based game Diplomacy, where seven players compete for world domination by forging and breaking alliances with each other. Our study with players from the Diplomacy community gathers 17,289 messages annotated by the sender for their intended truthfulness and by the receiver for their perceived truthfulness. Unlike existing datasets, this captures deception in long-lasting relationships, where the interlocutors strategically combine truth with lies to advance objectives. A model that uses power dynamics and conversational contexts can predict when a lie occurs nearly as well as human players. A functioning society is impossible without trust. In online text interactions, users are typically trusting (Shneiderman, 2000), but this trust can be betrayed through false identities on dating sites (Toma and Hancock, 2012), spearphishing attacks (Dhamija et al., 2006), sockpuppetry (Ku-mar et al., 2017) and, more broadly, disinformation campaigns (Kumar and Shah, 2018). Beyond such one-off antisocial acts directed at strangers, deception can also occur in sustained relationships, where it can be strategically combined with truthfulness to advance a long-term objective (Cornwell and Lundgren, 2001; Kaplar and Gordon, 2004). We introduce a dataset to study the strategic use of deception in long-lasting relationships. To collect reliable ground truth in this complex scenario, we design an interface for players to naturally generate and annotate conversational data while playing a negotiation-based game called Diplomacy. These annotations are done in real-time as the players send and receive messages. While this game setup might not directly translate to real-world situations, it enables computational frameworks for studying deception in a complex social context while avoiding privacy issues. After providing background on the game of Diplomacy and our intended deception annotations (Section 2), we discuss our study (Section 3). To probe the value of the resulting dataset, we develop lie prediction models (Section 4) and analyze their results (Section 5). The Diplomacy board game places a player in the role of one of seven European powers on the eve of World War I. The goal is to conquer a simplified map of Europe by ordering armies in the field against rivals. Victory points determine the success of a player and allow them to build additional armies; the player who can gain and maintain the highest number of points wins. 1 The mechanics of the game are simple and deterministic: armies, represented as figures on a given territory, can only move to adjacent spots and the side with the most armies always wins in a disputed move. The game movements become publicly available to all players after the end of a turn. Because the game is deterministic and everyone begins with an equal amount of armies, a player cannot win the game without forming alliances with other playershence the name of the game: Diplomacy. Conquering neighboring territories depends on support from another player's armies. After an alliance has outlived its usefulness, a player often dramatically breaks it to take advantage of their erstwhile ally's vulnerability. Table 1 shows the end of one such relationship. As in real life, to succeed a betrayal must be a surprise to the victim. Thus, players pride themselves on being able to lie and detect lies. Our study uses their skill and passion to build a dataset of deception created by battle-hardened diplomats. Senders annotate whether each message they write is an ACTUALLIE and recipients annotate whether each message received is a SUSPECTED LIE . Further details on the annotation process are in Section 3.1. Figure 1 shows the raw counts of one game in our dataset. But numbers do not tell the whole story. We analyze this case study using rhetorical tactics (Cialdini and Goldstein, 2004), which Oliveira et al. (2017) use to dissect spear phishing e-mails and Anand et al. (2011) apply to persuasive blogs. Mentions of tactics are in italic (e.g., authority ); context for quotes in Appendix, Table 7. For the rest of the paper, we will refer to players via the name of their assigned country. 1 In the parlance of Diplomacy games, points are supply centers in specific territories (e.g., London). Having more supply centers allows a player to build more armies and win the game by capturing more than half of the 34 supply centers on the board. Through two lie-intense strategiesconvincing England to betray Germany and convincing all remaining countries to agree to a drawItaly gains control of the board. Italy's first deception is a plan with Austria to dismantle Turkey. Turkey believes Italy's initial assurance of non-aggression in 1901. Italy begins by excusing his initial silence due to a rough day at work, evoking empathy and likability . While they do not fall for subsequent lies, Turkey's initial gullibility cements Italy's first-strike advantage. Meanwhile, Italy proposes a long-term alliance with England against France, packaging several small truths with a big lie. The strategy succeeds, eliminating Italy's greatest threat. Local threats eliminated, Italy turns to rivals on the other end of the map. Italy persuades England to double-cross its long-time ally Germany in a moment of scarcity : if you do not act now, there will be nowhere to expand. England accepts help from ascendant Italy, expecting reciprocity . However, Italy aggressively and successfully moves against England. The last year features a meta-game deception. After Italy becomes too powerful to contain, the remaining four players team up. Ingeniously, Italy feigns acquiescence to a five-way draw, individually lying to each player and establishing authority while brokering the deal. Despite Italy's record of deception, the other players believe the proposal (annotating received messages from Italy as truthful) and expect a 1907 endgame, the year with the most lies. Italy goes on the offensive and knocks out Austria. Italy's summary of the game in their own words is in the Appendix, Table 6. Each game has relationships that are forged and then riven. In another game, an honest attempt by a strong Austria to woo an ascendant Germany backfires, knocking Austria from the game. Germany builds trust with Austria through a believed fictional experience as a Boy Scout in Maine ( likability ). In a third game, two consecutive unfulfilled promises by an ambitious Russia leads to a quick demise, as their subsequent excuses and apologies are perceived as lies (failed consistency ). In another game, England, France, and Russia simultaneously attack Germany after offering duplicitous assurances. Game outcomes vary despite the identical, balanced starting board, as different players use unique strategies to persuade, and occasionally deceive, their opponents. Statements can be incorrect for a host of reasons: ignorance, misunderstanding, omission, exaggeration. Gokhman et al. (2012) highlight the difficulty of finding willful, honest, and skilled deception outside of short-term, artificial contexts (De-Paulo et al., 2003). Crowdsourced and automatic datasets rely on simple negations (Prez-Rosas et al., 2017) or completely implausible claims (e.g., Tipper Gore was created in 1048 from Thorne et al. (2018)). While lawyers in depositions and users of dating sites will not willingly admit to their lies, the players of online games are more willing to revel in their deception. We must first define what we mean by deception. Lying is a mischaracterization; it's thus no surprise that a definition may be divisive or the subject of academic debate (Gettier, 1963). We provide this definition to our users: Typically, when [someone] Figure 2: Every time they send a message, players say whether the message is truthful or intended to deceive. The receiver then labels whether incoming messages are a lie or not. Here Italy indicates they believe a message from England is truthful but that their reply is not. lies [they] say what [they] know to be false in an attempt to deceive the listener (Siegler, 1966). An orthodox definition requires the speaker to utter an explicit falsehood (Mahon, 2016); skilled liars can deceive with a patina of veracity. A similar definition is required for prosecution of perjury, leading to a paucity of convictions (Bogner et al., 1974). Indeed, when we ask participants what a lie looks like, they mention evasiveness, shorter messages, over-qualification, and creating false hypothetical scenarios (DePaulo et al., 2003). Previous work on the language of Diplomacy (Nic-ulae et al., 2015) lacked access to players' internal state and was limited to post-hoc analysis. We improve on this by designing our own interface that gathers players' intentions and perceptions in real-time (Section 3.1). As with other highly subjective phenomena like sarcasm (Gonzlez-Ibez et al., 2011; Bamman and Smith, 2015), sentiment (Pang et al., 2008) and framing (Greene and Resnik, 2009), the intention to deceive is reflective on someone's internal state. Having individuals provide their own labels for their internal state is essential as third party annotators could not accurately access it (Chang et al., 2020). Most importantly, our gracious players have allowed this language data to be released in accordance with IRB authorized anonymization, encouraging further work on the strategic use of deception in long-lasting relations. 2 2 Data available at http://go.umd.edu/diplomacy_data and as part of ConvoKit http://convokit.cornell.edu . This dataset requires both a social and technical setup: finding a community that plays Diplomacy online and having them use a framework for annotating these messages. We need two technical components for our study: a game engine and a chat system. We choose Backstabbr 3 as an accessible game engine on desktop and mobile platforms: players input their moves and the site adjudicates game mechanics (Chiodini, 2020). Our communication framework is atypical. Thus, we create a server on Discord, 4 the group messaging platform most used for online gaming and by the online Diplomacy community (Coberly, 2019). The app is reliable on both desktop and mobile devices, free, and does not limit access to messages. Instead of direct communication, players communicate with a bot; the bot does not forward messages to the recipient until the player annotates the messages (Figure 2). In addition, the bot scrapes the game state from Backstabbr to sync game and language data. Annotation of lies is a forced binary choice in our experiment. Explicitly calling a statement a lie is difficult, and people would prefer degrees of deception (Bavelas et al., 1990; Bell and DePaulo, 1996). Thus, we follow previous work that views linguistic deception as binary (Buller et al., 1996; Braun and Van Swol, 2016). Some studies make a more fine-grained distinction; for example, Swol et al. (2012) separate strategic omissions from blatant lies (we consider both deception). However, because we are asking the speakers themselves (and not trained annotators) to make the decision, we follow the advice from crowdsourcing to simplify the task as much as possible (Snow et al., 2008; Sabou et al., 2014). Long messages can contain both truths and lies, and we ask players to categorize these as lies since the truth can be a shroud for their aims. The Diplomacy players maintain an active, vibrant community through real-life meetups and online play (Hill, 2014; Chiodini, 2020). We recruit top players alongside inexperienced but committed players in the interest of having a diverse pool. Our experiments include top-ranked players and community leaders from online platforms, grizzled in-person tournament players with over 100 past games, and board game aficionados. These players serve as our foundation and during initial design helped us to create a minimally annoying interface and a definition of a lie that would be consistent with Diplomacy play. Good playersas determined by active participation, annotation and game outcomeare asked to play in future games. In traditional crowdsourcing tasks compensation is tied to piecework that takes seconds to complete (Buhrmester et al., 2011). Diplomacy games are different in that they can last a month. . . and people already play the game for free. Thus, we do not want compensation to interfere with what these players already do well: lying. Even the obituary of the game's inventor explains Diplomacy rewards all manner of mendacity: spying, lying, bribery, rumor mongering, psychological manipulation, outright intimidation, betrayal, vengeance and backstabbing (the use of actual cutlery is discouraged) (Fox, 2013). Thus, our goal is to have compensation mechanisms that get people to play this game as they normally would, finish their games, and put up with our (slightly) cumbersome interface. Part of the compensation is non-monetary: a game experience with players that are more engaged than the average online player. To encourage complete games, most of the payment is conditioned on finishing a game, with rewards for doing well in the game. Players get at least $40 upon finishing a game. Additionally, we provide bonuses for specific outcomes: $24 for winning the game (an evenly divisible amount that can be split among remaining players) and $10 for having the most successful lies, i.e., statements they marked as a lie that others believed. 5 Diplomacy usually ends with a handful of players dividing the board among themselves and agreeing to a tie. In the game described in Section 2.1, the remaining four players shared the winner's pool with Italy after 10 in-game years, and Italy won the prize for most successful lies. 5 The lie incentive is relatively small (compared to incentives for participation and winning) to discourage an opportunistic player from marking everything as a lie. Games were monitored in real-time and no player was found abusing the system (marking more than  20% lies). Table 2 quantitatively summarizes our data. Messages vary in length and can be paragraphs long (Figure 3). Close to five percent of all messages in the dataset are marked as lies and almost the same percentage (but not necessarily the same messages) are perceived as lies, consistent with the veracity effect (Levine et al., 1999). In the game discussed above, eight percent of messages are marked as lies by the sender and three percent of messages are perceived as lies by the recipient; however, the messages perceived as lies are rarely lies (Figure 4). We collect anonymous demographic information from our study participants: the average player identifies as male, between 20 and 35 years old, speaks English as their primary language, and has played over fifty Diplomacy games. 6 Players self-assess their lying ability before the study. The average player views themselves as better than average at lying and average or better than average at perceiving lies. 6 Our data skews 80% male and 95% of the players speak English as a primary language. Ages range from eighteen and sixty-four. Game experience is distributed across beginner, intermediate, and expert levels. In a post-game survey, players provide information on whom they betrayed and who betrayed them in a given game. This is a finer-grained determination than the post hoc analysis used in past work on Diplomacy (Niculae et al., 2015). We ask players to optionally provide linguistic cues to their lying and to summarize the game from their perspective (examples in Appendix, Table 6). Four possible combinations of deception and perception can arise from our data. The sender can be lying or telling the truth. Additionally, the receiver can perceive the message as deceptive or truthful. We name the possible outcomes for lies as Deceived or Caught, and the outcomes for truthful messages as Straightforward or Cassandra, 7 based on the receiver's annotation (examples in Table 3, distribution in Figure 4). We build computational models both to detect lies to better understand our dataset. The data from the user study provide a training corpus that maps language to annotations of truthfulness and deception. Our models progressively integrate informationconversational context and in-game power dynamicsto approach human parity in deception detection. 7 In myth, Cassandra was cursed to utter true prophecies but never be believed. For a discussion of Cassandra's curse vis a vis personal and political oaths, see Torrance (2015). We investigate two phenomena: detecting what is intended as a lie and what is perceived as a lie. However, this is complicated because most statements are not lies: less than five percent of the messages are labeled as lies in both the ACTUAL LIE and the SUSPECTED LIE tasks (Table 2). Our results use a weighted F 1 feature across truth and lie prediction, as accuracy is an inflated metric given the class imbalance (Japkowicz and Stephen, 2002). We thus adopt an in-training approach (Zhou and Liu, 2005) where incorrect predictions of lies are penalized more than truthful statements. The relative penalty between the two classes is a hyper-parameter tuned on F 1 . Before we move to computational models for lie detection, we first establish the human baseline. We know when senders were lying and when receivers spotted a lie. Humans spot 88.3% of lies. However, given the class imbalance, this sounds better than it is. Following the suggestion of Levine et al. (1999), we focus on the detection of lies, where humans have a 22.5 Lie F 1 . To prevent overfitting to specific games, nine games are used as training data, one is used for validation for tuning parameters, and two games are test data. Some players repeat between games. Logistic regression models have interpretable coefficients which show linguistic phenomena that correlate with lies. A word that occurs infrequently overall but often in lies, such as honest' and can-didly', helps identify which messages are lies. Niculae et al. (2015) propose linguistic Harbingers that can predict deception. These are word lists that cover topics often used in interpersonal communication claims , subjectivity , premises , contingency , comparisons , expansion , temporal language associated with the future , and all other temporal language (complete word list in Appendix, Table 8). The Harbingers word lists do not provide full coverage, as they focus on specific rhetorical areas. A logistic regression model with all word types as features further improves F 1 . Power dynamics influence the language and flow of conversation (Danescu-Niculescu-Mizil et al., 2012, 2013; Prabhakaran et al., 2013). These dynamics may influence the likeliness of lying; a stronger player may feel empowered to lie to their neighbor. Recall that victory points (Section 2) encode how well a player is doing (more is better). We represent the power differential as the difference between the two players. Peers will have a zero differential, while more powerful players will have a positive differential with their interlocutor. The differential changes throughout the game, so this feature encodes the difference in the season the message was sent. For example, a message sent by an Italy with seven points to a Germany with two points in a given season would have a value of five. While less interpretable, neural models are often more accurate than logistic regression ones (Ribeiro et al., 2016; Belinkov and Glass, 2019). We build a standard long short-term memory network (Hochreiter and Schmidhuber, 1997, LSTM ) to investigate if word sequencesignored by logistic regressioncan reveal lies. Integrating message context and power dynamics improves on the neural baseline. A Hierarchical LSTM can help focus attention on specific phrases in long conversational contexts. In the same way it would be difficult for a human to determine prima facie if a statement is a lie without previous context, we posit that methods that operate at the level of a single message are limited in the types of cues they Human Context LSTM+Power+BERT Context LSTM+Power Context LSTM+BERT Context LSTMLSTM Bag of Words+PowerBag of Words Harbingers+PowerHarbingersMajority Class Random 58.1 56.1 57.2 52.7 55.8 53.8 54.9 54.3 52.9 52.8 47.8 39.8 Macro F1 22.5 20.9 27.0 13.5 19.2 13.7 20.2 19.1 23.7 24.6 14.9 Lie F1 A c t u a l L i e 0 20 40 60 Context LSTM+Power+BERT Context LSTM+Power Context LSTM+BERT Context LSTMLSTM Bag of Words+PowerBag of Words Harbingers+PowerHarbingersMajority Class Random 53.6 53.3 53.3 54.3 53.8 51.6 51.5 45.1 45.9 48.3 38.3 0 10 20 12.4 13.0 15.1 15.0 13.6 13.9 13.7 15.5 14.7 11.8 S u s p e c t e d L i e Figure 5: Test set results for both our ACTUAL LIE and SUSPECTED LIE tasks. can extract. The hierarchical LSTM is given the context of previous messages when determining if a given message is a lie, which is akin to the labeling task humans do when annotating the data. The model does this by encoding a single message from the tokens, and then running a forward LSTM over all the messages. For each message, it looks at both the content and previous context to decide if the current message is a lie. Fine-tuning BERT (Devlin et al., 2019) embeddings to this model did not lead to notable improvement in F 1 , likely due to the relative small size of our training data. Last, we incorporate information about power imbalance into this model. This model approaches human performance in terms of F 1 score by combining content with conversational context and power imbalance. This section examines specific messages where both players and machines are correctly identifying lies and when they make mistakes on our test set. Most messages are correctly predicted by both the model and players (2055 of 2475 messages); but this is because of the veracity effect. The picture is less rosy if we only look at messages the sender marks as ACTUAL LIE : both players and models are generally wrong (Table 5). Both models and players can detect lies when liars get into specifics. In Diplomacy, users must agree to help one another through orders that stipulate I will help another player move from X to Y. The in-game term for this is support; half the messages where players and computers correctly identify lies contain this word, but it rarely occurs in the other quadrants. Models seem to be better at not falling for vague excuses or fantastical promises in the future. Players miss lies that promise long-term alliances, involve extensive apologies, or attribute motivation as coming from other countries' disinformation ( Model Correct ). Unlike our models, players have access to conversations with other players and accordingly players can detect lies that can easily be verified through conversations with other players ( Player Correct ). However, ultimately most lies are believable and fool both models and players ( Both Wrong ). For example, all messages that contain the word true are predicted as truthful by both models and play-Model Prediction Correct Wrong P l ay er P re d i c t i o n Correct Both Correct Not sure what your plan is, but I might be able to support you to Munich. ers. Many of these messages are relatively tame; 8 confirming the Pinocchio effect found by Swol et al. (2012). If liars can be detected when they wax prolix, perhaps the best way to avoid detection is to be terse and to the point. Sometimes additional contextual information helps models improve over player predictions. For example, when France tells Austria I am worried about a steamroller Russia Turkey alliance, the message is incorrectly perceived as truthful by both the player and the single-message model. However, once the model has contexta preceding question asking if Austria and Turkey were cooperatingit can detect the lie. Finally, we investigate categories from the Harbingers (Niculae et al., 2015) word lists. Lies are more likely to contain subjectivity and premises while true messages include expansion phrases (later, additionally). We also use specific words in the bag of words logistic regression model. The coefficient weights of words that express sincerity (e.g., sincerely, frankly) and apology (e.g., accusation, fallout, alterna-tives) skew toward ACTUAL LIE prediction in the logistic regression model. More laid back appella-8 Examples include It's true[Budapest] back to [Ru-mania] and [Serbia] on to [Albania] could position for more forward convoys without needing the rear fleet... and idk if it's true just letting u know since were allies. tions (e.g., dude, man) skew towards truthfulness, as do words associated with reconnaissance (e.g., fyi,useful, information) and time (e.g., weekend, morning). Contested areas on the Diplomacy map, such as Budapest and Sevastopol, are more likely to be associated with lies, while more secure ones like Berlin, are more likely to be associated with truthful messages. Early computational deception work focuses on single utterances (Newman et al., 2003), especially for product reviews (Ott et al., 2012). But deception is intrinsically a discursive phenomenon and thus the context in which it appears is essential. Our platform provides an opportunity to observe deception in the context in which it arises: goal-oriented conversations around in-game objectives. Gathering data through an interactive game has a cheaper per-lie cost than hiring workers to write deceptive statements (Jurgens and Navigli, 2014). Other conversational datasets are mostly based on games that involve deception including Werewolf (Girlea et al., 2016), Box of Lies (Soldner et al., 2019), and tailor-made games (Ho et al., 2017). However, these games assign individuals roles that they maintain throughout the game (i.e., in a role that is supposed to deceive or in a role that is deceived). Thus, deception labels are coarse: an individual always lies or always tells the truth. In contrast, our platform better captures a more multifaceted reality about human nature: everyone can lie or be truthful with everyone else, and they use both strategically. Hence, players must think about every player lying at any moment: given the evidence, do I think this person is lying to me now ? Deception data with conversational labels is also available through interviews (Prez-Rosas et al., 2016), some of which allow for finer-grained deception spans (Levitan et al., 2018). Compared with game-sourced data, however, interviews provide shorter conversational context (often only a single exchange with a few follow-ups) and lack a strategic incentiveindividuals lie because they are instructed to do so, not to strategically accomplish a larger goal. In Diplomacy, users have an intrinsic motivation to lie; they have entertainment-based and financial motivations to win the game. This leads to higher-quality, creative lies. Real-world examples of lying include perjury (Louwerse et al., 2010), calumny (Fornaciari and Poesio, 2013), emails from malicious hackers (Dhamija et al., 2006), and surreptitious user recordings. But real-world data comes with real-world complications and privacy concerns. The artifice of Diplomacy allows us to gather pertinent language data with minimal risk and to access both sides of deception: intention and perception. Other avenues for less secure research include analyzing dating profiles for accuracy in self-presentation (Toma and Hancock, 2012) and classifying deceptive online spam (Ott et al., 2011). In Dante's Inferno , the ninth circle of Hella fate worse even than that reserved for murderersis for betrayers. Dante asks Count Ugolino to name his betrayer, which leads him to say: but if my words can be the seed to bear the fruit of infamy for this betrayer who feeds my hunger, then I shall speakin tears (Alighieri and Musa, 1995, Canto XXXIII) Similarly, we ask victims to expose their betrayers in the game of Diplomacy. The seeds of players' negotiations and deceit could, we hope, yield fruit to help others: understanding multi-party negotiation and protecting Internet users. While we ignore nuances of the game board to keep our work general, Diplomacy is also a rich, multi-agent strategic environment; Paquette et al. (2019) ignore Diplomacy's rich language to build bots that only move pieces around the board. An exciting synthesis would incorporate deception and language generation into an agent's policy; our data would help train such agents. Beyond playing against humans, playing with a human in the loop ( HITL ) resembles designs for cybersecu-rity threats (Cranor, 2008), annotation (Branson et al., 2010), and language alteration (Wallace et al., 2019). Likewise, our lie-detection models can help a user in the moment better decide whether they are being deceived (Lai et al., 2020). Computers can meld their attention to detail and nigh infinite memory to humans' grasp of social interactions and nuance to forge a more discerning player. Beyond a silly board game, humans often need help verifying claims are true when evaluating health information (Xie and Bugg, 2009), knowing when to take an e-mail at face value (Jagatic et al., 2007), or evaluating breaking news (Hassan et al., 2017). Building systems to help information consumers become more discerning and suspicious in low-stakes settings like online Diplomacy are the seeds that will bear the fruits of interfaces and machine learning tools necessary for a safer and more robust Internet ecosystem. We thank Chris Martin for the introduction to the Diplomacy community and for study suggestions. Feedback from Philip Resnik, Alexander Fraser, Bill Ferguson, James Ryan, and Vinodkumar Prabhakaran helped shape the paper's structure. The information provided in this document is derived from an effort sponsored by the Defense Advanced Research Projects Agency ( DARPA ) and Air Force Research Laboratory ( AFRL ), and awarded to Raytheon BBN Technologies under contract number FA865018-C-7885. Danescu-Niculescu-Mizil is supported by NSF award IIS -1750615 and by NSF grant IIS -1910147. Opinions, findings, conclusions, or recommendations expressed here are those of the authors and do not necessarily reflect views of the sponsors. We thank Sebastien A., Joe Brelsford ( Trust-worthyWarMonger ), Sam Brothers, Max Christie, Jordan Connors ( Conq ), Anna Conte, Bill Hack-enbracht, Jack Henrichs, Melissa Lewis, Michael Lotfy ( Blitzkrieg13 ), Joshua Lovett-Graff, Mitch McConeghey, Marko Papic, Christopher Rawles, David Van Slyke ( happypopday ), Reno Vargh-ese, Tyler Waaler, Joseph Wheeler ( Sloth ), Phillip Wilcox, Jorge Zhang ( Caped Baldy ), Daniel Zhu, papa_k , questionmark , and the dozens of other players that made the games possible.
abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain method objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain method other abstain abstain method abstain method abstain abstain abstain abstain abstain method method method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain other other objective other other other other objective other other other other other other other abstain other abstain abstain abstain method method abstain abstain abstain abstain abstain other other other other other other