Even to a simple and short news headline, readers react in a multitude of ways: cognitively (e.g. inferring the writer's intent), emotionally (e.g. feeling distrust), and behaviorally (e.g. sharing the news with their friends). Such reactions are instantaneous and yet complex, as they rely on factors that go beyond interpreting factual content of news. We propose Misinfo Reaction Frames (MRF), a pragmatic formalism for modeling how readers might react to a news headline. In contrast to categorical schema, our free-text dimensions provide a more nuanced way of understanding intent beyond being benign or malicious. We also introduce a Misinfo Reaction Frames corpus, a crowdsourced dataset of reactions to over 25k news headlines focusing on global crises: the Covid-19 pandemic, climate change, and cancer. Empirical results confirm that it is indeed possible for neural models to predict the prominent patterns of readers' reactions to previously unseen news headlines. Additionally, our user study shows that displaying machine-generated MRF implications alongside news headlines to readers can increase their trust in real news while decreasing their trust in misinformation. Our work demonstrates the feasibility and importance of pragmatic inferences on news headlines to help enhance AI-guided misinformation detection and mitigation. Many objects, persons, and experiences in the world are framed in terms of their potential role in supporting, harming, or enhancing people's lives or interests. We can know that this is so if we know how to interpret expressions in which such things are evaluated... Charles J. Fillmore (1976) Epidemics and cases of disease in the 21st century are "staged" Headline Reader "they are lying to us." Effectively predicting how a headline may influence a reader requires knowledge of how readers perceive the intent behind real and fake news. While most prior NLP research on misinformation has focused on fact-checking, preventing spread of misinformation goes beyond determining veracity (Schuster et al., 2020; Ren et al., 2021). For example, in Figure 1, mistrust in the government may lead readers to share pandemic conspiracy headlines like Epidemics and cases of disease in the 21st century are staged even if they suspect it is misinformation. The widespread circulation of misinformation can have serious neg-3108 (cid:90) News Headline (cid:98) Writer's Intent (cid:98) Reader Reaction Spread Real News? ative repercussions on readers  it can reinforce sociopolitical divisions like anti-Asian hate (Vid-gen et al., 2020; Abilov et al., 2021), worsen public health risks (Ghenai and Mejova, 2018), and undermine efforts to educate the public about global crises (Ding et al., 2011). We introduce Misinfo Reaction Frames (MRF), a pragmatic formalism to reason about the effect of news headlines on readers. Inspired by Frame semantics (Fillmore, 1976), our frames distill the pragmatic implications of a news headline in a structured manner. We capture free-text explanations of readers reactions and perceived author intent, as well as categorical estimates of veracity and likelihood of spread (Table 2). We use our new formalism to collect the MRF corpus, a dataset of 202.3k news headline/annotated dimension pairs (69.8k unique implications for 25.1k news headlines) from Covid-19, climate and cancer news. We train reaction inference models to predict MRF dimensions from headlines. As shown by Table 1, reaction inference models can correctly label the veracity of headlines (85% F1) and infer commonsense knowledge like a cat being arrested for disobeying curfew =  lockdowns are enforced. However, models struggle with more nuanced implications a cat arrested for disobeying curfew =  government incompetence. We test generalization of reaction frame inference on a new cancer domain and achieve 86% F1 by finetuning our MRF model on 574 annotated examples. To showcase the usefulness of the MRF framework in user-facing interventions, we investigate the effect of MRF explanations on reader trust in headlines. Notably, in a user study our results show that machine-generated MRF inferences affect readers' trust in headlines and for the best model there is a statistically significant correlation (Pearson's r =0.24, p =0.018) with labels of trustworthiness (5.3). Our framework and corpus highlight the need for reasoning about the pragmatic implications of news headlines with respect to reader reactions to help combat the spread of misinformation. We publicly release the MRF corpus and trained models to enable further work ( https://github.com/ skgabriel/mrf-modeling ). 1 We explore promising future directions (and limitations) in (6). Motivation for Our Formalism In contrast to prior work on misinformation detection (Ott et al., 2011; Rubin et al., 2016; Rashkin et al., 2017; Wang, 2017; Hou et al., 2019; Volkova et al., 2017; Jiang and Wilson, 2018) which mostly focuses on linguistic or social media-derived features, we focus on the potential impact of a news headline by modeling readers' reactions. This approach is to better understand how misinformation can be countered, as it has been shown that interventions from AI agents are better at influencing readers than strangers (Kulkarni and Chi, 2013). In order to model impact, we build upon prior work that aims to describe the rich interactions involved in human communication, including semantic frames (Fillmore, 1976), the encoder-decoder theory of media (Hall, 1973) 2 , Grice's conversational maxims (Grice, 1975) and the rational speech act model (Goodman and Frank, 2016) 3 . By describing these interactions with free-text implications invoked by a news headline, we also follow from prior work on pragmatic frames of connotation and social biases (Speer and Havasi, 2012; 2 This theory proposes that before an event is communicated, a narrative discourse encoding the objectives of the writer is generated. 3 Here pragmatic interpretation is framed as a probabilistic reasoning problem. Rashkin et al., 2018; Sap et al., 2019, 2020; Forbes et al., 2020). While approaches like rational speech acts model both a pragmatic speaker and listener, we take a reader-centric approach to interpreting in-tent of a news headline given that the writer's intent is challenging to recover in the dynamic environment of social media news sharing (Starbird et al., 2019). By bridging communication theory, data annotation schema and predictive modeling, we define a concrete framework for understanding the impact of a news headline on a reader. Defining the Frame Structure Table 1 shows real and misinformation news examples from our dataset with headlines obtained from sources described in 3.1. We pair these headline examples with generated reaction frame annotations from the MRF corpus. Each reaction frame contains the dimensions in Table 2. We elicit annotations based on a news headline , which summarizes the main message of an article. We explain this further in 3.1. An example headline is  Covid-19 may strike more cats than believed . To simplify the task for annotators and ground implications in real-world concerns, we define these implications as relating to one of 7 common themes (e.g. technology or government 3110 entities) appearing in Covid and climate news. 4 We list all the themes in Table 3, with some themes being shared between topics. To construct a corpus for studying reader reactions to news headlines, we obtain 69,885 news implications (See 3.1) by eliciting annotations for 25,164 news headlines (11,757 Covid related articles, 12,733 climate headlines and 674 cancer headlines). There are two stages for collecting the corpus (1) news data collection and (2) crowdsourced annotation. A number of definitions have been proposed for labeling news articles based on reliability. To scope our task, we focus on false news that may be unintentionally spread (misinformation). This differs from disinformation, which assumes a malicious intent or desire to manipulate (Fallis, 2014). We examine reliable and unreliable headline extracted from two domains with widespread misinformation: Covid-19 (Hossain et al., 2020) and climate change (Lett, 2017). We additionally test on cancer news (Cui et al., 2020) to measure out-of-domain performance. Climate Change Dataset We retrieve both trustworthy and misinformation headlines related to climate change from NELA-GT-2018-2020 (Gruppi et al., 2020; Norregaard et al., 2019), a dataset of news articles from 519 sources. Each source in this dataset is labeled with a 3-way trustworthy score (reliable / sometimes reliable / unreliable). We discard articles from sometimes reliable sources since the most appropriate label under a binary labeling scheme is unclear. To identify headlines related to climate change, we use keyword filtering. 5 We also use claims from the SciDCC dataset (Mishra and Mittal, 2021), which consists of 11k real news articles from ScienceDaily, 6 and Climate-FEVER (Diggelmann et al., 2020), which consists of more than 1,500 true and false climate claims 4 We use a subset of the data (approx. 200 examples per news topic) to manually identify themes. Note that themes are not disjoint and a news article may capture aspects of multiple themes. 5 We kept any article headline that contained at least one of environment, climate, greenhouse gas, or carbon tax. We remove noisy examples obtained using these keywords with manual cleaning. 6 https://www.sciencedaily.com/ Theme Climate Covid Climate Statistics  Natural Disasters  Entertainment  Ideology  Disease Transmission  Disease Statistics  Health Treatments  Protective Gear  Government Entities   Society   Technology   Table 3: Themes present in articles by each news topic. Covid-19 Dataset For trustworthy news regarding Covid-19, we use the CoAID dataset (Cui and Lee, 2020) and a Covid-19 related subset of NELA-GT-2020 (Gruppi et al., 2020). CoAID contains 3,565 news headlines from reliable sources. These headlines contain Covid-19 specific keywords and are scraped from nine trustworthy outlets (e.g. the World Health Organization). For unreliable news (misinformation), we use The CoronaVirusFacts/DatosCoronaVirus Alliance Database, a dataset of over 10,000 mostly false claims related to Covid-19 and the ESOC Covid19 Misinformation Dataset, which consists of over 200 additional URLs for (mis/dis)information examples. 89 These claims originate from social media posts, manipulated media, and news articles, that have been manually reviewed and summarized by fact-checkers. Cancer Dataset We construct an evaluation set for testing out-of-domain performance using cancer real and misinformation headlines from the 7 The data also includes some claims for which there is not enough info to infer a label. In this section we outline the structured annotation interface used to collect the dataset. Statistics for the full dataset are provided in Table 4. Annotation Task Interface We use the Amazon Mechanical Turk (MTurk) crowdsourcing platform. 10 We provide Figure 2 in the Appendix to show the layout of our annotation task. For ease of readability during annotation, we present a headline summarizing the article to annotators, rather than the full text of the article. Annotators then rate veracity and likelihood of spread based on the headline, as well as providing free-text responses for writer intent, reader perception and reader action. 11 We structure the annotation framework around the themes described in 2. Quality Control We use a three-stage annotation process for ensuring quality control. In the initial pilot, we select a pool of pre-qualified workers by restricting to workers located in the US who have had at least 99% of their human intelligence tasks (hits) approved and have had at least 5000 hits approved. We approved workers who consistently submitted high-quality annotations for the second stage of our data annotation, in which we assessed the ability of workers to discern between misinformation and real news. We removed workers whose accuracy at predicting the label (real/misinfo) of news headlines fell below 70%. Our final pool consists of 80 workers who submitted at least three annotations during the pilot tasks. We achieve pairwise agreement of 79% on the label predicted by annotators during stage 3, which is comparable to prior work on Covid misinformation (Hossain et al., 2020). To account for chance agreement, we also measure Cohen's Kappa  = . 51 , which is considered moderate agreement. Additional quality control measures were taken as part of our extensive annotation post-processing. For details, see Appendix A.2. Annotator Demographics We provided an optional demographic survey to MTurk workers during annotation. Of the 63 annotators who reported ethnicity, 82.54% identified as White, 9.52% as Black/African-American, 6.35% as Asian/Pacific 10 https://www.mturk.com/ 11 These news events are either article headlines or claims. Islander, and 1.59% as Hispanic/Latino. For self-identified gender, 59% were male and 41% were female. Annotators were generally well-educated, with 74% reporting having a professional degree, college-level degree or higher. Most annotators were between the ages of 25 and 54 (88%). We also asked annotators for their preferred news sources. New York Times, CNN, Twitter, Washington Post, NPR, Reddit, Reuters, BBC, YouTube and Facebook were reported as the 10 most common news sources. We test the ability of large-scale language models to predict Misinfo Reaction Frames. For free-text inferences (e.g. writer intent, reader perception), we use generative language models, specifically T5 encoder-decoder (Raffel et al., 2020) and GPT-2 decoder-only models (Radford et al., 2019). For categorical inferences (e.g. the gold label), we use either generative models or BERT-based discriminative models (Devlin et al., 2019). We compare neural models to a simple retrieval baseline ( BERT-NN ) where we use gold implications aligned with the most similar headline from the training set. 12 4.1 Controlled Generation For generative models, we use the following input sequence x = h 1 ... h T || s d || s t , where h is a headline of length T tokens, s t  { [ covid ] , [ climate ] } is a special topic control token, and s d is a special dimension control token representing one of six reaction frame dimensions. Here || represents concatenation. The output is a short sequence representing the predicted inference (e.g.  to protest  for reader action,  misinfo  for the gold label). For GPT-2 models we also append the gold output inference y = g 1 ... g N during training, where N is the length of the inference. Inference We predict each token of the output inference starting from the topic token s t until the [ eos ] special token is generated. In the case of data with unknown topic labels, this allows us to jointly predict the topic label and output inference. We decode using beam search, since generations by beam search are known to be less diverse but more 12 Similarity is measured between headlines embedded with MiniLM, a distilled transformer model (Wang et al., 2020). We use the Sentence-BERT package (Reimers and Gurevych, 2019). For discriminative models, we use the following input sequence where [CLS] and [SEP] are model-specific special tokens. The output is a categorical inference. All our models are optimized using cross-entropy loss, where generally for a sequence of tokens t Here P  is the probability given a particular language model  . Since GPT-2 does not explicitly distinguish between the input and output (target) sequence during training, we take the loss with respect to the full sequence. For T5 we take the loss with respect only to the output. To improve generalization of MRF models, we use an additional masked fine-tuning step. We first train a language model  on a set of Covid19 training examples D covid and climate training examples D climate . Then we use the Textrank algorithm (Mihalcea and Tarau, 2004) to find salient keyphrases in D covid and D climate , which we term k covid and k climate respectively. We determine domain-specific keyphrases by looking at the complement of k covid  k climate k  covid = k covid \ k covid  k climate k  climate = k climate \ k covid  k climate , and only keep the top 100 keyphrases for each domain. We mask out these keyphrases in the training examples from D covid and D climate by replacing them with a < mask > token. Then we continue training by fine-tuning on the masked examples. A similar approach has been shown to improve generalization and reduce shortcutting of reasoning in models for event detection (Liu et al., 2020). In this section, we evaluate the effectiveness of our proposed framework at predicting likely reactions, countering misinformation and detecting misinformation. We first describe setup for experiments (5.1), as well as evaluation metrics for classification and generation experiments using our corpus (5.2.1,5.2.2). We also show the performance of large-scale language models on the task of generating reaction frames (5.3) and provide results for classification of news headlines (5.4). We determine the test split according to date to reduce topical and news event overlap between train and test sets. 13 We use the HuggingFace Transformers library (Wolf et al., 2020). Hyperparameters are provided in Appendix A.3. We compare reaction inference systems using common automatic metrics. We also use human evaluation to assess quality and potential use of generated writer intent inferences. These metrics include the BLEU (-4) ngram overlap metric (Papineni et al., 2002) and BERTScore (Zhang et al., 2020), a model-based metric for measuring semantic similarity between generated inferences and references. For classification we report macro-averaged precision, recall and F1 scores. 1415 We use publicly available implementations for all metrics (nltk 16 for BLEU). For human evaluation, we assess generated inferences using the same pool of qualified workers who annotated the original data. We randomly sample model-generated writer's intent implications from T5 models and GPT-2 large over 196 headlines where generated implications were unique for each model type. 17 We elicit 3 unique judgements per headline. Implications are templated in the 13 We use news articles from 2021 and the last two months of 2020 for the test set. We ensure there is no exact overlap between data splits. 14 We compute these using scikit-learn: https:// scikit-learn.org/stable/index.html 15 For measuring likelihood of spread, predicted and averaged values are rounded to the nearest integer. 16 https://www.nltk.org/ 17 98 misinfo and 98 real headlines in the dev. set 3113 Writer Intent Reader Perception Reader Action Model BLEU-4  BERTScore  BLEU-4  BERTScore  BLEU-4  BERTScore  BERT-NN 31.45 86.29 35.69 91.04 45.47 84.76 T5-base 51.48 88.03 31.98 92.87 53.55 85.27 dev. Overall Quality We ask the annotators to assess the overall quality of generated implications on a 1-5 Likert scale (i.e. whether they are coherent and relevant to the headline without directly copying). Influence on Trust We measure whether generated implications impact readers' perception of news reliability by asking annotators whether a generated implication makes them perceive the news headline as more (+) or less (-) trustworthy. Perceived Sociopolitical Acceptability We ask annotators to rate their perception of the beliefs invoked by an implication in terms of whether they represent a majority (mainstream) or minority (fringe) viewpoint. 18 A/B Testing For A/B testing, annotators are initially shown the headline with the generated implication hidden. We ask annotators to rate trustworthiness of headlines on a 1-5 Likert scale, with 1 being clearly misinformation and 5 being clearly real news. After providing this rating, we reveal the generated implication to annotators and have them rate the headline again on the same scale. Annotators were not told whether or not implications were machine-generated, and we advised annotators to 18 We refer to minority viewpoint broadly in terms of less frequently adopted or extreme social beliefs, rather than in terms of viewpoints held by historically marginalized groups. Results We found that the T5-large model was rated as having slightly higher quality generations than the other model variants (Table 6). Most model generations were rated as being socially acceptable . However in as many as 25.34% of judgements, generations were found to be not socially acceptable. Interestingly, all models were rated capable of influencing readers to trust or distrust headlines, but effectiveness is dependent on the quality of the generated implication. In particular for T5-base, we found a consistent correlation between the actual label and shifts in trustworthiness scores before and after annotators see the generated writer's intent. Annotators reported that writer intents made real news appear more trustworthy and misinformation less trustworthy. 19 5.4 Detecting Misinformation To test if we can detect misinformation using propagandistic content like loaded or provocative lan-19 While for most models the trend is a decrease in trust for both real news and misinformation, for the T5-base model there is a statistically significant correlation of Pearson's r = . 24 showing shifts in trust align with gold labels. guage (e.g. Covid-19 vaccines may be the worst threat we face ), we use a pre-trained BERT propaganda detector (Da San Martino et al., 2019) which we denote here as ( Prop-BERT ). 20 For our zero-shot setting, we classify a news event as real if it is not associated with any propaganda techniques and misinformation otherwise. As shown by Table 7, F1 results are considerably lower than task-specific models. This is likely due to the fact both real and misinformation news uses propaganda techniques. Neural misinformation detection models are able to outperform humans at identifying misinformation (achieving a max F1 of 85.26 compared to human performance F1 of 75.21 21 ), but this is still a nontrivial task for large-scale models. When we use Covid-BERT (Muller et al., 2020), a variant of BERT pretrained on 160M Covid-related tweets, we see an improvement of 5.46% over BERT without domain-specific pretraining (Table 7). This indicates greater access to domain-specific data helps in misinformation detection, even if the veracity of 20 The model predicts if any of 18 known propaganda techniques are used to describe a news event. See the paper for the full list. 21 We count disagreements as being labeled misinformation here, discarding disagreements leads to F1 of 74.97. Performance on Out-of-Domain Data We test the ability of reaction frames to generalize using 100 cancer-related real and misinformation health news headlines (Cui et al., 2020), see Table 7. For the misinformation detection task, we evaluate gold F1 using the Prop-BERT zero-shot model, MRF-finetuned BERT-large, Covid-BERT , T5-large and GPT-2 large models. We observe that after one epoch of re-training, masked fine-tuning substantially boosts unsupervised performance of generative MRF models ( GPT-2 large + masked and T5-large + masked ), making them more robust than BERT variants. We compare this performance against the T5-large and GPT-2 large model finetuned on only 574 cancer examples ( GPT-2 large + sup and T5-large + sup ), and observe that this leads to a performance increase of up to 43.49 %, achieving similar F1 performance to our domains with full data supervision. Our framework presents new opportunities for studying perceived intent and impact of misinfor-3115 We can estimate content virality. Given the user-annotated labels for likelihood of reading or sharing, we can estimate whether the information in the associated article is likely to propagate. headlines. Using annotated writer intents, we can determine common themes and perceived intentions in misinformation headlines across domains (e.g. mistrust of vaccination across medical do-mains). Given the performance of predictive models highlighted by Tables 5 and 6, we can also extend this analysis to unseen headlines. We can categorize headlines by severity of likely outcomes. False headlines that explicitly incite violence, or otherwise encourage actions that lead to psychological or physical harm (e.g. not vaccinating) may be deemed more malicious than false headlines with more benign consequences (e.g. some examples of satire). Future work may explore categorizing severity of headlines based on potential harms resulting from implications. headlines may fool readers. We can use these labels to determine which types of misinformation headlines appear most like real news to generally knowledgeable readers. These may also help in designing misinformation countering systems and better adversarial examples to improve robustness of misinformation detection models. We can generate counter-narratives to misinformation. Our results indicate it is possible to generate effective explanations for the intent of headlines that discourage trust in misinformation (Section 5.3), see Appendix A.5 for examples. We encourage future work that further improves performance of these models (e.g. through integration of domain knowledge). Limitations. Given these future directions, we also consider key limitations which must be addressed if we move beyond viewing Misinfo Reaction Frames as a proof-of-concept and use the dataset as part of a large-scale system for evaluating or countering misinformation. Since we focus on news headlines, the context is limited. The intent of a headline may be different from the actual intent of the corresponding article, especially in the case of clickbait. We find headlines to be suitable as online readers often share headlines without clicking on them (Gabielkov et al., 2016), however future work may explore extending reaction frames to full news articles. There is also annotator and model bias. Readers involved in our data curation and human evaluation studies are  generally knowledgeable , as proved by their ability to discern misinformation from real news. We see this bias as a potential strength as it allows us to find ways to counter misinformation in cases where readers are well-informed but still believe false information. However, annotators may have undesirable political or social biases. In such cases, gender bias may lead an annotator to assume that a politician mentioned in a headline is male or to dismiss inequality concerns raised by a scientist belonging to a minority group as play-ing the race card. These biases can also appear in pre-training data, leading to model bias. 22 Subjectivity in annotation is a point of discussion in many pragmatic-oriented tasks, e.g. social norm prediction (Jiang et al., 2021) and toxicity detection (Halevy et al., 2021; Sap et al., 2021). We encourage conscious efforts to recruit diverse pools of annotators so multiple perspectives are considered, and future work on modeling reaction frames can consider learning algorithms that mitigate harmful effects of biases, depending on use case (Khalifa et al., 2021; Gordon et al., 2022). Lastly, we only consider English-language news and annotate with workers based in the US. It may be that news headlines would be interpreted differently in other languages and cultures. We introduced Misinfo Reaction Frames, a pragmatic formalism for understanding reader perception of news reliability. We show that machine-generated reaction frames can change perceptions of readers, and while large-scale language models are able to discern between real news and misinformation, there is still room for future work. Generated reaction frames can potentially be used in a number of downstream applications, including better understanding of event causality, empathetic response generation and as counter-narratives. 22 Removing these examples from data curation or trying to control for annotator neutrality does not erase the causes that lead to the existence of these biases. The fact that harmful biases can manifest in the viewpoints of informed readers speaks to the pervasiveness of certain stereotypes. There is a risk of frame-based machine-generated reader interpretations being misused to produce more persuasive misinformation. However, understanding the way in which readers perceive and react to news is critical in determining what kinds of misinformation pose the greatest threat and how to counteract its effects. Furthermore, while transformer models have contributed to much of the recent algorithmic progress in NLP research and are the most powerful computational models available to us, work has highlighted shortcomings in their performance on domain-specific text (Moradi et al., 2021) and noted that these models can easily detect their own machine-generated misinformation (Zellers et al., 2019). Therefore, we do not see this potential dual-use case as an imminent threat, but urge implementation of systemic changes that would discourage such an outcome in the future -e.g. regulation that would lead to required safety and fairness measures before large-scale systems are deployed in the wild (European Commission, 2021). We emphasize that annotations may reflect perceptions and beliefs of annotators, rather than universal truths (Britt et al., 2019). Especially considering demographic homogeneity of online crowd-source workers, we urge caution in generalizing beliefs or taking beliefs held in certain social/cultural contexts to be factual knowledge. We obtained an Institutional Review Board (IRB) exemption for annotation work, and ensured annotators were fairly paid given time estimations. Broader impact. The rapid dissemination of information online has led to an increasing problem of falsified or misleading news spread on social media like Twitter, Reddit and Facebook (Vosoughi et al., 2018; Geeng et al., 2020). We specifically designed the Misinfo Reaction Frames formalism to allow us to identify and predict high-impact misinformation that is more likely to spread. This can allow for future research on factors that make misinformation particularly dangerous, as well as systems that are more effective at mitigating spread. The authors thank members of the DARPA Se-maFor program, UW NLP, the UW CSE 599 social computing class and Amy X. Zhang for helpful discussions, as well as the anonymous reviewers and Akari Asai for comments on the draft. This research is supported in part by NSF (IIS-1714566), NSF (2041894), DARPA MCS program through NIWC Pacific (N66001-19-2-4031), DARPA Se-maFor program, and Allen Institute for AI.
abstain abstain objective objective method abstain result objective abstain method abstain abstain abstain abstain abstain abstain abstain abstain objective method abstain abstain objective objective result result other objective other abstain abstain other method method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain method method method abstain abstain method method abstain abstain abstain other method abstain abstain method abstain method method abstain method abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain method abstain method other abstain abstain abstain abstain abstain abstain method method method method method abstain abstain abstain method method method method method abstain abstain method abstain method method abstain abstain abstain abstain method abstain abstain abstain abstain result method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain method abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain other other other