We propose the task of updated headline generation , in which a system generates a headline for an updated article, considering both the previous article and headline. The system must identify the novel information in the article update, and modify the existing headline accordingly. We create data for this task using the NewsEdits corpus (Spangher and May, 2021) by automatically identifying contiguous article versions that are likely to require a substantive headline update. We find that models conditioned on the prior headline and body revisions produce headlines judged by humans to be as factual as gold headlines while making fewer unnecessary edits compared to a standard headline generation model. Our experiments establish benchmarks for this new contextual summarization task. Automatic text summarization condenses the most important and salient information from a large quantity of text. The task takes many different forms depending on the type of information being summarized, the modality of the information, the type of summary desired and the needs of the end user. Examples include news headline generation (Banko et al., 2000; Zajic et al., 2002; Dorr et al., 2003; Takase et al., 2016; Matsumaru et al., 2020), summarization of social media (Liu et al., 2012; Ding and Jiang, 2015; Kim et al., 2019), and medical documents (Schulze and Neves, 2016; Liang et al., 2019; Adams et al., 2021). news stories are revised as events unfold (Tannier and Moriceau, 2013), social media streams evolve as people post content (Tarnpradab et al., 2021), and biomedical texts are revised as clinical trial results emerge (uptodate, 2021). In such dynamic settings, existing summaries should be updated as new information becomes available. To address this, we could in principle leverage static summarization systems for generating a summary of the underlying content at any given point in time. However, a more natural approach would be to produce a new summary based on what the reader already knows and what content changed . Consider the case of a news article being updated as events unfold (Figure 1). The article first reports that a man is charged with stealing an ice cream van, and the article is later updated when the man admits to the crime. By the time the article is updated, the reader already knows what was stolen, who was charged, and where it happened. At this point, the reader is most interested in what changed, namely the admission of guilt. In the case of news articles, the new headline must both convey critical new information and provide a holistic overview for readers unfamiliar with the story. Updating a summary instead of wholesale replacement falls outside the scope of static summarization systems. To address these shortcomings, we envision a summarization system that combines an existing summary with information updates. More concretely, following prior work of using headlines as article summaries (Graff et al., 2003), we consider the task of news headline generation. We instead propose updated headline generation , which entails updating headlines based on changes to the content of the article. In this work, we make the following contributions: 6438 Figure 1: Example of a news story where both the body and headline are revised after publication. 1  Evaluate the contribution of different types of information  previous headline, edits to the article body  to a model that makes updates to an existing news headline. Conduct a human evaluation demonstrating that leveraging this additional context leads to headlines which are as factual as standard headline generation models, while applying fewer unnecessary edits. Perform an error analysis to determine which types of headline updates are addressed by our model, and what challenges remain. A news article consists of a body ( B ) and a headline ( H ). Headline generation (Banko et al., 2000; Zajic et al., 2002; Dorr et al., 2003; Takase et al., 2016; Matsumaru et al., 2020) asks a system to consider B and produce H . We propose updated headline generation as a modification of this task. A system receives an existing article ( B 1 , H 1 ) and an updated version of the article body ( B 2 ). The goal is to update H 1 to produce a new headline ( H 2 ) that reflects important new information in B 2 . This task introduces several challenges. First, a system must identify the most critical new information in B 2 . Changes to the article can be small or very significant, and it must determine 1 Available at: https://github.com/panthap2/ updated-headline-generation which of these changes, if any, should be reflected in the headline. Second, it needs to consider how to modify H 1 . Oftentimes a revision to an article will preserve most of the structure of H 1 , even if a completely rewritten headline might convey the same information. New information should be reflected in an updated headline with minimal edits, for the sake of continuity and minimizing cognitive load on a reader who is following an evolving story. Third, there are different types of updated stories that each require a different style of headline update. Stories can be updated as the underlying event progresses (e.g., criminal investigations, natural disasters, voting on legislation or appointments, live events), new or corrected information becomes available (e.g., number of people injured following an accident), or public figures react to the event (e.g., political figure commenting on a situation). See Table 1 for examples. The NewsEdits (Spangher and May, 2021) corpus contains articles with revision histories derived from 22 wires: 5 from News Sniffer 2 and the remainder from Twitter accounts powered by Diff-Engine . 3 It consists of over one million articles with 4.6 million revisions. In this work, we focus on the 5 English language wires from News Sniffer (Washington Post, NY Times, Independent, BBC, Guardian), as we found them to have cleaner revision histories. From the revision history of a given article, we extract body-headline pairs by examining consecutive versions, ( B k , H k ), ( B k +1 , H k +1 ), resulting in examples of the form {( B 1 , H 1 ), ( B 2 , H 2 )}. We 2 https://www.newssniffer.co.uk/ 3 https://github.com/DocNow/diffengine 6439 B 1 H 1 B 2 H 2 Nearly a million people in southern Vietnam face evacuation from the path of a deadly tropical storm ... exclude cases without a change in the body, and group examples into two different classes: positive  examples where the headline is updated (i.e., H 1  = H 2 )  and negative  the headline remains unchanged (i.e., H 1 = H 2 ). We observed that the headline change associated with a particular body change sometimes occurred in the subsequent revision (not contemporaneous). So, we also include positive examples which have the following property across three consecutive revisions: only the body is changed between the first and second versions and only the headline is changed between the second and third versions, i.e., ( B 1 , H 1 )  ( B 2 , H 1 )  ( B 2 , H 2 ). We do not include ( B 1 , H 1 )  ( B 2 , H 1 ) as a negative example for such cases. To avoid spurious positive examples, we tried removing versions that were incorrectly paired together, 4 or where the headline change was trivial. 5 This process produced a dataset of 144,218 positive 4 B 1 and B 2 are sometimes completely unrelated, likely due to an error in the News Sniffer collection. We removed examples in which B 2 was published more than a week after B 1 , and we exclude articles that yield more than 8 version pairs (95th percentile). 5 Trivial headline changes included modifications limited to spacing and punctuation, as well as simple rephrasing (i.e., changes to stopwords or the surface form of a lemma). and 794,372 negative examples. Even after filtering by heuristic, we found that many of the remaining headline changes still do not reflect a substantive update to the article. These include purely stylistic changes, embellishments, and rephrasings. To filter such cases, we develop a classifier which is trained to determine whether H 1 needs to be updated based on the changes between B 1 and B 2 . The classifier achieves 51.9 F1, indicating that this is a challenging problem; the training and evaluation data are silver-labeled, and noisy. We filter the remaining positive examples with this classifier. Empirically, we find that training on this filtered subset leads to improved performance. We provide a complete description of the classifier and attendant experiments in Appendix A. 6440 3.1 The HREN Dataset After data cleaning and filtering, we obtain the H eadline R evision for E volving N ews dataset (HREN), which contains 69,243 examples with meaningful headline edits. Descriptive statistics for each fold are listed in Table 2. Average number of tokens per document are broken by source text type, with B edits and B edits (change only) described in Section 4.2. We partition the data into 80/10/10 training, validation, and test splits. While constructing the data, we took care to ensure that the underlying articles from which examples are drawn are disjoint for partitions, and that the timestamps corresponding to examples in the training set strictly precede those in the validation set, which in turn precede those in the test set. 6 This ensures that we train on strictly historical data. Our main experiments use HREN, though we include negative examples and filtered positive examples in some later experiments. COPYH 1 : Updated headlines usually copy parts of the original headline and the overall structure. For instance, in Figure 1, 8 of the 9 tokens in the updated headline come from the original one. So, we consider copying H 1 as the prediction. LEAD -1: Newsroom style guides dictate that the most significant information should appear first (Siegal and Connolly, 1999). Consequently, the lead sentence typically includes information that is mentioned in the headline, as shown in Figure 1. This baseline uses the lead sentence of B 2 as the prediction for H 2 . SUBSTITUTION : Many headlines can be correctly updated by a simple token replacement, reflecting an analogous replacement in the body. Table 1: H 1 is At least 19 Hurt in Tractor-Trailer and Bus crash on I-64 in Virginia" and a sentence in B 1 At least 19 people..." is updated to At least 24 people" in B 2 , prompting a similar change in the headline H 2 . So, if a single token ( t 1 = 19 ) appearing in both H 1 and B 1 is replaced with a new 6 Similar time-based partitioning was done for the classifier. See Appendix B for date cutoffs for each fold. token ( t 2 = 24 ) in B 2 , we form H 2 by substituting t 1 with t 2 in H 1 . We only consider single-token replacements and copy H 1 if a substitution cannot be made. Note that this is a high precision baseline, with 10.8% of headlines able to be updated by this heuristic. 4.2 Context Representations We study various configurations for representing the input context for training the models. H 1 : Many headline updates follow a natural progression of events (e.g., Lori Loughlin Expected to Plead Guilty via Zoom in College Admissions Case"  Lori Loughlin Pleads Guilty via Zoom in College Admissions Case"). In these cases, knowing the old headline may be sufficient to predict the subsequent headline. Therefore, we consider providing only H 1 to a statistically trained model. H 1 + B 2 : We provide both H 1 and B 2 . Faithfulness to the article body is paramount for automatic headline generation (Matsumaru et al., 2020), and leveraging the original headline removes some of the burden of generating a headline from scratch. H 1 + B 2 + B 1 : We provide all available context to the model, so that the model can compare story versions and consider the old headline during decoding. H 1 + B edits : Asking the model to compare two full articles may be unrealistic. Instead, we provide the sequence of edits between B 1 and B 2 : < KEEP > A 22-year old man has < KEEP_END > < REPLACE_OLD > been charged after < REPLACE_NEW > admitted stealing < REPLACE_END > < KEEP > an ice cream... This sequence consists of edit actions: insert , delete , replace , and keep , and are represented in the format proposed by Panthaplackel et al. (2020). We study whether providing explicit body edits helps a model learn to apply analogous headline edits. H 1 + B edits (change only): Rather than feeding in the full edit sequence, we discard keep spans. While this removes information about where the edits are made, it significantly reduces the amount of context a model must reason about (Table 2). We evaluate two encoder-decoder models that utilize each of the representations described in Section 4.2. Note that we first preprocess all representations using the Penn Treebank tokenizer 7 to tokenize and split text into sentences and words, prior to model-specific preprocessing. Pointer Networks consist of separate LSTM encoders for body and headline text, and these are concatenated to form the initial states for an LSTM decoder, equipped with attention (Vinyals et al., 2015; Wang et al., 2016). The hidden states are concatenated for both attention and copy mechanisms. We posit that this model might be effective at headline updating, as this task benefits from copying tokens from the input context (especially H 1 ). We initialize embeddings for the model with GloVe (Pennington et al., 2014). BART (Lewis et al., 2020) is a pretrained transformer network considered state-of-the-art for summarization. Because we focus on the news domain, we consider a version of BART already fine-tuned for summarization on news articles from CNN-Daily Mail (Hermann et al., 2015). 8 We further fine-tune on our data, by concatenating inputs into a single sequence, separated by special tokens (e.g., < OLD_HEADLINE > , < NEW_BODY > ). We evaluate all context representations with both of these architectures, with the exception of H 1 + B 2 + B 1 for BART, due to limitations in fitting the entire input context within BART's 1024 token limit. We use beam search with a beam size of 20 to decode for all models along with bigram blocking (Paulus et al., 2018). 9 These decoding hyperparameters were found to work well across models during preliminary experimentation based on an unweighted average across automated metrics. We evaluate with common text-generation metrics: METEOR (Banerjee and Lavie, 2005), ROUGE-L (Lin and Och, 2004), and BLEU-4 (Pa-pineni et al., 2002). Given the editing nature of our task, we also use two edit-specific metrics: GLEU (Napoles et al., 2015) and SARI (Xu et al., 7 https://nlp.stanford.edu/nlp/javadoc/ javanlp/edu/stanford/nlp/process/PTBTokenizer.html 8 https://huggingface.co/facebook/ bart-large-cnn 9 We chose bigram instead of the more typical trigram blocking as headlines tend to be short. 2016). SARI measures the average n-gram F1 scores corresponding to edit operations (add, delete, and keep). GLEU closely follows BLEU except that it places more importance on n-grams which have been correctly changed. We compute statistical significance at the p < 0 . 05 level using bootstrap tests (Berg-Kirkpatrick et al., 2012). Rule-Based Baselines: Our results (Table 3) show that rule-based baselines achieve relatively high performance, even beating the headline generation setting ( B 2 ) for the pointer network and BART in some cases. Due to the high lexical overlap between H 1 and H 2 , the COPYH 1 baseline can perform well on automated metrics, specifically the three text-generation metrics. The SUBSTITUTION baseline performs slightly better than simply copying H 1 by making simple substitutions in 10.8% of examples, demonstrating improvements in the two edit-based metrics. The LEAD -1 baseline performs lower than the other baselines on most metrics due to the discrepancy between the structure and style of the lead sentence and headlines, with the average lead sentence length being 36.7 tokens. However, the SARI score is substantially higher. 10 Using H 1 : For both the pointer network and BART, providing only H 1 results in lower performance than COPYH 1 for most metrics, except for SARI, which is designed to evaluate edits. Higher SARI suggests that these models are able to make the necessary edits in some cases by guessing the natural progression of events, without the news body, such as forecasting the order of events following a police investigation (e.g., suspect is arrested, charged, and then appeared in court). However, the SARI score is still much lower than if only B 2 is provided, as in standard headline generation. This highlights the importance of the latest version of the article body in updated headline generation. Nonetheless, by comparing performance of B 2 and H 1 + B 2 across both architectures, we see the extent to which H 1 can guide headline generation models in selecting important content and determining structure for the output. This demonstrates the inadequacy of framing this as a static headline generation task. The improvements on edit metrics are more limited because a model which has access 10 SARI is calculated as the average of N-gram F1 scores of add, delete, and keep edit operations. Because the lead sentence will not contain many n-grams in H 1 that should be deleted, and will contain some n-grams that should be inserted into H 2 , the SARI score is high for this baseline. Using Body Edits: To investigate whether providing body edits can further improve performance by helping a model learn to correlate them with H 1 and apply analogous updates, we consider different ways of incorporating B 1 . First, in the pointer network, we evaluate performance when just feeding it in as another input ( H 1 + B 2 + B 1 ). We observe no improvement in performance, suggesting that the model fails to implicitly learn the edits. Next, we consider collapsing B 1 and B 2 into a sequence of edits ( H 1 + B edits ), with which we see a slight improvement in performance over H 1 + B 2 for the pointer network but a reduction in performance for BART. We believe this is because BART struggles to model longer input sequences. When we reduce the context length and provide only the changes in the edit sequence ( H 1 + B edits (change only)), we see an improvement in BART. Note that the performance of H 1 + B edits (change only) is lower for the pointer network across most metrics. This may be due to a lack of pretraining, whereas BART is already equipped with a strong language model. Pointer Network vs. BART: While both model classes perform well, BART models tend to perform better overall, demonstrating the value of BART's larger transformer-based architecture and pretraining. Nonetheless, the benefits of using H 1 and body edits generalize across both architectures. We expect that the performance of more recent summarization models such as PEGASUS (Zhang et al., 2020) or SimCLS (Liu and Liu, 2021) will exhibit a similar trend as BART, but we welcome evaluation of other large pretrained summarization Fact  Focs  MnEd  Hdln Grm  Cpy H 1 4 . Design We conduct a human evaluation of the (more performant) BART models with the following configurations: B 2 , H 1 + B 2 , H 1 + B edits (change only). As points of reference, we also evaluate the gold headline ( H 2 ) and the output of the COPYH 1 baseline. Annotators were presented with a visual diff between B 1 and B 2 along with H 1 , and were asked to judge a candidate updated headline according to five dimensions on a Likert scale  whether the updated headline was factual , grammatical , appears to be written in headlinese , focus es on important changes/information in the updated body (similar to the relevance criterion commonly used to evaluate natural language generation models (Sai et al., 2020)), and makes only minimal edits to the original headline. We introduce the last dimension since we frame our task as an editing task. The underlying idea behind editing is that change should only be made to be parts that warrant it; all other 6443 parts that do not need to be changed should be preserved, which is consistent with how humans edit text (Panthaplackel et al., 2020). Additionally, this is consistent with the task motivation, in which we expect a reader to interpret the important changes in a minimally edited headline with less cognitive effort. We sampled 200 test examples, 143 from HREN and 57 from the unfiltered sample, 11 resulting in 806 unique annotation tasks. 12 Each task was independently annotated by three paid annotators who were trained on this task  native English speakers, two of whom were journalism majors. See Appendices C and D for more details on the annotation procedure. Results We present average annotator ratings for each dimension in Table 4. Following the human evaluation analyses in Reiter and Belz (2009) and Wiseman et al. (2021), we compute statistical significance using multi-way ANOVA tests, followed by Tukey's post hoc HSD tests for pairwise statistical significance (at the p < 0 . 05 level). For headlinese and grammatical , we find no significant difference between the approaches; all achieve relatively high scores. With respect to factual and focus , all approaches perform similarly except for COPYH 1 which significantly underper-forms the others, by inaccurately reflecting the state of matters after the story is updated and failing to highlight important changes in B 2 . On the other hand, COPYH 1 achieves the best performance on minimal edits by definition (i.e., H 1 has minimal edits with itself). As expected, without access to H 1 , the headline generation model ( B 2 ) achieves the lowest performance on this dimension. Overall, we find that the two BART models which also include H 1 as context performed better, even beating gold headlines on this dimension. This is unsurprising as gold headlines often undergo stylistic rewrites, in addition to reflecting changes to the facts of evolving news stories. For example, Byron Burger Menu Reassured' Allergy Death Owen Carey is rewritten as Byron Burger Death: Owen Carey's Family Demand Law Change  the form of the headline changes in addition to the release of new information. Although H 1 + B edits (change only) performs slightly better than H 1 + B 2 on automated metrics in Table 3, we find that they 11 Results on the unfiltered examples are in Appendix A.3. perform similarly on the five dimensions. In summary, incorporating H 1 leads to predictions which make fewer unnecessary edits to the original headline, while simultaneously maintaining performance with respect to factuality, focus, headlinese, and grammaticality of headline generation models (on par with gold headlines). Case Study Table 5 presents BART predictions for the example in Figure 1 under different context representations. 13 Given only H 1 , the model predicts an updated headline by speculating about what might follow a person being charged with a crime. Using only B 2 , the model generates a headline which reflects that a person has admitted to the crime, but it deviates from the form of the original headline by inserting the name of the person and altering terminology. These aspects of the story have not changed, and should not be changed in the headline. With H 1 + B 2 , the prediction captures the major change in the article and better retains the form of H 1 , but it still makes an unnecessary change by inserting the person's age into the headline. Given H 1 + B edits , the model learns to only edit the part which is relevant to the body changes, but the terminology used to perform the edit (i.e., pleads guilty ) varies from the article ( admits to the crime.) In contrast, H 1 + B edits (change only) is able to simultaneously perform minimal edits and correlate edits between the article and headline. Performance by Edit Level Headlines require more extensive edits when there are more substantial changes to the article. We perform a fine-grained analysis to better understand how various context representations fare for these different types of examples. We group examples based on 13 Additional examples are provided in Appendix F. 6444 Figure 2: Absolute difference in GLEU between various BART models and standard headline generation ( B 2 ) for each Jaccard headline similarity bucket. the Jaccard similarity between H 1 and the gold H 2 ; low similarity means significant edits, high similarity means minimal edits. For the BART-based model we calculate the GLEU score for each bucket because we find that it is better suited for simultaneously evaluating whether appropriate edits were made along with generation quality. Figure 2 shows the change in performance attributed to each of the context representations relative to headline generation ( B 2 ). For low similarity values, none of the specialized context representations outperform standard headline generation. This suggests that when more substantial edits are needed, starting from scratch may be best. As the similarity increases, models which utilize H 1 perform substantially better. For moderate similarity, having B 2 instead of body edits performs marginally better, but this changes as the similarity score increases, with B edits (change only) leading to drastic improvements. Analyzing Attention To better understand how models explicitly make use of the old headline, we analyze how the H 1 + B edits (change only) BART model's decoder attends to H 1 . For this, we follow the methodology of Vig and Belinkov (2019). Namely, we label each context token by which span it belongs to: H 1 , one of the edit spans, or a span delimiter token ( Other ). We compute the average attention paid to each context token class by the BART decoder across all examples and layers. We find that even though only 1.5% of the context tokens are from H 1 , they attract over 17% of the BART decoder's attention. Interestingly, the attention paid to added content and headline tokens increases in later layers at the expense of Other tokens (Figure 3). We posit that this is because the initial layers need to attend to special tag tokens in order to understand which type of span each enclosed token belongs to. This may also arise from the fact that initially the decoder attends to all tokens relatively uniformly (49.9% of tokens are Other on average). However, even in the initial layers, the tokens in H 1 are attended to more than would be expected by a uniform attention distribution, likely because H 1 text always appears near the start of the context. Because of this, locating the H 1 tokens is less dependent on identifying enclosing tags  absolute position also helps. We also find that the decoder attends to tokens in H 1 more often than would be expected under a uniform attention model, until it needs to refer to a new piece of information that was added to the article body. Figure 4 displays the relative attention paid to each token type for a decoded headline exemplifying this phenomenon. See Appendix E for additional detail on the decoder attention analysis. Error Cases Finally, we inspected cases where annotators assigned very low or high scores. We observe with B 2 alone, the headline generation model makes factual errors by mixing up important details when two similar types of entities are discussed in the article (e.g., mixing up the victim and suspect of a crime, mixing up locations and dates). Additionally, it makes factual errors by omitting something important, which drastically changes the meaning (e.g., missing a letter in the acronym for an organization). On the other hand, because H 1 often includes important background that can be directly copied, we find fewer such factual errors caused by omission for the H 1 + B 2 and H 1 + B edits (change only) models. Having H 1 also helps in maintaining important details (e.g., event location) and specifying the level of detail that is needed. In general, H 1 is most useful when there is high lexical overlap with the lead sentences of B 2 . If the content is significantly different (e.g., the focus of the article changes), it becomes less useful and can even hurt performance in some cases, since H 2 is likely very different from H 1 . Body edits are most useful when there are few edits and these edits can be easily grounded in H 1 . For H 1 + B edits (change only), we also noticed errors where the model incorrectly correlates body edits with H 1 , resulting in it erroneously inserting body tokens that are edited into the headline. Decoded Token Figure 4: Difference between average attention placed on each span type during decoding of H 2 and that expected under a model that attended uniformly to all tokens. X-axis: words in the decoded headline; Y-axis: types of context tokens. Red cells indicate that a token type is being attended to more than would be expected under uniform attention, whereas blue cells indicate the opposite. H 1 : White House to Ask for $12 Billion Down Payment for Harvey Relief , Source: https: //www.newssniffer.co.uk/articles/1447256 . subtopics relevant to our task. For instance, multidocument summarization (Barzilay and McKeown, 2005) pertains to generating a unified summary by synthesizing non-redundant content from multiple related documents. In our setting, we consider multiple documents (i.e., the old and new versions of an article) as well, but we also have an existing summary, and our task requires reasoning about how the non-redundant content from the newer version of the article affects this existing summary. With update summarization (Dang et al., 2008), there is an older set of documents as well as a newer set of documents, and the goal is to generate a summary which captures only added and changed information. In contrast, our task aims to incorporate these changes into an already existing holistic summary. Natural language edits: Our work focuses on learning from edits in news articles to apply update and existing headline. Prior work studies the nature of edits in various texts including news (Faigley and Witte, 1981; Tamori et al., 2017) and Wikipedia (Yang et al., 2017; Faruqui et al., 2018). There has also been extensive work on generating edits for tasks such as grammatical error correction (Bryant et al., 2019), sentence simplification (Zhu et al., 2010), style transfer (Fu et al., 2018), fact-based sentence editing (Shah et al., 2020; Iso et al., 2020), text improvement (Tanaka et al., 2009), and comment updating based on source code changes (Panthaplackel et al., 2020). In this work, we show that headline generation models can benefit from access to the past state of the article. Our proposed model, H 1 + B edits (change only), can generate headline predictions that are statistically tied with gold headlines in terms of factuality, while making fewer unnecessary edits. By releasing the HREN dataset, we hope to encourage the community to produce higher quality tools for aiding journalists, as well as encourage research in NLP over dynamic texts. Sheena Panthaplackel receives support from Bloomberg's Data Science Ph.D. Fellowship Program. We would like to thank Alex Spangher for sharing an early version of the NewsEdits corpus. We would also like to thank the Bloomberg AI group and Lina Vourgidou for early feedback on this project, and illuminating conversations on the application of machine learning and natural language processing to the newsroom. Finally, we thank the reviewers for their constructive feedback.
objective abstain method result objective abstain abstain other abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method objective objective objective objective method abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method method method abstain abstain method abstain abstain abstain other abstain method method abstain method abstain method abstain method abstain abstain method abstain abstain abstain abstain method abstain method abstain method method abstain abstain other abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain result abstain abstain result result abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain result abstain abstain abstain abstain abstain method method abstain method result abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain result other other other other abstain other objective other objective method other other result objective method other other other other other