In many documents, such as semi-structured webpages, textual semantics are augmented with additional information conveyed using visual elements including layout, font size, and color. Prior work on information extraction from semi-structured websites has required learning an extraction model specific to a given template via either manually labeled or distantly supervised data from that template. In this work, we propose a solution for zero-shot open-domain relation extraction from webpages with a previously unseen template, including from websites with little overlap with existing sources of knowledge for distant supervision and websites in entirely new subject verticals. Our model uses a graph neural network-based approach to build a rich representation of text fields on a webpage and the relationships between them, enabling generalization to new templates. Experiments show this approach provides a 31% F1 gain over a baseline for zero-shot extraction in a new subject vertical. Semi-structured websites offer rich sources of high-quality data across many areas of knowledge (Dong et al., 2014). These websites present information via text that is accompanied by rich visual and layout features that can be generalized beyond a single website. However, most prior work on information extraction (IE) from websites has largely ignored most of these features, instead relying only on HTML features specific to an individual website (Ferrara et al., 2014). This requires training data for every website targeted for extraction, an approach that cannot scale up if training data must be manually created. To circumvent manual data annotation, previous work used a distant supervision process requiring a knowledge base aligned to the website targeted Figure 1: Our zero-shot open-domain information extraction process learns generalizable graph-based representations of how relations are visually presented on semi-structured websites, allowing for training on one vertical (such University sites) and extraction from another (such as Movie sites). for extraction (Gentile et al., 2015; Lockard et al., 2018), including for OpenIE extraction (Banko et al., 2007; Bronzi et al., 2013; Lockard et al., 2019). These methods, however, can only learn a website-specific model based on seed knowledge for the site, but cannot be generalized to the majority of websites with knowledge from new verticals, by long-tail specialists, and in different languages. In this paper, we introduce the task of zero-shot relation extraction from semi-structured websites, in which a learned model is applied to extract from a website that was not represented in its training data (Figure 1). Moreover, we introduce ZEROSHOTCERES , a graph neural network model that encodes semantic textual and visual patterns common across different training websites and can generalize to extract information from documents with never-before-seen templates and topics. Unlike unstructured text, which can be modeled as a sequence, or images, which can be modeled as a two-dimensional grid of pixels, it is not obvious how to operate over the many shapes and sizes of text fields on a semi-structured webpage. We illustrate our intuition using the webpage snippets in Figure 1: Despite their differences, each site uses alignment of relation and object strings, either vertically or horizontally, to help indicate relationships; in addition, relation strings are often more prominent than their objects, either in size or boldness. Such features are semantically meaningful to readers and often consistent from site to site; thus, encoding them into the representation of webpages will allow us to generalize to unseen sites. Our model, ZEROSHOTCERES , encodes these diverse feature types in a graph representation in which each text field becomes a node in a graph, connected by edges indicating layout relationships on the page. This abstracts away the details of the page while maintaining the core visual structure presented to the reader. A graph neural network is then applied to produce a new representation of each text field, informed by the surrounding page context. This representation is then used to extract entities and relationships from the document. This allows us to extract not only in the closed-domain setting, but also allows us to conduct OpenIE on websites about entirely new subject verticals not seen during training. Our contributions are threefold: (a) We introduce a graph neural network model for webpage representation that integrates multi-modal information including visual, layout, and textual features, enabling generalization for IE from never-before-seen websites. (b) We propose the first approach to enable Open Information Extraction from semi-structured websites without prior knowledge or training data in the subject vertical. (c) Our method works in both OpenIE and ClosedIE settings. We conduct evaluations showing the effectiveness of the technique and exploring the challenges of zero-shot semi-structured IE, achieving a 31% improvement in F1 compared to an OpenIE baseline. The graph model gives a 26% F1 boost when extracting according to a defined schema (ClosedIE). DOM-based ClosedIE: The conventional approach to extraction from semi-structured websites is wrapper induction (Kushmerick et al., 1997), in which training data for documents from a given template is used to learn a rule-based extractor based on DOM (i.e., HTML) features to apply to other documents of the same template, extracting relations according to a pre-defined ontology (ClosedIE). Since this approach requires training data for each template targeted for extraction, recent work has focused on reducing the manual work needed per site. Fonduer (Wu et al., 2018) provides an interface for easily creating training data, Vertex (Gulhane et al., 2011) uses semi-supervision to minimize the number of labels needed, LODIE (Gentile et al., 2015) and Ceres (Lockard et al., 2018) automatically generate training data based on distant supervision, and DIADEM (Furche et al., 2014) identifies matching rules for specific entity types. DOM-based OpenIE: WEIR (Bronzi et al., 2013) and OpenCeres (Lockard et al., 2019) offer OpenIE approaches to DOM extraction. The latter method uses visual features in a semi-supervised learning setting to identify candidate pairs that are visually similar to known (relation, object) pairs; however, the ultimate extraction model learned is still site-specific and based on DOM features rather than the more generalizable visual or textual features. Pasupat and Liang (2014) present a zero-shot method for extraction from semi-structured webpages, but limit their work to extraction of entities rather than relationships and do not consider visual elements of the page. Multi-modal extraction: The incorporation of visual information into IE was proposed by Aumann et al. (2006), who attempted to learn a fitness function to calculate the visual similarity of a document to one in its training set to extract elements like headlines and authors. Other recent approaches that attempt to address the layout structure of documents are CharGrid (Katti et al., 2018), which represents a document as a two-dimensional grid of characters, RiSER, an extraction technique targeted at templated emails (Kocayusufoglu et al., 2019), and that by Liu et al. (2018), which presents an RNN method for learning DOM-tree rules. However, none of these address the OpenIE setting, which requires understanding the relationship between different text fields on the page. The approaches most similar to ours are Gra-phIE (Qian et al., 2019) and the approach by Liu et al. (2019). Both approaches involve constructing a graph of text fields with edges representing Figure 2: A depiction of the web page representation module (left) and relation classifiers (right). horizontal and vertical adjacency, followed by an application of a GCN. However, neither approach makes use of visual features beyond text field adjacency nor DOM features, and both only consider extraction from a single text field rather than OpenIE. In addition, they show only very limited results on the ability of their model to generalize beyond the templates present in the training set. 3 Problem and Approach Overview 3.1 Zero-shot relation extraction from semi-structured websites We address the problem of extracting entities and the relationships between them as expressed by never-before-seen semi-structured websites. A semi-structured website typically belongs to a subject vertical V , where V is a general field of knowledge such as movies, finance, or sports. A semi-structured website consists of a set of detail pages sharing a similar template, each of which contains a set of facts about a page topic entity e topic . The HTML document w defines a set of text fields T , which the web browser renders as a webpage according to the instructions defined in the HTML and any referenced auxiliary files such as CSS or Javascript. The text fields have both textual and visual features, described in Section 4.2.1. Our goal is to extract (subject, relation, object) knowledge triples, where the subject is e topic , the object is a text field t  T containing the name of an entity (or atomic attribute value), and the relation indicates the relationship between the two entities. For this work, we assume the page topic entity has already been identified, (such as by the method proposed by Lockard et al. (2018) or by using the HTML title tag) and thus limit ourselves to identifying the objects and corresponding relations. We consider the following two settings: Relation Extraction (ClosedIE): Let R define a closed set of relation types, including a special type indicating No Relation. Relation Extraction is the assignment of each text field t to one r i  R , which indicates the relationship between the entity e object mentioned in t and e topic . Open Relation Extraction (OpenIE): Given a pair of text fields ( i, j ) , Open Relation Extraction is a binary prediction of whether i is a relation string indicating a relationship between the entity e object mentioned in j and e topic . Unlike prior work that requires the learning of a model specific to the semi-structured website targeted for extraction, we look at zero-shot extraction. Given a semi-structured website W targeted for extraction, zero-shot extraction is the learning of a model without any use of pages from W during training. We consider two zero-shot settings: Unseen-Website Zero-shot Extraction is the learning of a model without any use of pages from W , but with pages from some other website(s) from vertical V during training. Unseen-Vertical Zero-shot Extraction is the learning of a model without any use of pages from W or of pages from any website with vertical V during training. Figure 2 depicts our approach for zero-shot relation extraction (detailed in Section 5) leveraging a web page representation that will capture the similarities in visual and textual semantics across websites (Section 4). Our web page representation module first converts each page into a layout graph (Section 4.1) that abstracts away the details of the page structure while maintaining the adjacency relationships between text fields. We represent each text field with an initial feature vector of visual and textual attributes. This input is passed into a graph neural network that allows for information to flow between nodes, producing a new text field representation that captures contextual information (Section 4.2). To obtain a web page encoding, we leverage a pre-training step with auxilliary loss function L pre that encourages the model to produce an intermediary representation useful for IE. This is performed via a three-way classification that determines if a text field contains a relation name, the object of some relation, or irrelevant text (Section 4.3). After pre-training, the weights of this GNN are frozen and it can be applied to new pages, with its output used as input into a relation extraction module, optimized with task-specific loss function L task , where the task is either OpenIE or ClosedIE, described in Section 5. The resulting approach minimizes our overall loss LZSCERES , with: LZSCERES = L pre + L task (1) 4 Web Page Encoder The key idea behind our solution is to train webpage representations to capture the fundamental similarities in visual and textual semantics across websites to express relations, objects, and their relationships. The fundamental characteristics we capture, generalizable across templates and verticals, thus allow us to carry over our knowledge across websites and enable zero-shot extraction. There are two key parts in our solution. First, we build a graph to capture the layout relationships in a more abstract form that allows us to more easily learn the common features across different sites such as the fact that relation strings are often to the left or above their objects (Section 4.1). Second, we apply a Graph Neural Network (GNN) to learn representations for each node capturing contextual information about its neighborhood on the webpage (Section 4.2), allowing information to flow through the nodes, providing context (e.g., flowing through Cast to a far-away node Uma Thurman via the closer node Ethan Hawke in Figure 3). This Figure 3: A cropped portion of the detail page from all-movie.com for the film Tape . representation will be useful for relation extraction as described in Section 5. We encode the layout relationships between text fields in the form of a graph, G , consisting of a set of nodes N , each corresponding to a text field, and a set of edges E corresponding to relationships between the text fields. The edges capture three forms of adjacency, as shown in the example in Figure 3: Horizontal: Edges are added when two text fields are horizontal neighbors on the page; that is, they have a shared vertical location and there are no other text fields between them. Vertical: Edges are added when two text fields are vertical neighbors on the page; that is, they have an overlapping horizontal location and there are no other text fields between them. DOM: Edges are added when two text fields are siblings or cousins in the DOM tree; that is, the absolute XPaths identifying their locations differ only at a single index value. To build a representation of each text field that incorporates the surrounding page context, we use Graph Attention Networks (GAT) (Velickovic et al., 2018). The feature vector for each text field (de-scribed below) and the page graph form the input to a GAT, which then produces a new representation for each text field based on the surrounding context in the graph. where N i is the set of neighbors of node i in the graph, and h l  1 j is the representation of node j from the preceding layer; h 0 j indicates the input features for the node. (For each node, we add a self loop to the graph; that is, including i in N i .) W lG is a learned weight matrix applied to the node features for layer l  1 and  is a non-linear function, in our case a ReLU. The attention weight  ij determines how influenced a node's representation is by each of its neighbors, calculated as follows:  ij = exp (cid:16)  (cid:0) a (cid:62) [ W lG h l  1 i ; W lG h l  1 j ] (cid:1)(cid:17) (cid:80) k  N i exp (cid:16)  (cid:0) a (cid:62) [ W lG h l  1 i ; W l g h l  1 k ] (cid:1)(cid:17) , (3) where a is a weight vector applied against the concatenation (represented by  ; ) of the two node's features as transformed by W lG and  is a ReLU. This produces a new contextualized set of features for each node that are informed by the surrounding page context. We describe the original input features for each text field in the next section. For each text field on the page, we produce an initial feature vector containing both visual feature vector V and textual feature vector T . We define the input feature vector h 0 i for text field i as: h 0 i = [ T ( i ); V ( i )] (4) where  ;  represents concatenation. Visual Features: A numeric feature vector is constructed representing the bounding box coordinates of the text field, the height and width of the bounding box, and the font size, along with one-hot features representing the typeface, font weight, font style, color, and text alignment. Textual Features: In ClosedIE, to capture its semantics, the textual content of the text field is processed with a pre-trained BERT (Devlin et al., 2018) model. To produce a representation of the entire text field, we simply average the BERT-Base output for each token in the text field. For OpenIE, since the goal is to generalize to entirely new subject verticals that may contain text not seen during training, only a single textual feature is used 1 : the percent of pages on the site on which the string in the text field appears. This frequency measure helps differentiate relation strings, which are likely to be common, from object strings, which are more likely to be rare. To encourage the GNN weights to capture the features necessary to represent relationships on the page, we use a pre-training step to learn the GNN representation before incorporating it into the extraction model. The pre-training task is a simplified form of the OpenIE task. To speed up training by avoiding the pairwise decisions necessary for OpenIE, we instead perform a multi-class classification of each text field into a class c in the set { Relation, Object, Other } : p (cid:16) c | h li ;  (cid:17) = softmax (cid:16) W pre h li (cid:17) (5) where h li is the output of the GNN for the text field, W pre is a weight matrix, and  comprises WG and W pre . Given a training set with T text fields, each with a ground truth class y prei , we minimize the cross-entropy loss L pre : L pre =  T (cid:88) i =1 log p (cid:16) y prei | h li ,  (cid:17) (6) To discourage overfitting to spurious details in the small number of websites in our training set, we freeze the GNN weights after pre-training and do not update them during the full OpenIE training. After pre-training we discard the linear layer W pre since it is not needed for subsequent steps; instead, we directly use the GNN output h l . Once we have the new representation h lt of each text field t produced by the above GNN process, we can perform our final classification. For OpenIE, the classification decision must be made over a pair of text fields, i and j , the first containing the candidate relation string and the second containing the candidate object string. To 1 This feature is also used during ClosedIE avoid examining all possible pairs of fields, we first apply the candidate pair identification algorithm from Lockard et al. (2019), which filters down to a set of potential pairs based on physical and layout distance between text fields. For each candidate pair, we concatenate the GNN-produced contextual features h l for both text fields with the original features h 0 for both text fields (since some information can be diluted in the GNN), as well as a pairwise feature vector that simply contains the horizontal and vertical distance between the two text fields, and pass them into a binary classifier: r OIEi = FNN (cid:16) [ h 0 i ; h 0 j ; h li ; h lj ; pairwise i,j ] ,  OIE (cid:17) (7) where FNN is a feed-forward neural network with parameters  OIE , ; indicates concatenation, and r OIEi is the predicted probability that the two text fields constitute a (relation, object) pair. We then optimize for cross-entropy loss across training examples T with y OIEi = 1 if the pair is positive: LOIE = T (cid:88) i =1 y OIEi log r OIEi + (cid:0) 1  y OIEi (cid:1) log (cid:0) 1  r OIEi (cid:1) , (8) 5.2 ClosedIE For ClosedIE, we perform a multi-class classification using the contextual representation produced by the GNN ( h li ) along with the original features ( h 0 i ) for text field i : r CIEi = FNN (cid:16) [ h 0 i ; h li ] ,  CIE (cid:17) (9) where FNN is a feed-forward neural network parameterized by  CIE , ; indicates concatenation, and r CIE i is the predicted probability of relation r in set R . We optimize for cross entropy loss LCIE : LCIE =  T (cid:88) i =1 log p (cid:16) y CIEi | h 0 i , h li ,  CIE (cid:17) (10) where y CIEi is the true class for example i . For both ClosedIE and OpenIE we use one hidden layer in the feed-forward network. For both OpenIE and ClosedIE, our primary dataset is the extended version (Lockard et al., 2019) of the SWDE dataset (Hao et al., 2011), which contains gold labels for OpenIE extractions for 21 English-language websites (each with one template) in three subject verticals (Movie, NBA, and University), with between 400 and 2,000 pages per site. We generated ClosedIE labels by converting the OpenIE labels to ClosedIE labels via manual alignment of OpenIE relations between websites, giving a set of 18 relations for the Movie vertical, 14 for NBA, and 13 for University. More information on training data creation and a complete listing of ClosedIE relations is available in the Appendix. We used three SWDE Movie sites (AMCTV, AllMovie, and IMDb) as a development set and did not evaluate on them for the reported results. For each model tested (both our own and the base-lines), we classify the training setting into the following categories indicating the level of vertical or site-specific knowledge used, in decreasing level of difficulty. Level IUnseen-Vertical Zero-shot (OpenIE only): A model is trained on sites from two of the three verticals (e.g. NBA and University) and applied to sites from the other vertical (Movie). This is the hardest case and is important when we wish to extract knowledge from new verticals where we do not have any prior knowledge or annotations. Level IIZero-shot with Vertical Knowledge: A model is trained on all sites but one (spanning Movie, NBA, and University) and then applied to the held-out site. As in cross-validation, experiments are repeated with each site having a turn being held out. It is easier than Level I but is still important for a new website that may not have data overlapping with other websites in the same vertical. For the ClosedIE setting, we train only on in-vertical sites. Level IIISite-specific Knowledge: This is the traditional setting used by two of our baselines where we have seed knowledge overlapping with the website data to allow training a specific model for the website. Whereas Level I-II are both zero-shot settings, Level III is not, as it allows site-specific training data via weak supervision. (We do not present results using full supervision from manual annotations since it is known from prior work (e.g., Gulhane et al. (2011)) that full supervision from the target website yields highly accurate semi-structured extractors; we note that ZSCERES also achieves comparable results (  0 . 95 F1) in this setting. We repeated our experiments 10 times and we report the results averaged across the runs. For OpenIE, we follow the lenient scoring method for SWDE introduced by Lockard et al. (2019), scoring an extraction as correct if the relation string matches any of acceptable surface forms listed by the ground truth for that object. Models are constructed in PyTorch (Paszke et al., 2017), with graph functions implemented in DGL (Wang et al., 2019) and optimization performed using Adam (Kingma and Ba, 2014) and a batch size of 20. For OpenIE, we use a hidden layer size of 25 for the GAT and 100 for the feed-forward layer. For ClosedIE, we use a hidden layer size of 200 for all layers. We use a 2-layer GAT and dropout of 0.25. We obtain visual features by rendering the page using the headless Chrome browser and querying the values using Selenium 2 . Extraction Threshold: Since our zero-shot setting means we cannot use a development set of pages from the target site to tune the decision threshold, we instead set the threshold for each experiment to the value that attains the optimal F1 on the experiments where other sites were held-out. OpenIE Postprocessing Rules: To ensure consistency among the extracted values, we keep only the highest confidence extraction in the case that the same text field is extracted as both a relation and object, or if multiple relations are extracted for the same object. In addition, some pages in the dataset contain relational tables, from which we sometimes extract the column headers as relations with the column contents as objects. While we believe a post-processing step could potentially recover these relational contents from our extractions, the SWDE data does not contain ground truth for such facts. Instead, we apply the heuristics described by (Cafarella et al., 2008) to identify these tables and remove them from our extractions. (:) and assumes they are relation strings, then extracts the text field to the right or below, whichever is closer, as the object. We consider it as Level I knowledge since it requires no training. WEIR (OpenIE) This approach by Bronzi et al. (2013) discovers relations by aligning multiple pages about the same entity. Because it requires sites to be grouped by vertical and uses a gazetteer list of entity names for the alignment, it has Level III knowledge. OpenCeres (OpenIE) This applies the model by Lockard et al. (2019), which requires a knowledge base matching some facts presented on the target website, using Level III knowledge. ZSCERES-FFNN (Feed-forward neural net-work): This model takes the same features and training data as the full ZSCERES model but removes the GNN component, with versions tested with both Level I (ZSCERES-FFNN Unseen-Vertical) and Level II (ZSCERES-FFNN Unseen-Website) knowledge. ZSCERES-GNN : This applies the full model described in Section 4.2, with versions tested with both Level I (ZSCERES-GNN Unseen-Vertical) and Level II (ZSCERES-GNN Unseen-Website) knowledge. Level-I Knowledge: Table 1 shows that ZSCERES is able to extract facts in entirely new subject verticals 31% more accurately than the colon baseline. Across all SWDE sites (micro-averaging across all extractions), ZSCERES-GNN achieves an F1 of 0.45, in comparison with 0.43 for ZSCERES-FFNN, showing that the additional information provided by the page encoder allows for a better representation of the relationships between text fields. By successfully learning general patterns of relational presentation on webpages, ZSCERES-GNN is able to train solely on a set of 16 websites about Movies and NBA players, and then extract from University websites more accurately than the WEIR and OpenCeres systems, which take advantage of Level III knowledge to learn models specific to those University sites. While OpenCeres's rich vertical knowledge allows it to attain better results in Movie and NBA, ZSCERES-GNN still posts much stronger results than the other baselines in these two verticals. Level-II Knowledge: Figure 4 shows that adding the in-vertical sites to the training set (but still withholding the test site) allows the model to achieve performance better than the Level I training set that uses only out-of-vertical data. Table 2 shows the results for ClosedIE extraction. ZSCERES-GNN attains an overall F1 of 0.58 averaged across the three verticals. This significantly outperforms the feed-forward model that did not use the GNN, which attained an F1 of 0.46. While our performance on this dataset is far below Figure 5: Performance on the ClosedIE Movie vertical increases significantly as more sites are added to the training data. the state-of-the-art for semi-structured ClosedIE (above 0.9 for all verticals), prior systems all learn site-specific models based on manual labeling or prior knowledge aligned to the website, while we have only Level II Knowledge available. Figure 5 shows how adding additional training data improves performance in the Movie vertical. It appears that adding additional training sites would further improve the performance. Table 3 shows the contributions of different elements of the model in the OpenIE and ClosedIE settings as calculated on the development set of three sites in the Movie vertical. These ablations show that the GNN helps in both settings, with a larger effect in ClosedIE, which is likely due to sharing the rich information about the text of nearby text fields. Pre-training is important in OpenIE but does not have a significant effect for ClosedIE. This is not surprising given that the pre-training task is closely related to the OpenIE task. Both DOM and spatial adjacency edges contribute to the success of the page layout graph for the GNN. In the ClosedIE setting, the text and layout relationships alone will generally contain sufficient information to make an extraction, while in OpenIE the visual elements (such as whether text is bold or underlined) are a strong source of consistency across websites. OpenIE: To understand what cases our ZSCERESGNN model is missing, we sampled 100 error cases in each vertical from the Unseen-Vertical experiment and manually examined them. Some examples of both erroneous and correct extractions are shown in Table 4 in the Appendix. False positives were largely due to the presence of two different types of n-ary relationships on the page. The first class of errors involving n-ary relationships, making up 43% of all false positives, were where several facts have a multi-way relationship with the page topic, but individually the fields are not meaningful. For example, the NBA site US-AToday includes a Latest notes section with links to several articles relevant to the page topic entity, mentioning the date, headline, and summary. We extract all of these objects with the Latest notes relation, but to obtain meaningful knowledge it would be necessary to additionally associate the correct date, headline, and summary with each other. While we can envision methods for doing this via post-processing, the SWDE benchmark considers these to be errors. In the second class, ZSCERES correctly extracted (relation, object) pairs, but from page sections that contain facts about entities other than the page topic. For example, on the MatchCollege site, a section of Similar Local Colleges contains some of the same relations presented for the page topic, in similar formatting. These types of errors made up another 6% of false positives. Of the remaining errors, 33% were due to the extraction of pairs where the extracted relation did not represent a relationship, while another 14% were due to the extraction of pairs with a correct relation string and incorrect object. Most false negatives occurred in long vertical lists, where some values were extracted, but not all. ClosedIE : False negatives were most likely to occur on long lists of values (such as cast lists), where values toward the bottom of the list were sometimes missed. Recall also suffered on relations where the relation name varied significantly from site to site, or where ambiguity existed. For example, the string Produced by is used by some sites to indicate the producer of the film, while on other sites it indicates the production company. We have introduced a zero-shot method for learning a model for relation extraction from semi-structured documents that generalizes beyond a single document template. Moreover, this approach enables OpenIE extraction from entirely new subject verticals where no prior knowledge is available. By representing a webpage as a graph defined by layout relationship between text fields, with text fields associated with both visual and textual features, we attain a 31% improvement over the baseline for new-vertical OpenIE extraction. Future extensions of this work involve a more general pre-training objective allowing for the learned representations to be useful in many tasks as well as distantly or semi-supervised approaches to benefit from more data. We would like to acknowledge grants from ONR N0001418-1-2826, DARPA N66001-19-2-403, NSF (IIS1616112, IIS1252835), Allen Distinguished Investigator Award, and Sloan Fellowship.
abstain abstain objective objective abstain abstain abstain abstain abstain method abstain abstain method result abstain method method method abstain abstain abstain objective objective result objective abstain objective abstain other other other other other other other other other other other other other other other abstain other other other other abstain abstain method abstain abstain abstain abstain method abstain method abstain method method abstain method abstain abstain method method method method method abstain abstain method abstain abstain abstain method abstain abstain method method other abstain abstain method method abstain abstain method abstain abstain method abstain method method method abstain abstain abstain method method method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective other