Generalized zero-shot text classification aims to classify textual instances from both previously seen classes and incrementally emerging unseen classes. Most existing methods generalize poorly since the learned parameters are only optimal for seen classes rather than for both classes, and the parameters keep stationary in predicting procedures. To address these challenges, we propose a novel Learn to Adapt (LTA) network using a variant meta-learning framework. Specifically, LTA trains an adaptive classifier by using both seen and virtual unseen classes to simulate a generalized zero-shot learning (GZSL) scenario in accordance with the test time, and simultaneously learns to calibrate the class prototypes and sample representations to make the learned parameters adaptive to incoming unseen classes. We claim that the proposed model is capable of representing all prototypes and samples from both classes to a more consistent distribution in a global space. Extensive experiments on five text classification datasets show that our model outperforms several competitive previous approaches by large margins. The code and the whole datasets are available at https://github.com/Quareia/LTA . Text classification plays an important role in many natural language processing (NLP) applications, such as question classification, news categorization, user intent classification and so on (Minaee et al., 2021). Although a wide variety of methods have been proved successful in supervised text classification, they often break down when applied to make predictions for incrementally emerging classes without labeled training data (Pourpanah et al., 2020). Unlike zero-shot learning (ZSL) that aims to classify unseen class instances at test time (Romera-Paredes and Torr, 2015; Wang et al., 2019), generalized zero-shot learning (GZSL), which we focus  Corresponding author. on in this work, aims to classify text samples from both previous seen and emerging novel classes. Since there is a strong bias towards seen classes (Xian et al., 2019a), GZSL is a more challenging yet critical problem. Previously methods mainly focus on transductive approaches for generalized zero-shot text classification. Rios and Kavuluru (2018) use a graph convolution network to enhance the unseen class label embeddings. Zhang et al. (2019) and Song et al. (2020) generate illusion feature embeddings for unseen classes based on side information, i.e. , class-level attributes or text description. More recently, Ye et al. (2020) use reinforced self-training methods to leverage unlabeled data during training. With the assumption that no knowledge about unseen categories is available during the model learning phase, researchers resort to inductive approaches to handle generalized zero-shot text classification. ReCapsNet (Liu et al., 2019) uses a dimensional attention-based intent capsule network and constructs zero-shot class prototypes by similarity matrix transformation. SEG (Yan et al., 2020) exploits an outlier detection approach that can be directly applied on ReCapsNet, which discriminates the domain first, then outputs the final class label. However, the existing methods still have two key limitations. Firstly, while the goal of these methods is to transfer beneficial knowledge for unseen classes, these models merely learn optimal parameters by minimizing the loss of instances from seen classes, regardless of explicitly calibrating the predictions on unseen classes. Therefore, domain bias problem (Xian et al., 2019a) towards seen classes is not fairly resolved. Secondly, although some of them take into account the inter-class relationship when constructing prototypes for unseen classes (Liu et al., 2019), the models keep static no matter what different new classes emerge in future applications. As a result, these models show a large quality gap between instances from seen classes 517 and from emerging unseen classes. To address these problems, motivated by the success of meta-learning in the few-shot learning task (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018; Finn et al., 2017), we present a novel Learn to Adapt (LTA) network for generalized zero-shot text classification. Concretely, the proposed LTA learns over multiple learning episodes that mimic GZSL setting explicitly during training, making the learning setting consistent with the test environment and thereby improving generalization. The model notably extends its ability from two views: prototype adaptation and sample adaptation. In each episode, the LTA adjusts the representative prototypes of both seen classes and "fake" unseen classes, with the assumption that unseen classes will help in calibrating representation of seen ones and thereby enable the model to learn the class-sensitive representations. The updating for all prototypes is then used to generate a set of calibration parameters, called semantic components, to guide the adaptation of sample embeddings, which is designed to compensate for the shrinking features (Chen et al., 2018) that are ignored during training if they are not discriminating for seen classes, but could be critical for recognizing unseen classes. The refined sample embeddings are then classified based on similarity scores with all adapted class prototypes. The same setting can be directly applied in testing, where the LTA executes class prediction and adapts the learned model rationally in an on-the-fly manner. In summary, our contributions include: (i) We propose a novel Learn to Adapt (LTA) network for generalized zero-shot text classification which is capable of adapting incrementally between seen classes and emerging unseen classes at test time. (ii) We propose a methodology for calibrating both prototypes and sample embeddings to deduce a global representation space, efficiently avoiding over-fitting on seen classes. (iii) Experimental results on five generalized zero-shot text classification datasets show that our method outperforms previous methods with a large margin. Generalized Zero-Shot Learning The challenge of zero-shot learning (ZSL) has been the focus of attention in recent years, especially in the applications of image classification (Socher et al., 2013; Xian et al., 2019a; Wang et al., 2019), intent detection detection (Xia et al., 2018; Liu et al., 2019; Yan et al., 2020), and question classification (Fu et al., 2018). Different from ZSL, generalized zero-shot learning (GZSL) that attempts to categorize instances from both seen and unseen classes is a more realistic condition that matches with practical applications. For example, a question classifier for a question answering system has to classify not only the questions ever asked but also new questions incrementally emerging from the users. There are two key issues that GZSL has to address: (1) how to incrementally learn beneficial knowledge for unseen classes from seen ones, and (2) how to tackle the domain bias caused by the extremely imbalanced data of seen and unseen domains. To alleviate the first issue, some of the earliest works on ZSL attempt to learn a matching model between instance embedding and class prototype embeddings represented by extra information including class-level attribute, text description, or their combinations (Frome et al., 2013; Nam et al., 2016; Zhu et al., 2019; Xia et al., 2018). In a similar vein, other methods (Wang et al., 2018; Rios and Kavuluru, 2018; Si et al., 2021) also investigate the semantic relationship between the side information for obtaining better prototype representation. Nevertheless, these models are trained using data from seen classes and fail to incrementally adapt to emerging new classes. The key problem of the second issue is that the model is trained with data from the seen classes and the parameters are actually optimized on the seen domain, thus they are not aware of unseen classes. Assuming the extra information about unseen classes is available, another prominent approach attempts to use generative models to generate virtual samples or features for unseen domains (Xian et al., 2018; Schnfeld et al., 2019; Zhang et al., 2019; Song et al., 2020). By using synthesized samples, the generative approaches can convert GZSL problem to the conventional supervised learning problem where biases towards seen classes are largely alleviated. Additionally, studies also extend to exploit the unlabeled data for unseen classes (Xian et al., 2019b; Rahman et al., 2019; Ye et al., 2020). However, these models assume that they have access to the extra information about the unseen classes, which is not very realistic since often neither the test data nor their label descriptions is available at the training phase (as supposed in 518 this work). In contrast, our model can involve all classes (seen and unseen) jointly during inference, essentially it is trained towards continuous generalization for new classes, hence it is capable to adapt to incoming new classes dynamically. Episode-Based Training in GZSL Our approach is primarily based on the episodic training paradigm that has been widely used in few-shot learning (FSL) (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018). The primitive goal of episodic training is to quickly learn a meta-task from sampled classes. A particular advantage of episodic training is that, by constructing meta-tasks, the setting of training is consistent with that of testing, which is essential for classification problems. Studies extend to exploit episodic training in the "generalized" settings. Verma et al. (2020) constructs model-agnostic meta-tasks to train generative models on GZSL. In addition, Gidaris and Komodakis (2018) utilizes weight generators to update unseen prototypes in generalized FSL(GFSL). Subsequently, to update both seen and unseen prototypes, Ye et al. (2021) exploits attention mechanism while Shi et al. (2020) takes advantage of graph neural networks in GFSL. Yu et al. (2020) use a generative network to generate unseen prototypes in GZSL. These methods only consider the prototype adaptation while the sample embeddings are still static whatever the unseen classes are. Additionally, Bao et al. (2020) uses distributional signatures to update sample embeddings in GFSL. Considering that distributional signatures can be equal for two different tasks, our method uses a novel semantic update extractor to update samples following the prototype adaptation rather than statistical information. A compelling property of our method is that it tackles knowledge transferring and domain bias simultaneously in an episodic training framework by adapting both prototypes and sample embeddings, and draws a fast adaptation to the novel classes without the cost of dramatic damage in discriminating the seen classes. Formally, let Y s = { y s 1 , ..., y sC s } and Y u = { y u 1 , ..., y uC u } denote C s seen classes and C u unseen classes respectively, and Y = Y s  Y u denote the global label space with Y s  Y u = Suppose we have a collection of training samples D s = { ( x sj , y sj , a sj ) } Mj =1 , that consists of M samples from C s seen classes, where x sj  X s represents j -th text utterance, y sj and a sj are its one-hot class label and corresponding class-level textual description, respectively. At the test time, provided with a class description set A u = { a uj } C u j =1 for unseen classes, the GZSL task is to classify the test instance into either a seen or an unseen class. Encoder An textual input x with T words is encoded by a BERT (Devlin et al., 2019) (or any other textual encoder) into a sequence of hidden vectors H = [ h 1 , h 2 , ..., h T ]  RT  d h , where d h is the dimension of the hidden vectors. The text embedding f ( x )  R d h is then obtained by averaging over the T hidden vectors. Training In the training stage, we apply an episodic learning paradigm, which trains the model by simulating multiple generalized zero-shot text classification tasks on seen classes. Following the principle that train and test conditions must match (Vinyals et al., 2016) and recent studies on "gen-eralized" setting (Gidaris and Komodakis, 2018; Shi et al., 2020; Ye et al., 2021; Bao et al., 2020; Verma et al., 2020; Yu et al., 2020), the i -th episode involves an N s -way K -shot learning task for seen classes, denoted as D si = { ( x sj , y sj , a sj ) } N s  K j =1 with K labelled instances for each of the N s classes, which are randomly sampled from the seen data D s , and a N u -way K -shot learning task for "fake" unseen classes, denoted as D ui = { ( x uj , y uj , a uj ) } N u  K j =1 which is also from D s , with N s + N u  C s . More precisely, let Y si and Y ui denote the sampled seen class space and sampled "fake" unseen class space respectively, with Y si  Y s , Y ui  Y s , and Y si  Y ui =  . For a new query instance x , the generalized zero-shot learning model performs:  y = arg max y  { Y si Y ui } p ( y | x, D si , D ui ) (1) The model has to maintain a globally consistent joint class prototype space as well as dynamic adaptation to unseen classes with zero labeled instances. In this end, we design a Learn to Adapt (LTA) network which first introduces a pre-trained and learnable look-up table S to store embeddings of the seen prototypes, and obtain the "fake" seen classes S i from S . The "fake" unseen prototypes 519   word-level encoder   mean pooling TransformerEncoder Layer Residual + Semantic Update Extractor word-level encoder  weighted pooling seenclasses unseenclasses samples Prototype embedding Sample embedding Before adaptation Prototype adaptation Sample adaptation Figure 1: Illustration of the proposed LTA framework. are encoded into a matrix U i with a BERT encoder using "fake" unseen class descriptions. Then the S i and the U i are concatenated and fed into a transformer encoder layer to explicitly calibrate the seen prototype space and unseen prototype space. Meanwhile, a matrix of semantic components C is generated conditioned on the updating of the prototypes. With the belief that the instance feature space should be also calibrated according to the prototypes in an on-the-fly manner, C is further used for updating the feature embedding output by the same encoder. The proposed LTA network first introduces a learnable look-up table S  RC s  d h from which to extract the "fake" seen prototypes S i  R ( C s  N u )  d h on demand. Following Gidaris and Komodakis (2018); Ye et al. (2021); Shi et al. (2020), the S is firstly initialized by the prototypes trained using a supervised metric learning classifier on seen classes. The detail of the supervised metric learning classifier will be described in the experiment section. We claim that this initialization step will reduce the variance caused by the sampling episode sequences. The "fake" unseen prototypes U i is produced by the BERT encoder f (  ) using their corresponding class descriptions: U i = [ f ( a y )] y Y ui  RN u  d h . Then the joint prototype matrix R is obtained by concatenating S i and U i , R = [ S i , U i ]  RC s  d h , with r k as the k -th prototype. Then R is fed into a single Transformer encoder layer (Vaswani et al., 2017) to explicitly model the updates for both seen prototypes and novel prototypes: Z = TransformerEncoder ( R ) = Concat ( head 1 , ..., head h ) W o where head i = Softmax ( RW q W k R d h ) RW v (2) R = R + Z (3) where Z  RC s  d h highlights the adjustment after mutual reflections, W o , W q , W k , W v  R d h  d h are trainable parameters, and the updated prototypes R  RC s  d h is regarded as the calibrated representative prototypes of both seen and unseen categories, with r k as the adjusted k -th prototype. The self-attentions used in Transformer is agile to capture the inter-class relationship of seen and unseen classes and thereby it is beneficial to derive globally discriminative prototypes. The prototype adaptations simultaneously update both seen and unseen classes, which enables the model to represent and discriminate the newly incoming categories in an on-the-fly manner. As been discussed in (Chen et al., 2018), the zero-shot learning tasks are prone to produce semantics loss, where some features would be discarded during training if they are not discriminating for seen classes, but critical for recognizing unseen classes. We observe that the similar problem is exacerbated in GZSL task due to the extreme unbalance between seen and unseen classes. We tackle this problem by introducing sample adaptation following the trajectories of prototypes adaptation. In concrete, we apply a semantic update extractor via attention mechanism to capture synchronous updating of the prototypes: F = ZW 1 (4) A = Softmax ( W 3 ReLU ( W 2 FT )) (5) C = AF (6) where W 1  R d h  d h , W 2  R d a  d h , W 3  R d r  d a are trainable parameters, A denotes the attention weight matrix and C  R d r  d h extracts different semantic components with c l as its l -th semantic components. To offset the semantic loss mentioned above, we use these semantic components to guide the adaptation of sample embeddings. Concretely, we compare the attention score for each h t to get the most related semantic adjustment and reconstruct the contribution of each word-level feature: e t = Softmax (  max l ( h t c l  h t  c l  )) (7) g ( x ) = T (cid:88) t =1 e t h t (8) where the self-attention weight e t is used to reweight the t -th word of sample x to be classified, and  is a learnable temperature scalar to control the differentiation of Softmax scores (Gidaris and Komodakis, 2018). In this way, the different attention weights discriminate the importance of words rather than averaging them. One notable reason of choosing of the above feature-level calibration is that, in classification task, the encoder is trained to produce feature embedding that collapses to its ground-truth prototype, therefore the adjustment of feature embedding should cater to the adjustment of a reliable global prototype space. In addition, since this calibration is applied after the encoding, it reduces the complicated parameter tuning for a massive encoder ( e.g. , BERT), which elegantly helps the GZSL task to fast adapt to the incoming test instances. where s ( a, b ) =   ab  a  b  is cosine similarity with a learnable temperature scalar  . Finally the model is trained by minimizing the losses across N episodes: L = 1 N (cid:88) i L i (10) where L i is the loss of the i -th episode: L i =  1 ( N s + N u ) K (cid:88) ( x,y,a ) D si D ui log p ( y = y | x ) (11) The training process is summarized in Algorithm 1. Algorithm 1: LTA training algorithm. Intent Classification Datasets. We collect four intent classification datasets. (1) SNIPS-SLU (Coucke et al., 2018), a widely used benchmark for English GZSL intent detection with 5 seen intents and 2 unseen intents. (2) SMP-18 (Zhang et al., 521 2017), a Chinese dialogue corpus for user intent detection with 24 seen intents and 6 unseen intents. (3) ATIS (Hemphill et al., 1990), an English airline travel domain dataset, from which we extract 17 intents with at least 5 samples, and split them into 12 seen intents and 5 unseen intents. (4) CLINC (Larson et al., 2019) is a recently published intent detection dataset includes 22,500 in-scope queries covering 150 intent classes from 10 domains. We randomly split them into 120 seen intents and 30 unseen intents. Question Classification Dataset. In order to draw a comprehensive analysis of the proposed method, we construct a question classification task from the Quora Question Pairs dataset 1 , which is aimed to identify duplicate questions. We collect questions with at least 5 duplicate samples into classes. In each class, we choose the question with minimum words as the label description, called the standard question, which is widely used in real-world question-answering systems (Sakata et al., 2019). Table 1 summarizes all datasets statistics. It is worth to note that intents in ATIS are highly unbalanced with flight accounts for about 87% of training data. Dataset Settings. Following (Siddique et al., 2021), we randomly sample seen and unseen classes for 10 runs instead of manual selection used in (Yan et al., 2020), which leads to more fair results because every class could be unseen class. We randomly take 70% samples of each seen class as the training set and the remaining 30% as the seen test, and take all the samples of unseen classes as the unseen test. All the textual labels of the same class are regarded as the description for this class. Supervised Learning Methods. To show the performances on seen classes with supervised learning instead of GZSL setting, we use (1) BiLSTM (Schuster and Paliwal, 1997) and (2) BERT (De-vlin et al., 2019) as the encoder with a linear Softmax classifier, which only requires samples and one-hot label. Metric Learning Methods. Metric-based embedding methods are commonly used as baselines for GZSL. Thus we introduce three different metric learning methods: (1) EucSoftmax : We adapt (Snell et al., 2017) that uses squared Euclidean distance as the metric and Softmax classifies; (2) Zero-shot DNN : We adapt (Kumar et al., 2017) that uses squared Euclidean distance and triplet loss to maintain a margin for different classes. We choose the label embedding (prototype) as the anchor and the closest sample as negative sample in each triplet tuple; (3) CosT : We adapt (Gidaris and Komodakis, 2018) which uses cosine distance as the metric with a learnable temperature scalar. SOTA Methods. We also compare our model with two recent state-of-the-art (SOTA) methods: (1) ReCapsNet (Liu et al., 2019) uses a dimensional attention-based intent capsule network and a matrix transformation method for GZSL. (2) SEG (Yan et al., 2020) is an outlier detection approach that can be directly applied on ReCapsNet. SEG acts as a domain discriminator which first determines whether a test sample belongs to seen classes or unseen classes and then classifies in their own domain. RIDE (Siddique et al., 2021) is not considered because they use outer knowledge that is not available in our settings, and they limit the intent labels to only two components "Action" and "Object". Evaluation Metrics. We basically use accuracy (Acc) to estimate the performances on seen and unseen test sets. Besides, we adopt Macro-F1 (F1) rather than Micro-F1 to better evaluate the performances on imbalanced and few-shot datasets, because Macro-F1 gives the average weight of F1 scores for each class. For overall assessments, we adopt the widely used Harmonic Mean (HM) of 522 Table 2: Results (in %) on four intent benchmarks. Acc and F1 on seen and unseen test sets rather than the overall metrics on the whole test set, because the overall metrics are disturbed by the ratio of seen and unseen test set sizes. Implementation Details. We use the pretrained BERT-base encoder with d h = 768 on intent classification datasets and BiLSTM with d h = 128 hidden vector size each direction on Quora dataset. The scalars of our model is set to be  = 10 . 0 ,  = 10 . 0 , d a = d h , which is trained via Adam (Kingma and Ba, 2015) optimizer, with learning rates 10  5 for BERT encoder, 10  4 for BiLSTM encoder and 10  3 for the other parameters. We use h = 4 heads Transformer encoder layer in prototype adaptation. During training, in order to treat seen classes and unseen classes as equal, we set N s i = N u i in every meta-test set, and we set K = 5 and N s i = N u i = [2 , 2 , 2 , 10 , 20] , d r = [4 , 16 , 32 , 64 , 64] for SNIPS-NLU, SMP-18, ATIS, CLINC and Quora datasets, respectively. We also conduct an ablation study to investigate the effectiveness of each proposed component. As depicted in Table 2 and Table 3, "w / o Init" refers to the model that randomly initializes R rather than pretrained prototypes. "w / o SA" refers to the model that only uses prototype adaptation without "sample adaptation". "w / o A" means none of the adaptation steps is applied. The results on four intent datasets and Quora dataset are given in Table 2 and Table 3, respectively. It is observed that our proposed methods achieve the overall best performances compared to baselines. Detailed and interesting observations can also be derived from the results: (1) The metric-learning methods as the basic baselines, achieve compa-523 timeoil change when tire change proto adaptation rable results on Seen Test for all datasets. However, they all suffer from the domain bias problem and the performance drops with a large margin on Unseen Test , where the prediction is complicated due to zero-shot scenarios. (2) The performances on SNIPS-NLU and SMP-18 of ReCapsNet and SEG are worse than those in their original paper although we use the open-source codes in our experiments, that is because we randomly split the test unseen classes thus making it more challenging. Besides, these methods fail to recognize unseen samples well on datasets with a large scale of categories, yielding worse 0% Acc and F1 on Quora. The most likely reason is that ReCapsNet uses label embedding similarities to construct unseen prototypes in capsule network, which imposes a non-trivial computational and memory burden. (3) Our method shows its privilege for all datasets. In particular, with the help of continuous adapting ability, it observes a smaller gap between seen and unseen domains, which proves the adaptation on the testing phase effectively works. Although the performance on seen domain drops sightly, LTA outperforms the competitive metric-learning baselines by 9.54% HM Acc and 12.90% HM F1 averagely on the whole datasets, indicating that our model fairly balances the seen and unseen classes. Ablation Study. To better understand the contribution of each component of our method, we explore three variants of LTA. We can observe that LTA with both prototype adaptation and sample adaptation outperforms those without any adaptation step in all cases. The "LTA w / o Init" has relatively stable performances. Note that "LTA w / o SA" with only prototype adaptation achieves worse performance compared to "LTA w / o A" on SNIPS-NLU, SMP-18 and CLINC. It indicates that 0 10 20 30 40 50 60 70 #Unseen Classes 45 50 55 60 65 70 HMA cc u r a c y ( % ) MetricLTA w/o A LTA w/o SA LTA the single prototype adaptation step may cause negative transfer, further illustrating the importance of the sample adaptation. As the partition of seen and unseen classes is fixed in previous experiments, to study the robustness of the proposed adaptation method, we conduct the experiment across unseen class sets of different scales on CLINC dataset. Specifically, we select 70 classes as seen classes and 10 classes as validating unseen classes. The number of testing unseen classes is varied from 1 to 70, which are randomly sampled from the remaining 70 classes. Each experiment is repeated 50 times with different sampling sets for a more stable result. Figure 3 (a) shows the HM accuracy on all classes as the number of the unseen classes increases. We can see that our LTA model outperforms the metric learning baseline and ablation models in all cases, where the improved performance is mainly attributed to the improvements on unseen classes as shown in Figure 3 (b). These results suggest that our adaptation method is robust and effective for adapting to increasing new classes as well as improving the overall performance of all classes. To demonstrate how our adaptation method works, we further visualize the encoded representation via PCA in Figure 2. When there is no unseen class, seen classes (yellow and red) is discriminative enough. But when the new class "tire change" (purple) comes, it is ambiguous with class "oil change when" (red). We observe that the seen and unseen class prototypes are updated to be far away from each other after prototype adaptation as shown in (a), which eases the domain bias problem. However, the performance is unsatisfactory since the sample representations are still not discriminative no matter how the prototype updates. As we can see, with the sample adaptation as shown in (b), the sample representations are independently clustered by the adapted prototypes and easy to be distinguished. To further study how the sample adaptation works, we select a representative case "when is it time for a tire change" and show its attention weights used as calibration parameters in (c). The case is still misclassified after the prototype adaptation due that the common word "time" and "change" also appear in seen classes. After the sample adaptation, however, it can be seen that the word "tire" which is a keyword for classifying, gets the highest attention while the other confusing words do not. This result suggests that calibrating using attention weights helps acquire a prototype-aware representation that guides the sample adaptation. This paper proposed a novel adaptive meta-learning network for generalized zero-shot text classification. The model was trained under a consistent setting with testing. In particular, it efficiently alleviated the bias towards seen classes by utilizing both prototype adaptation and sample adaptation. Experiments on five text classification datasets validated that our model achieved compelling results on both seen classes and unseen classes meanwhile was capable of fast adapting to new classes. We thank the anonymous reviewers for their insightful comments. The research is supported in part by the Major Research Plan of National Natural Science Foundation of China (Grant No.92067202).
abstain abstain objective abstain objective result other abstain abstain abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective objective result other other other other other other other other other other other other method objective method other other other other other other other other other objective objective abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain other abstain abstain abstain abstain method method method other abstain abstain abstain abstain other abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain result result abstain abstain result abstain abstain method method abstain method abstain abstain method abstain method method method abstain abstain method method abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain result abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain result abstain result abstain abstain abstain abstain objective abstain abstain objective other other