Abstract Spatial commonsense, the knowledge about spatial position and relationship between objects (like the relative size of a lion and a girl , and the position of a boy relative to a bicycle when cycling ), is an important part of commonsense knowledge. Although pretrained language models (PLMs) succeed in many NLP tasks, they are shown to be ineffective in spatial commonsense reasoning. Starting from the observation that images are more likely to exhibit spatial commonsense than texts, we explore whether models with visual signals learn more spatial commonsense than text-based PLMs. We propose a spatial commonsense benchmark that focuses on the relative scales of objects, and the positional relationship between people and objects under different actions. We probe PLMs and models with visual signals, including vision-language pretrained models and image synthesis models, on this benchmark, and find that image synthesis models are more capable of learning accurate and consistent spatial knowledge than other models. The spatial knowledge from image synthesis models also helps in natural language understanding tasks that require spatial commonsense. Code and data are available at https://github.com/ xxxiaol/spatial-commonsense . Spatial perception, the ability to detect the spatial position and to infer the relationship between visual stimuli (Donnon et al., 2005; Saj and Baris-nikov, 2015), is basic but important for human beings (Pellegrino et al., 1984). It is of everyday use, from understanding the surrounding environment, like when seeing a woman sitting in a car with her hands on the steering wheel, we know she is probably driving , to processing spatial information and performing reasoning, like navigating  Corresponding author. through a dense forest . We regard the knowledge needed in spatial perception as spatial commonsense. Humans start to develop spatial perception and acquire spatial commonsense from infancy, and apply the commonsense through lifetime (Kuipers et al., 1990; Poole et al., 2006). Although text-based Pretrained Language Models (PLMs) achieve great performance on various commonsense reasoning tasks (Davison et al., 2019; Zhou et al., 2020), they are shown to be ineffective when dealing with spatial commonsense. Zhang et al. (2020) and Aroca-Ouellette et al. (2021) show that current PLMs lack the ability to reason about object scales. Bhagavatula et al. (2020) find that BERT (Devlin et al., 2019) under-performs on instances involving spatial locations. The struggle of PLMs with spatial commonsense is partly because spatial commonsense is rarely expressed explicitly in texts. We may write sentences like lions are big animals , but we seldom explicitly mention how big lions are; we also rarely write about the spatial relationship between a boy and a bicycle when he is cycling. Spatial commonsense is exhibited in images more commonly (Cui et al., 2020). As shown in Figure 1, the two Wikipedia articles provide little 2365 spatial information, but a picture of a lion and a girl provides a reference to the size of a lion; and a painting of a boy riding a bicycle depicts that he sits on the bicycle. Hence, a natural idea is to elicit spatial knowledge from models with visual signals. We first study whether models with visual signals learn more spatial knowledge than text-only models . We select Vision-Language PreTrained Models (VL-PTMs) and Image Synthesis Models (ISMs) for investigation. VL-PTMs encode texts and images together, fusing their features to deal with downstream tasks. ISMs take texts as input, and generate images based on the texts. To evaluate the spatial commonsense in PLMs and models with visual signals, we design a benchmark that involves two subtasks: 1) comparing sizes and heights of different objects (like a lion and a girl ), and 2) determining the positional relationship between a person and an object when a certain action happens (like a boy's position when riding a bicycle ). The subtasks are designed to examine the model's capability to master two kinds of spatial commonsense: understanding spatial scales, and the relationship between surrounding objects and ourselves. As shown in Figure 2, we probe models with text prompts on this benchmark. We feed text prompts with masks to PLMs and VL-PTMs, and take the possible word with the highest probability as their prediction. We probe ISMs in a similar way: we first feed the text prompts to ISMs and then evaluate the generated images. We evaluate the images with two methods: automatically comparing bounding boxes of objects and conducting human evaluation. Results show that models with visual signals learn more accurate spatial commonsense than PLMs. Besides the performance comparison, we are also interested in how is the quality of spatial commonsense learned by different models? We investigate how consistent the spatial knowledge learnt by a model is, like whether it can manifest a lion is larger than a girl and a girl is smaller than a lion simultaneously; and to what extent models can generalize the knowledge when uncommon scenarios like an enchantress lights the sparkler appear. We observe that ISMs are capable of generating consistent spatial knowledge and the performance is robust in uncommon scenarios. The following problem is how to benefit natural language understanding tasks with the spatial knowledge from ISMs? We investigate this in the question answering scenario. Take a question like A boy is riding a bicycle. Is he on the bicycle? We generate an image about the question context a boy who is riding a bicycle with a text prompt using ISMs, and feed both the question and the generated image into vision-language models to predict an answer. This framework outperforms strong question answering models pretrained on texts only. While this is a simplified scenario of spatial commonsense reasoning, it manifests a possible way to employ the spatial knowledge learned by ISMs in natural language understanding. Motivated by the observation that images contain more spatial commonsense than texts, we 1) design a framework, including the data and probing methods, to compare the spatial reasoning ability of models with different modalities; 2) propose methods to evaluate the quality of learned spatial commonsense, and find that models with visual signals, especially ISMs, learn more precise and robust spatial knowledge than PLMs; and 3) demonstrate the improvement in spatial commonsense question answering with the help of visual models. Object Scales. Bagherinezhad et al. (2016) build a dataset for objects' size comparison, and Elazar et al. (2019) provide distributional information about objects' lengths. Forbes and Choi (2017) also involve spatial comparison but are criticized for ill-defined comparison (Elazar et al., 2019). Aroca-Ouellette et al. (2021) design a physical reasoning dataset that requires not only spatial commonsense but also a complex reasoning process, which is extremely challenging for existing models. We choose the formulation of object comparison in pairs as this kind of knowledge is easy to be probed from different models. Spatial Relationship. Collell et al. (2018) introduce a dataset of spatial templates for objects under different relations, but the spatial relations are represented as relative positions of bounding boxes, which are hard to express in language. Yatskar et al. (2016) extract statements of spatial relationship from object co-occurrences in MS-COCO (Lin et al., 2014). Mirzaee et al. (2021) design a textual spatial reasoning benchmark, and Johnson et al. (2017) and Hudson and Manning (2019) involve spatial reasoning in images, but they focus on logical reasoning rather than commonsense. Contrast 2366 A sofa is [MASK] than a mountain. to them, we build a dataset to describe the spatial relationship between people and objects in certain actions with preposition words. Early attempts in probing PLMs (Liu et al., 2019a; Hewitt and Manning, 2019) mainly train a classifier on the task of interest with the encoded representations. However, the probing performance is highly influenced by the probe design (Pimentel et al., 2020), thus is hard to reflect the ability of PLMs. Recently, prompt-based methods (Petroni et al., 2019; Zhou et al., 2020) become more prevalent to study what knowledge PLMs already encode. PLMs take a prompt as input, and generate the continuation (for generative PLMs) or predict masked words (for discriminative PLMs). This does not need additional training, and only a small development set is used to choose optimal prompts and answers (Jiang et al., 2020). In this work, we probe PLMs and VL-PTMs with prompts. Prompt-based methods are also used in model training (Schick and Schtze, 2021; Zhou et al., 2021), while we focus on the knowledge already learned by models. Basaj et al. (2021); Oleszkiewicz et al. (2021) try to apply the probing methods into the computer vision domain, but they focus on probing representations of visual models. In contrast, we probe ISMs by evaluating the generated images. Size and Height. Inspired by the cognitive discovery (Hersh and Caramazza, 1976) that people tend to categorize objects scales into fuzzy sets, we select 25 common objects in daily life, and categorize them into 5 groups as shown in Table 1a to construct the dataset for size comparison. Typical Size 1 ant, coin, nut, bullet, dice 2 bird, cup, shell, bottle, wallet 3 tyre, chair, microwave, dog, suitcase 4 human, sofa, bookshelf, tiger, bed 5 house, cinema, mountain, truck, plane objects in the former group are smaller than those in the latter group. We form 250 pairs of objects from different groups, like  ant, bird  , where the first object is smaller than the second in commonsense. Models are asked to compare the size of objects in pairs. To avoid an imbalance of answer distribution, we also consider the reversed pairs like  bird, ant  , so there are 500 instances in total. The dataset for comparing objects' heights is constructed similarly, as shown in Table 1b. We also form 500 instances with the objects. The comparison between objects is validated by 5 human annotators for both datasets. Positional Relationship. The positional relationship dataset consists of human actions regarding objects and the most likely positional relation between the person and the object. We consider four types of positional relations: above, below, inside, beside , as they do not overlap with each other. We select common objects, and write actions between people and the objects. The actions do not contain prepositions, like sit on the chair . Each ob-2367 A man <verb> the car. ject is accompanied by two actions with different positional relations. Take Figure 3 as an example. The man is beside the car when washing the car, whereas he is inside the car when driving it. Therefore, the relation cannot be easily inferred from collocations between the person and the object. The dataset contains 224 instances, validated by 5 annotators. We probe PLMs and VL-PTMs through masked word prediction. Given a text prompt with masks and a set of possible words, a model calculates the probability of each possible word filling the masked position. The word with the highest possibility is regarded as the prediction. We also probe ISMs through text prompts. The input is a piece of descriptive text, and the output is the image generated by an ISM. We assess the image with two methods as described in 3.3. PLMs are found to perform poorly in scenarios involving complex reasoning over spatial knowledge (Aroca-Ouellette et al., 2021), and we want to investigate whether they even fail in early stages, like whether they have learned spatial knowledge. So we probe models with simple tasks. In the subtask of size and height, the prompt for PLMs and VL-PTMs is in the form of O a is [MASK] than O b , where  O a , O b  is an object pair. The possible answer set is { larger, smaller } for size and { taller, shorter } for height. The prompt for ISMs is in the form of O a and O b , and the objects in generated images are compared for size and height. In the subtask of positional relationship, the prompt for PLMs and VL-PTMs contains an event scenario and a masked token for the positional relationship, like A woman washes the car. She is [MASK] the car. The possible answer set is { above, below, inside, beside } . The prompt for ISMs describes the scenario only, like A woman washes the car. We assess the images generated by ISMs with two methods. We first use the spatial information of bounding boxes (referred to as ISM (Box)). For each object mentioned in the prompt, we select the classified bounding box with the highest confidence. To mitigate the effect of viewpoint (an object closer to the camera may appear larger in the image), we compute the average depth of the box as the object's depth. We use the object detector from Zhang et al. (2021), and the depth estimator from Godard et al. (2019). When probing the relative size, we compare area  depth 2 of the two objects' boxes; and when probing the relative height, we compare height  depth . When classifying positional relations, we use the mapping rules between spatial relations and image regions from Visual Dependency Grammar (VDG) (Elliott and Keller, 2013). We list the rules in Appendix A.1. Some generated images are vague while object detection models are trained to process clear pictures, so a number of objects are not recognized. To precisely assess the generated images, we conduct human evaluation on all images (referred to as ISM (Human)). Annotators are asked to compare the size/height of the objects in the images (for the first subtask) and classify the positional relationship between the person and the object (for the second subtask). Each image is evaluated by two annotators, and the average performance is reported. Specifically, we report the accuracy and macro F1 between models' predictions and correct answers. Besides the performance of ISMs on the subset of recognized instances, we also report the performance on the full dataset, giving the unrecognized instances a random guess. We take BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019b) as examples of text-only PLMs. For VL-PTMs, we choose VinVL (Zhang et al., 2021), which performs well in various vision-language tasks. It uses a transformer-based backbone and is pretrained on various vision-language datasets including image caption datasets, visual QA datasets, etc. As it preserves the masked word prediction objective like PLMs, it can also be probed with prompts. We choose VQGAN+CLIP 1 1 Originated by Ryan Murdoch, @advadnoun on Twitter. Implementation details are in Appendix A.2. (a) Comparing sizes of objects. Both objects are recognized by the object detection model in 15% images and are recognized by humans in 86% images. as a representative of ISMs. It uses CLIP (Rad-ford et al., 2021) to guide VQGAN (Esser et al., 2021) to generate images that best match the given text. To make a fair comparison regarding model size, we select BERT-large, RoBERTa-large, and VinVL-large. We use VQGAN with codebook size Z = 16384 and downsampling factor f = 16 , and CLIP with ViT-B/32 (Dosovitskiy et al., 2020) architecture. All four models are of similar sizes. As language models are sensitive to the expressions in probing (Liu et al., 2021) (like changing an answer choice from larger to bigger , the predictions of BERT may differ a lot), we generate new prompts and answers based on those originally designed in the benchmark, and search for the optimal ones for PLMs and VL-PTMs. Similar to Jiang et al. (2020), we use back-translation to generate 10 candidates for prompts and answers, and filter out the repeated ones. To select prompts and answers, we split the dataset into 5 folds, where different folds do not share the same objects. For each run, one fold is used as the development set to choose the best candidate, and the model is probed on other folds with the chosen prompt. We report average performance of 5 runs. Size and Height. Table 2 reports the probing performance of comparing the scales of objects. We also demonstrate probing results on RelativeSize (Bagherinezhad et al., 2016) in Appendix B. We observe that PLMs perform similarly. Even the A house and a bird A bottle and a bookshelf A plane and a bullet Size Height A bird and a trash can A trash can and a theatre An apartment and a horse Human Box Human Box Human Box Human Box Human Box Human Box Human Box Human Box Human Box Human Box Human Box Human Box Figure 4: Images generated by ISM in scale comparison. best PLMs are slightly better than random guesses, indicating they are ineffective in predicting object scales. Although RoBERTa is trained on more texts and assumed to encode more knowledge, its performance is similar to BERT's. It shows that PLMs do not learn much spatial commonsense from texts even if the pretrained corpus greatly increases. With the help of visual features in pretraining, VinVL greatly outperforms PLMs. ISM (Box), which simply compares bounding boxes in images generated by the ISM, also outperforms PLMs. Since only a small portion of instances are recognized with bounding boxes, if we only consider the predictions on these instances, the gap between ISM (Box) and PLMs is more than 15%. These 2369 Model Acc (avg. /  ) F1 (avg. /  ) BERT 26.1 / 4.15 19.0 / 5.20 RoBERTa 31.0 / 15.4 20.1 / 9.29 VinVL 56.1 / 7.09 41.8 / 6.69 Model Acc F1 Best PLM  31.0 (32.5) 20.1 (17.6) VinVL  56.1 ( 56.0 ) 41.8 ( 36.0 ) ISM (Box)  33.0 (42.5) 26.5 (26.1) Best PLM  31.0 (30.5) 20.1 (20.1) VinVL  56.1 (56.4) 41.8 (42.9) ISM (Human)  73.4 ( 75.4 ) 65.1 ( 68.0 ) Table 3: Probing performance on positional relationship (%). indicate that models with visual signals learn accurate spatial commonsense knowledge from images. ISM (Box) outperforms VinVL on those recognizable instances (81.6 vs. 53.8), but the recognition ratio is admittedly low. We conduct human evaluation on the generated images for more precise assessment. More than 80% of images are recognized by humans and these images accurately reflect the spatial commonsense compared to PLMs and VinVL. 2 The gap between VinVL and ISM (Human) may be due to different ways of using visual signals in pretraining. A training objective of VinVL, and other VL-PTMs, is aligning text with image regions. The discriminative features of objects are amplified, while other features may not receive as much attention. For instance, the shape and color are the discriminative features of an ap-ple , and its size is not that important in recognition. In image synthesis, models need to learn comprehensive knowledge of objects for reconstruction, and spatial knowledge may be learned implicitly in this process. Figure 4 demonstrates images generated by the ISM given the prompts of object pairs. ISM grasps the main characteristics of the objects, including their scales. Some objects (like theatre at the bottom of the middle column) can be identified by humans but are difficult for the object detection model because they are obstructed by objects in the foreground. And some objects are generated in multiple fragments (like plane and horse in the right column), therefore cannot be recognized by either the object detection model or humans. Positional Relationship. The probing performance on positional relationship is shown in Table 3. VinVL outperforms PLMs more than 20%, and ISM (Human) outperforms PLMs more than 35%, suggesting that models with visual signals learn more knowledge of the scenarios, especially the positions of objects relative to people. The gap between PLMs and ISM (Box) is smaller compared to the gap in the subtask of size and height. One reason is that the rules defined in VDG cannot perfectly reflect the true positional relationship in images. For example, the man is beside the car in the left image of Figure 3, but he will be regarded as inside the car by the rules, as the region of car covers the region of man. Text-based PLMs tend to lean towards certain positional relations between a person and an object, without referring to the action. In 64% cases, RoBERTa chooses the same option for a  person, object  pair with different actions, while the proportion is 21% for VinVL, and 28% for ISM (Human). Models that master better spatial knowledge should be able to infer the relative scale of two objects from intermediate references. For example, if a model knows a dog is larger than an ant and a sofa is larger than a dog , it may learn a sofa is larger than an ant , even if it has not seen sofa and ant together. We inspect models on how consistent their probing results are. The consistency is measured in two aspects: symmetry and transitivity . Symmetry implies that if a model predicts A > B , then it should also predict B < A , and vice versa: A < B =  B > A . Here > and < are in terms of size or height. We enumerate the object pairs and count the percent-2370 a n t c o i n nu t b u ll e t d i c e b i r d c u p s h e ll b o tt l e w a ll e t t y r e c h a i r m i c r o w a v e d o g s u i t c a s e hu m a n s o f a b oo k s h e l f t i g e r b e d h o u s e c i n e m a m o un t a i n t r u c k p l a n e Current Object 0.0 0.5 1.0 1.5 2.0 R a t i o Each object's size prediction from RoBERTa #(c>a)/|A|, a A #(a>c)/|A|, a A #(c<a)/|A|, a A #(a<c)/|A|, a A a n t c o i n nu t b u ll e t d i c e b i r d c u p s h e ll b o tt l e w a ll e t t y r e c h a i r m i c r o w a v e d o g s u i t c a s e hu m a n s o f a b oo k s h e l f t i g e r b e d h o u s e c i n e m a m o un t a i n t r u c k p l a n e Current Object 0.0 0.5 1.0 1.5 2.0 R a t i o Each object's size prediction from VinVL #(c>a)/|A|, a A #(a>c)/|A|, a A #(c<a)/|A|, a A #(a<c)/|A|, a A Figure 5: Predictions from RoBERTa and VinVL in the subtask of objects' sizes. age of predictions that meet the symmetry criterion. Transitivity implies that if a model predicts A > B and B > C , then it should predict A > C . It also works for < , A < B  B < C =  A < C . We enumerate the triples  A, B, C  where the predicted relation between  A, B  is identical to the prediction between  B, C  , and count the percentage that the prediction between  A, C  meets the transitivity criterion. Note that we only evaluate whether the predictions are consistent with each other, regardless of the gold answers. We evaluate the consistency of predictions from PLMs that perform the best in the probing tasks (RoBERTa for size and BERT for height), VinVL, and ISM (Human). The results are in Table 4. VinVL outperforms the best PLM in both metrics, and the characteristics of them are similar: the transitive consistency is high, while the symmetric consistency is low. To further analyze this phenomenon, we exhibit each object's size predictions from RoBERTa and VinVL in Figure 5. The models exhibit different behaviors in recognizing object scales. As the objects (X-axis of Figure 5) are roughly listed from smaller to larger groups, the bottom blue bars are expected to follow a non-descending order from left to right, and the solid orange bars should be non-ascending. The predictions of VinVL are generally in line with this trend, while RoBERTa's predictions are disordered. For example, ant is predicted to be larger than other objects with high probability, and cinema is larger than others is unlikely to happen. On the other hand, if the model predictions are consistent, Model Acc (avg. /  ) F1 (avg. /  ) BERT 27.4 / 3.17 19.7 / 7.25 RoBERTa 29.5 / 16.0 20.1 / 9.90 VinVL 58.1 / 1.97 44.4 / 1.63 Model Acc F1 Best PLM  29.5 (28.4) 20.1 (19.1) VinVL  58.1 (52.3) 44.4 (41.0) ISM (Human)  66.5 ( 74.8 ) 59.4 ( 69.2 ) Table 5: Probing models on the generalized dataset of positional relationship. the two solid bars should sum to 1. However, the sum is far above 1 for most objects in VinVL's predictions. This bias towards words indicating the choice of large may come from the pretraining corpus. For example, sofa occurs twice as many times with words indicating large as with words indicating small in COCO (Lin et al., 2014), one of VinVL's pretraining datasets. ISM's predictions comply with the symmetry criterion, outperforming other models by 40%, while also having good transitive consistency. The knowledge probed from ISM is more consistent. Figure 6 exhibits the symmetric and transitive consistency of images generated by ISM. The consistency of scale knowledge makes the predictions more convincing, and gives models a chance to learn new comparisons between objects. cycle is a common scenario and may frequently exist in ISM's training set, so models can generate images more easily when being fed with the text prompts like a boy rides a bicycle . To further challenge ISM's capability, we make a generalized version of our original positional relationship dataset. It is designed to examine whether models are able to robustly reflect the spatial commonsense knowledge when facing uncommon scenarios. A generalized scenario is built upon the original one by replacing the person and object in the text prompts. We select the new person and new object from the subterms of the original ones (those with IsA relation in ConceptNet (Speer et al., 2017), like enchantress is a woman ). To ensure these newly constructed scenarios are not likely to appear in the training data of models, we search them in BookCorpus (Zhu et al., 2015) and remove the scenarios that have appeared. The newly generated scenarios are also validated by humans to ensure that they are reasonable. Results of probing PLMs, VinVL, and ISM 3 on the generalized dataset are in Table 5. PLMs and VinVL achieve similar performance on both the generalized dataset and the original one, indicating that they behave robustly when facing uncommon scenarios. The performance gap between other models and ISM (Human) slightly narrows down, but ISM (Human) still outperforms VinVL more than 8%. Figure 7 exhibits images generated by ISM with the generalized prompts. Although it is 3 We do not consider ISM (Box) because many new objects we used are unfamiliar to object detection models. Only 17% of the objects are in the object detection classes. difficult for ISM to generate unfamiliar objects, it is still capable of capturing the positional relations. We investigate how to acquire spatial knowledge from ISMs and whether the knowledge is effective in natural language understanding scenarios. To our best knowledge, there is no appropriate task that focuses on spatial commonsense, so we create a toy task by transforming our probing benchmark into the form of question answering (QA). Dataset. We construct a QA dataset of yes/no questions. Questions of objects' sizes are in the form of Is O a larger/smaller than O b ? And questions of objects' heights are like Is O a taller/shorter than O b ? , where O a and O b are two objects. Questions about positional relationship are accompanied with the action: for instance, A man washes the car. Is the man inside the car? To avoid bias in answer distribution, the numbers of yes and no are equal in gold answers. There are 500 questions for size, 500 for height, and 448 for positional relationship. Models. We use VinVL-base together with our image synthesis model VQGAN+CLIP to answer spatial commonsense questions. The VinVL here is finetuned on the VQA (Goyal et al., 2017) task. It takes images generated from ISM with textual prompts from questions, and predicts the answer based on the question and image together. Note that the VQA training corpus does not contain commonsense reasoning questions. We choose UnifiedQA (Khashabi et al., 2020) as a text-based QA model for comparison. Based on the pretrained T5 model (Raffel et al., 2019), UnifiedQA is continually trained on various QA tasks, including three yes/no datasets. We use UnifiedQA-large, which is comparable with our synthesis and reasoning model (ISM w/ VinVL) in size. Results. As shown in Table 6, ISM w/ VinVL outperforms UnifiedQA on all subtasks, showing that spatial knowledge from ISMs can be directly used by vision-language models without additional training. Although some images cannot be precisely recognized by object detection models, vision-language models may find regions that are related to the objects mentioned in questions, and make decisions based on the features of these regions. The results on the simple natural language task show that it is beneficial to tackle natural language tasks with vision-language methods, and ISMs can be a bridge between the two modalities . With the development of ISMs and object detection techniques, we believe the generated images will help more. We propose a new spatial commonsense probing framework to investigate object scales and positional relationship knowledge in text-based pretrained models and models with visual signals. Experimental results show that models with visual signals, especially ISMs, learn more accurate and consistent spatial commonsense than text-only models. Integrating ISMs with visual reasoning models outperforms PLMs in answering spatial questions. This manifests the potential of using spatial knowledge from ISMs in natural language understanding tasks. This work is supported in part by National Key R&D Program of China (No. 2020AAA0106600) and NSFC (62161160339). We would like to thank the anonymous reviewers and action editor for the helpful discussions and suggestions. Also, we would thank Quzhe Huang, Chen Zhang, Chen Henry Wu, Yuxuan Lai and Nan Hu for their detailed comments. For any correspondence, please contact Yansong Feng.
abstain abstain abstain abstain abstain abstain other abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain method abstain result method objective method abstain abstain objective result abstain objective abstain abstain method abstain abstain other other other other other method other other other other other method other other other other other abstain method other objective abstain objective abstain abstain objective abstain method abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method objective abstain method method method method abstain abstain method abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain method method method abstain method abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain method abstain abstain abstain abstain method objective abstain abstain abstain other other other other