Simile interpretation is a crucial task in natural language processing. Nowadays, pre-trained language models (PLMs) have achieved state-of-the-art performance on many tasks. However, it remains under-explored whether PLMs can interpret similes or not. In this paper, we investigate the ability of PLMs in simile interpretation by designing a novel task named Simile Property Probing, i.e., to let the PLMs infer the shared properties of similes. We construct our simile property probing datasets from both general textual corpora and human-designed questions, containing 1,633 examples covering seven main categories. Our empirical study based on the constructed datasets shows that PLMs can infer similes' shared properties while still underperforming humans. To bridge the gap with human performance, we additionally design a knowledge-enhanced training objective by incorporating the simile knowledge into PLMs via knowledge embedding methods. Our method results in a gain of 8.58% in the probing task and 1.37% in the downstream task of sentiment classification. The datasets and code are publicly available at https://github.com/Abbey4799/PLMs-Interpret-Simile. A simile is a figure of speech comparing two fundamentally different entities via shared properties (Paul, 1970). There are two types of similes as illustrated in Figure 1, closed similes explicitly reveal the shared properties between the topic entity and the vehicle entity, such as the property slow shared by lady and snail in the sentence The old lady walks as slow as a snail ; while open similes do not state the shared property such as the sentence The old lady walks like a snail . Similes play a vital role in human expression to make literal  Equal contribution  Corresponding author Figure 1: Examples of two types of similes. utterances more vivid and graspable and are widely used in the corpus of various domains (Liu et al., 2018; Chakrabarty et al., 2020a; Zhang et al., 2020). It is estimated that over 30% of the comparisons can be regarded as similes in product reviews (Nic-ulae and Danescu-Niculescu-Mizil, 2014). Simile interpretation is a crucial task in natural language processing (Veale and Hao, 2007; Qadir et al., 2016; Chakrabarty et al., 2021a), which can assist several downstream tasks such as understanding more sophisticated figurative language (Veale and Hao, 2007) and sentiment analysis (Niculae and Danescu-Niculescu-Mizil, 2014; Qadir et al., 2015). Take the simile the lawyer is like a shark for an example. Although all words in this simile are neutral, this simile expresses a negative affect since lawyer and shark share the negative property aggressive . In the past few years, large pre-trained language models (PLMs) have achieved state-of-the-art performance on many natural language processing tasks (Devlin et al., 2018; Liu et al., 2019b). Recent studies suggest that PLMs have possessed various kinds of knowledge into contextual representations (Goldberg, 2019; Petroni et al., 2019; Lin et al., 2019; Cui et al., 2021). However, the ability of PLMs to interpret similes remains under-explored. Although some recent work (Chakrabarty et al., 2021a) studies the ability of PLMs in choosing or generating the plausible continuations in narratives, this way cannot fully reveal the ability of PLMs to interpret similes. ity of PLMs in simile interpretation by designing a novel task named as Simile Property Probing , i.e., to let the PLMs infer the shared properties of similes. Specifically, we design a particular masked-word-prediction probing task in the form of multiple-choice questions. This probe masks the explicit property of a closed simile and then lets the PLMs discriminate it from three distractors. To make the questions convincing and challenging, the distractors should be not only true-negative as they would introduce logical errors once they are filled in the sentence, but also challenging as they are semantically close to the correct answer. To achieve this, we propose to obtain some similar properties of the golden one from ConceptNet (Liu and Singh, 2004) and COMET (Bosselut et al., 2019), from which we select the three best distractors according to their proximity to the golden property in the feature space. From two different types of data sources: textual corpus collection and human-designed questions, we collect a total of 1,633 probes with various usage frequencies and context diversities, covering seven categories as listed in Table 1. Based on our designed task, we evaluate the ability of BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019b) to infer the shared properties of similes. We perform an empirical evaluation in two settings: (1) zero-shot, where the models are off-the-shelf; (2) fine-tuned, where the models are fine-tuned with MLM objective via masking properties. We observe that PLMs have been able to infer properties of similes in the pre-training stage and the ability can be further enhanced by fine-tuning. However, fine-tuned PLMs still perform worse than humans. Moreover, we find that the simile components vehicle and topic contribute the most when inferring the properties. Inspired by the sufficient hints offered by the components vehicle and topic in our empirical study, we propose a knowledge-enhanced training objective to further bridge the gap with human performance. Considering property (p) as the relation between topic (t) and vehicle (v), we design a simile knowledge embedding objective function following conventional knowledge embedding methods (Bor-des et al., 2013) to incorporate the simile knowledge (t,p,v) into PLMs. To integrate simile knowledge and language understanding into PLMs, we jointly optimize the knowledge embedding objective and the MLM objective in our design. Overall, the knowledge-enhanced objective shows effectiveness in our probing task and the downstream task of sentiment classification. To summarize, our contributions are three-fold: (1) To our best knowledge, we are the first to systematically evaluate the ability of PLMs in interpreting similes via a proposed novel simile property probing task. (2) We construct simile property probing datasets from both general textual corpora and human-designed questions, and the probing datasets contain 1,633 examples covering seven main categories of similes. (3) We also propose a novel knowledge-enhanced training objective by complementing the MLM objective with the knowledge embedding objective. This method gains 8.58% in the probing task and 1.37% in the downstream task of sentiment classification. A sentence of simile generally consists of five major components (Hanks, 2013; Niculae and Danescu-Niculescu-Mizil, 2014), where four are necessary and the remaining one is optional. The four explicit components are as follows: (1) topic (or tenor) : the subject of the comparison acting as 7876 Figure 2: A process for designing our simile property probing task. source domain; (2) vehicle : the object of the comparison acting as target domain; (3) event : the predicate indicating act or state; (4) comparator : the trigger word of a simile such as as or like . The optional component property reveals the shared characteristics between the topic and the vehicle. There are two types of similes depending on whether the property is explicit or implicit (Beardsley, 1981). The similes which mention the property directly are named as the closed similes, while the others are open similes, as shown in Figure 1. 3 The Simile Property Probing Task 3.1 Task Formulation To estimate the ability of PLMs in simile interpretation, we design a particular Simile Property Probing task, which masks the explicit property of a closed simile, and then lets the PLMs discriminate it among four candidates. Considering that the shared properties between topic and vehicle may not be unique (Lacroix et al., 2005), we specifically design a multiple-choice question answering task (with only one correct answer) rather than a cloze task to probe the ability of PLMs to infer properties of similes, since the latter one may result in multiple correct answers. vehicle is masked, the probing task requires the PLMs to find the correct property from four options, where the other three options are hard distractors. We construct datasets for the proposed probing task in four steps. The overview of our probing data collection process is described in Figure 2. 3.2.1 Data Sources We construct our datasets from two different sources to detect the capability of PLMs from two perspectives: textual corpus collection and human-designed questions. To avoid laborious human labeling on the implicit properties of open similes, we collect closed similes with explicit properties. General Corpus. Following (Hanks, 2005; Niculae and Yaneva, 2013), we adopt two general corpora, British National Corpus (BNC) 1 and iWeb 2 . To identify closed similes, we extract the sentences matching the syntax as ADJ as (a, an, the) NOUN . Through syntactic pattern matching, we finally collect 1,917 sentences. Teacher-Designed Quizzes. Questions about similes designed by teachers from educational resources are ideal sources for assessing the ability to understand similes. Hence, we choose Quizizz 3 , an 1 https://www.english-corpora.org/bnc/ 2 https://www.english-corpora.org/iweb/ 3 https://quizizz.com/ 7877 Figure 3: Illustration of the distractor selection method. emerging learning platform founded in 2015. On this platform, users can create quizzes on a specific topic as teachers to assess students' understanding of related knowledge. We collect a set of quizzes with titles concerning similes and extract the complete closed simile sentences from the questions and answers in these quizzes. Finally, we retrieve 875 complete closed similes from 1,235 quizzes. To assure the quality of our constructed datasets and prepare for further analysis, three annotators are required to decide whether the extracted sentences are similes or not, and annotate their corresponding simile components. The inter-annotator agreement on identifying similes is 0.77 using Fleiss' Kappa score (Fleiss, 1971). All the properties in our datasets are single-token by replacing multi-token properties with their single-token synonyms in the knowledge base WordNet (Miller, 1995) and ConceptNet (Liu and Singh, 2004). To make our probes convincing, three distractors are designed against the original property in each simile with two criteria (Haladyna et al., 2002; Ren and Zhu, 2020): true-negative and challenging . We argue that well-designed distractors should be illogical when filled into the questions (true-negative) while being semantically related to the correct answer (challenging) . Our distractor design mainly involves three phases: 1) distractor generation; 2) distractor selection; 3) Human Confirmation. Distractor Generation. To meet the requirement of challenging , we generate distractor candidates from the four semantic-related components of a simile, i.e., topic, vehicle, event, and property. Given the original property, we harvest its antonyms from the knowledge base WordNet and ConceptNet. With regard to three other components, we extract their properties from two sources Dataset GeneralCorpus Quizzes #Sentence 775 858 #Unique topic concept 415 366 #Unique property concept 280 160 #Unique vehicle concept 522 250 #Unique event concept 147 66 #Unique topic-vehicle pair 743 684 #Unique topic-property-vehicle pair 751 701 Maximum sentence length 98 44 Average sentence length 25.80 12.69 Minimum sentence length 7 7 @Start 34.32% 20.40% @Middle 43.23% 63.29% @End 22.45% 16.32% Table 2: Statistics of our simile property probing datasets. as follows. Given a component, we utilize the HasProperty relation from ConceptNet (Liu and Singh, 2004) and COMET (Bosselut et al., 2019) to retrieve the property. Moreover, we rank the adjectives or adverbs concerning 4 each component in Wikipedia and BookCorpus corpus 5 by frequency and select the top ten candidates with a frequency of more than one. Distractor Selection. To select the most challenging distractors from the generated distractor candidates, we propose to measure the similarity between the original sentence with the correct property and the sentence with a distractor. Intuitively, the more similar the two sentences, the more challenging the distractor. An example of the distractor selection process is depicted in Figure 3. Given the original sentence or the new sentence replacing the correct property with a distractor, we first utilize RoBERTa LARGE to extract two types of features. One feature is context embedding, which is the sentence embedding of [CLS] , while the other feature is word embedding, which is the token embedding of the answer or distractors. We then concatenate the embeddings of the two features to compute the cosine similarity between sentences with the answer and a distractor. Finally, we select the top 3 distractors with the highest similarities. Human Confirmation. To ensure the distractors are true-negative , three human annotators are asked to label each selected distractor. If more than two annotators are uncertain about its correctness, we replace it with another suitable candidate. Table 2 presents the statistics of our constructed datasets. We count unique components and component pairs to present the usage frequencies of similes. The length of the sentences in each dataset indicates the diversities of context. Additionally, we analyze the distribution of the position of simile in the sentences in each dataset, where start , middle and end correspond to the positions of the three equally divided parts of each sentence. We also investigate the categories covered by our datasets. The results and details about the category classification are provided in Appendix C. Overall, the Quizzes dataset provides similes commonly expressed by people, while the General Corpus dataset presents similes with more diverse contexts. Besides evaluating the ability of PLMs in the zero-shot setting where the models are off-the-shelf, we also study whether the performance could be improved through fine-tuning with the MLM objective via masking properties. To achieve this, we collect training data from Standardized Project Gutenberg Corpus 6 (SPGC) (Gerlach and Font-Clos, 2020). SPGC is a 3 billion words corpus collected from about 60 thousand eBooks. We extract similes via matching the syntactic pattern (Noun ... as ADJ as ... NOUN) and end up with 4,510 sentences. Additionally, we adopt dependency parsing 7 to automatically annotate the simile components of each sentence without human labor. In this section, we first conduct a set of experiments to probe the ability of PLMs to infer properties in similes and then evaluate the influence of each component on the model performance. To disentangle what is captured by the original representations and what is introduced from fine-tuning stage, we apply two different types of settings: (1) zero-shot; (2) fine-tuning. In our first setting, we use BERT and RoBERTa with pre-trained masked-word-prediction heads to perform our probing task. In the second setting, we utilize the MLM training objective inherited from PLMs to fine-tune 6 https://github.com/pgcorpus/gutenberg/ 7 https://stanfordnlp.github.io/CoreNLP/ Setting Models GeneralCorpus Quizzes Gain ConScore (Zheng et al., 2019) 27.48 34.85 Meta4meaning (Xiao et al., 2016) 27.74 47.44 -EMB (Qadir et al., 2016) 28.27 47.90 -MIUWE (Bar et al., 2020) 30.97 53.85 -Zero-Shot BERTBASE 64.13 74.36 -BERTLARGE 72.39 83.22 RoBERTa BASE 69.55 82.87 RoBERTa LARGE 78.97 87.41 -Fine-tuned MLM-BERTBASE 67.74 82.05 +5.65 MLM-BERTLARGE 73.85 84.58 +1.40 MLM-RoBERTa BASE 70.58 84.69 +1.43 MLM-RoBERTa LARGE 78.97 88.97 +0.78 Human Performance 87.60 93.60 Table 3: Accuracy of different models in our simile property probing task. the models. We replace the property of each simile with the special token [MASK] in our constructed supervised datasets (Section 3.3) and ask models to recover the original property. The experimental details are provided in the Appendix B. We mainly compare the model accuracy of PLMs with the following baselines: (1) EMB (Qadir et al., 2016): It obtains the composite simile vector by performing an element-wise sum of the word embedding for the vehicle and event, then selects the answer with the shortest cosine distance from the composite vector. (2) Meta4meaning (Xiao et al., 2016) : This method prefers the properties which are strongly associated with both topic and vehicle. It also prefers the properties that are more relevant to the vehicle than to the topic. The association is measured by statistical significance. (3) ConScore (Zheng et al., 2019) : It suggests that better properties would have a smaller and balanced distance to the topic and vehicle in the word embedding space. (4) MIUWE (Bar et al., 2020) : The ranking method assigns each property a list of scores, including the statistical co-occurrences and similarity to the collocations of the topic and vehicle. The baselines above mainly consider the statistical information and embedding similarities between the properties and the simile components. The other baseline is human performance. We sample 250 random questions from both datasets, and for each question, we gather answers from three people. We take the majority vote as the human performance of our probing task and ensure that three annotators agree on the questions that they gave completely different annotation results. The accuracies of different methods under two different settings on our datasets are listed in Table 3, where the last column represents the average absolute gains of each PLM after fine-tuning with the MLM objective. All the results of our experiments are averaged over three random seeds. First of all, the prediction accuracies of both BERT and RoBERTa in the zero-shot setting are much higher than the baselines only considering the statistical information and embedding similarities between simile components. This phenomenon indicates that the knowledge learning from the pre-train stage can help infer the simile properties. Moreover, the performance can be further improved by training with the MLM objective, demonstrating that the fine-tuning phase with the supervised dataset can introduce related knowledge about similes. However, models still underperform humans by several accuracy points, leaving room for improvement in our probing task. Overall, all the models perform better on Quizzes Dataset than on General Corpus Dataset, indicating that more diverse contexts increase the difficulty of inferring the shared properties. Also, RoBERTa consistently outperforms BERT, likely due to a larger pre-training corpus containing more similes. More complementary results are provided in the Appendix A.1. Due to the high performance of off-the-shelf PLMs, we are interested in the contributions of each component to infer shared properties in the zero-shot setting. First, the information of each component is hidden through a certain strategy. Specifically, for topic , vehicle and comparator , we replace their tokens with a special token [UNK] which means unknown. With regard to event , we convert it into a suitable copula, such as am and is, to ensure the integrity of syntax. Furthermore, we also set up a baseline by randomly replacing a token with [UNK] in the context. Examples corresponding to all settings are shown in Table 8 in the Appendix B. We finally report the model accuracy and declined absolute accuracy after hiding the information of each component. The results in Table 4 show varying degrees of the decline of all settings. If the model's performance decreases more, it means that the influence of the component is more significant than others. Three Figure 4: An overview of our objective function design major components (i.e., vehicle, topic and comparator) obtain higher declined absolute accuracy than random token, which demonstrates that the information of these simile components is more valuable than other words to infer the shared properties. Among all the components, removing the comparator may cause the most significant performance drop. This result is mostly because PLMs cannot identify the sentence as a simile without an obvious indicator. When it comes to the remaining 3 components, vehicle contributes the most, followed by topic . Hence, we argue that it may be beneficial to explicitly leverage both the information of vehicle and topic to infer the properties. Benefiting from the result that topic and vehicle are the two most essential components for predicting the shared properties of similes, we catch an insight that property can be seen as the relation between topic and vehicle following a set of knowledge embedding (KE) methods (Bordes et al., 2013; Wang et al., 2014; Ji et al., 2015). To integrate the insight mentioned above into our training procedure, we design an objective function as shown in Figure 4. Inspired by triplets representing the relational facts, we can also extract the topic, property, and vehicle from a simile as a triplet ( t, p, v ) . The distance between topic and vehicle in the embedding space represents the plausibility of property. The plausibility can be measured by scoring functions (Bordes et al., 2013; Wang et al., 2014; Ji et al., 2015). To this end, we follow the scoring function from TransE (Bordes et al., 2013) and define the following Mean Square Error (MSE) loss as our KE loss: LKE = MSE ( E t + E p , E v ) (1) 7880 Datasets Models Topic Vehicle Event Comparator Random GeneralCorpus BERTBASE 59.87 (-04.26) 54.58 (-09.55) 62.84 (-01.29) 46.32 (-17.81) 63.05 (-01.08) BERTLARGE 67.74 (-04.65) 61.16 (-11.23) 70.19 (-02.20) 46.06 (-26.33) 69.07 (-03.32) RoBERTa BASE 65.29 (-04.26) 61.03 (-08.52) 68.52 (-01.03) 50.32 (-19.23) 67.31 (-02.24) RoBERTa LARGE 76.90 (-02.07) 69.68 (-09.29) 77.55 (-01.42) 54.97(-24.00) 77.72 (-01.25) Quizzes BERTBASE 67.02 (-07.34) 62.35 (-12.01) 73.43 (-00.93) 52.80 (-21.56) 71.91 (-02.45) BERTLARGE 77.86 (-05.36) 64.57 (-18.65) 82.63 (-00.59) 55.24 (-27.98) 79.91 (-03.31) RoBERTa BASE 76.11 (-06.76) 69.00 (-13.87) 81.47 (-01.40) 55.24 (-27.63) 77.58 (-05.29) RoBERTa LARGE 83.80 (-03.61) 74.24 (-13.17) 86.60 (-00.81) 60.84 (-26.57) 85.12 (-02.29) Table 4: Accuracy of PLMs in the zero-shot setting before and after hiding the information of each component on two datasets. where E t , E p , E v are the representations of topic, property and vehicle encoded by PLMs. We also try more advanced methods such as TransH (Wang et al., 2014) and TransD (Ji et al., 2015) for the knowledge embedding objective, and their results are presented in Table 7 in the Appendix A.2. Finally, our training procedure is to optimize MLM loss and KE loss jointly: L Ours =  LKE + LMLM (2) where  is a hyperparameter used to balance two objective functions. Table 5 presents the performance of the models fine-tuned with the MLM objective and our knowledge-enhanced objective on the two datasets, where the last column shows the performance gains brought by our improvement to the training objective. Overall, each model trained with our knowledge-enhanced objective outperforms the one trained with the MLM objective on both datasets, demonstrating the effectiveness of our objective in the probing task. For the Quizzes dataset, BERT achieves more performance gains than RoBERTa does, which is probably because RoBERTa has better modeled the relationship among topic , property and vehicle in the similes with simple syntactic structure during Models Original LMLML Ours BERTBASE 84.96 85.45 85.63 BERTLARGE 86.02 86.65 86.95 RoBERTa BASE 88.51 88.61 89.51 RoBERTa LARGE 88.84 89.08 90.21 Table 6: Accuracy of PLMs with three settings in the downstream task of sentiment classification. fine-tuning with the MLM objective. For the General Corpus dataset, the BASE version of models tends to yield higher performance improvements, probably because the models with larger parameter sizes can better capture the relationship among simile components in the similes with more diverse contexts when fine-tuning with the MLM objective. Similes generally transmit a positive or negative view due to the shared properties (Fishelov, 2007; Li et al., 2012; Qadir et al., 2015). Taking the simile the lawyer is like a shark as an example, the implicit shared property aggressive between lawyer and shark indicates the negative polarity. Therefore, we design a sentiment polarity downstream task to validate the improvement of our method to infer shared properties. Our experiments are based on the Amazon reviews dataset 8 which provides reviews and their corresponding sentiment ratings. Following (Mu-dinas et al., 2012; Haque et al., 2018), we first process the dataset into a binary sentiment classification task by defining the 1-star and 2-star ratings as negative, the 4-star, and 5-star ratings as positive, while excluding the 3-star neutral ratings. To further address the label imbalance problem, we then sample the positive and negative reviews at 1:1. The final dataset consists of 5,023 reviews and is split into a ratio of 6:2:2 for the train/dev/test set. multi-8 https://www.kaggle.com/bittlingmayer/amazonreviews layer perceptron (MLP) classifiers on top of PLM's contextualized representation. The parameters of PLM are fixed and from three settings: (1) zero-shot; (2) fine-tuned with the MLM objective in the probing task; (3) fine-tuned with the knowledge-enhanced objective in the probing task. The results are shown in the Table 6. First of all, fine-tuning with the MLM objective improves the performance of all models in the sentiment classification task, demonstrating that improving models' ability to infer the properties of similes can enhance models' understanding of the sentiment polarity. Moreover, the performance is further improved by our knowledge-enhanced objective, especially for RoBERTa whose main gains are mostly contributed by our additional knowledge embedding objective. This indicates the effectiveness of our knowledge-enhanced objective in the downstream task of sentiment analysis. Furthermore, we investigate the mechanism of how knowledge-enhanced objective brings improvement. We first calculate the L2 distance between the representations in the last hidden states of each pair of components. The results are shown in Figure 5. In all pairs, the distance given by our objective is generally shorter than MLM-BERT, which indicates that modeling the relationships among the three important components is efficient to enhance the model performance. Specifically, we visualize the final layer representation of a simile into two-dimensional spaces via Principal Component Analysis (PCA) (Pearson, 1901) in Figure 6. In both MLM and our objective, the models are required to fill in the masked token in the same simile sentence. The model fine-tuned with the MLM objective predicts wrongly, while our fine-tuned model predicts correctly. We find that our representations of the three components 6 4 2 0 2 4 6 8 PC1 4 2 0 2 4 6 PC 2 myfriendsandi sold our old toys and donatedthemoney to an old man whowasas [MASK] asa church mouse Simile Processing. Simile processing mainly involves 3 fields: simile detection, simile generation, and simile interpretation. The bulk of work in similes mainly focuses on identifying similes and their components (Niculae, 2013; Niculae and Danescu-Niculescu-Mizil, 2014; Liu et al., 2018; Zeng et al., 2020). Recent years have witnessed a growth of work to transfer literal sentences to similes (Zhang et al., 2020; Chakrabarty et al., 2020b). (Chakrabarty et al., 2021b) study the ability of PLMs to recognize textual entailment related to similes. With regard to simile interpretation, (Qadir et al., 2016; Xiao et al., 2016; Bar et al., 2020; Zheng et al., 2019) rank the properties by the statistical co-occurrence and embedding similarities with other simile components. (Chakrabarty et al., 2021a) interpret similes by choosing or generating continuation for narratives via PLMs. Different from these works, we investigate the ability of PLMs to infer shared properties of similes. Probing Tasks for PLMs. Many studies investigate whether PLMs encode knowledge in their contextual representations by designing probing tasks. Early studies mainly focus on the linguistic knowledge captured by PLMs (Liu et al., 2019a; Tenney et al., 2019). (Petroni et al., 2019) first propose a word prediction task to probe factual 7882 knowledge stored in PLMs. Similar methods are utilized to evaluate various commonsense knowledge, such as symbolic reasoning ability (Talmor et al., 2020; Zhou et al., 2020), numerical commonsense knowledge (Lin et al., 2020), properties associated with concepts (Weir et al., 2020). To our best knowledge, we are the first work to investigate the ability of PLMs to interpret similes by proposing a simile property probing task. Enhance PLMs via Knowledge Regularization. Recently, many researchers integrate external knowledge with PLMs by complementing the MLM objective with an auxiliary knowledge-based objective. For example, there are works that introduce span-boundary objective for span-level prediction (Joshi et al., 2020), copy-based training objective for mention reference prediction (Ye et al., 2020), knowledge embedding objective for factual knowledge (Wang et al., 2021) and arithmetic relationships of linguistic units for universal language representation (Li and Zhao, 2021). Different from these works, we incorporate simile knowledge with the training objective by modeling the relationship between the salient components of similes. In this work, we are the first to investigate the ability of PLMs in simile interpretation via a proposed novel simile property probing task. We construct two multi-choice probing datasets covering two data sources. By conducting a series of empirical experiments, we prove that PLMs exhibit the ability to infer simile properties in the pre-training stage and further induce more related knowledge during the fine-tuning stage, but there is still a gap between PLMs and humans in this task. Furthermore, we propose a knowledge-enhanced training objective to bridge the gap, which shows effectiveness in the probing task and the downstream task of sentiment classification. In future work, we are interested in exploring the interpretation of more sophisticated figurative language, such as metaphor or analogy. We would like to thank anonymous reviews for their helpful comments and suggestions. Also, thanks to Jingping Liu, Leyang Cui for their insightful feedback that helped improve the paper. We also thank Botian Jiang, Shuang Li for supporting our data collection. This research was supported by the National Key Research and Development Project (No. 2020AAA0109302), National Natural Science Foundation of China (No. 62072323), Shanghai Science and Technology Innovation Action Plan (No. 19511120400), Shanghai Municipal Science and Technology Major Project (No. 2021SHZDZX0103). We provide details of our work to address potential ethical concerns. In our work, we propose a simile property probing task and construct probing datasets from both general textual corpora and human-designed questions. First of all, all the data sources used in the data collection process are publicly available. Specifically, we follow the robots.txt 9 to respect the copyright when we collect similes from the learning platform Quizizz (Sec. 3.2.1). Moreover, there are three steps involving human annotation to ensure the quality of the datasets: simile and simile components recognition (Sec. 3.2.1), human confirmation for distractors (Sec. 3.2.2), and human performance (Sec. 4.1). To ensure the quality of annotation, all the annotators do not participate in our probing data collection, and they always label a small set of 50 examples to reach an agreement on the labeling criteria before the formal labeling. We protect the privacy rights of annotators and pay them above the local minimum wage.
abstain abstain abstain objective method result objective result other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method objective result abstain objective objective abstain objective objective objective method objective abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method abstain method method method abstain abstain other abstain abstain method method method abstain method method method abstain abstain method abstain method abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain method method method abstain method abstain abstain abstain abstain abstain method method abstain method method other abstain method method abstain abstain abstain abstain abstain abstain abstain method method abstain abstain result abstain abstain abstain abstain abstain abstain abstain method abstain method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other other objective other other other other other objective other other other objective objective method method objective objective other other other other method abstain abstain method abstain abstain abstain