what evaluation metrics are used?
coverage and precision
Verbs are important in semantic understanding of natural language. Traditional verb representations, such as FrameNet, PropBank, VerbNet, focus on verbs' roles. These roles are too coarse to represent verbs' semantics. In this paper, we introduce verb patterns to represent verbs' semantics, such that each pattern corresponds to a single semantic of the verb. First we analyze the principles for verb patterns: generality and specificity. Then we propose a nonparametric model based on description length. Experimental results prove the high effectiveness of verb patterns. We further apply verb patterns to context-aware conceptualization, to show that verb patterns are helpful in semantic-related tasks.
Verb is crucial in sentence understanding BIBREF0 , BIBREF1 . A major issue of verb understanding is polysemy BIBREF2 , which means that a verb has different semantics or senses when collocating with different objects. In this paper, we only focus on verbs that collocate with objects. As illustrated in Example SECREF1 , most verbs are polysemous. Hence, a good semantic representation of verbs should be aware of their polysemy. Example 1 (Verb Polysemy) eat has the following senses: Many typical verb representations, including FrameNet BIBREF3 , PropBank BIBREF4 , and VerbNet BIBREF5 , describe verbs' semantic roles (e.g. ingestor and ingestibles for “eat”). However, semantic roles in general are too coarse to differentiate a verb's fine-grained semantics. A verb in different phrases can have different semantics but similar roles. In Example SECREF1 , both “eat”s in “eat breakfast” and “eat apple” have ingestor. But they have different semantics. The unawareness of verbs' polysemy makes traditional verb representations unable to fully understand the verb in some applications. In sentence I like eating pitaya, people directly know “pitaya” is probably one kind of food since eating a food is the most fundamental semantic of “eat”. This enables context-aware conceptualization of pitaya to food concept. But by only knowing pitaya's role is the “ingestibles”, traditional representations cannot tell if pitaya is a food or a meal. Verb Patterns We argue that verb patterns (available at http://kw.fudan.edu.cn/verb) can be used to represent more fine-grained semantics of a verb. We design verb patterns based on two word collocations principles proposed in corpus linguistics BIBREF6 : idiom principle and open-choice principle. Following the principles, we designed two types of verb patterns. According to the above definitions, we use verb patterns to represent the verb's semantics. Phrases assigned to the same pattern have similar semantics, while those assigned to different patterns have different semantics. By verb patterns, we know the “pitaya” in I like eating pitaya is a food by mapping “eat pitaya” to “eat $ INLINEFORM0 food”. On the other hand, idiom patterns specify which phrases should not be conceptualized. We list verb phrases from Example SECREF1 and their verb patterns in Table TABREF7 . And we will show how context-aware conceptualization benefits from our verb patterns in the application section. Thus, our problem is how to generate conceptualized patterns and idiom patterns for verbs. We use two public data sets for this purpose: Google Syntactic N-Grams (http://commondatastorage.googleapis.com/books/syntactic -ngrams/index.html) and Probase BIBREF7 . Google Syntactic N-grams contains millions of verb phrases, which allows us to mine rich patterns for verbs. Probase contains rich concepts for instances, which enables the conceptualization for objects. Thus, our problem is given a verb INLINEFORM0 and a set of its phrases, generating a set of patterns (either conceptualized patterns or idiom patterns) for INLINEFORM1 . However, the pattern generation for verbs is non-trivial. In general, the most critical challenge we face is the trade-off between generality and specificity of the generated patterns, as explained below.