Answers.Body: <p>I guess your problem is a form of NER (Named-Entity Recognition) tagging. NER tags consist of PER(person), LOC(location), ORG(organization) and MISC(miscellaneous) and O(other). with the help of a NER-tagger algorithm, you probably would have:</p>
  
  <p><code>My(O) favorite(O) book(O) is(O) harry(B-MISC) potter(I-MISC) and(I-MISC) the(I-MISC) prisoner(I-MISC) of(I-MISC) azkaban(I-MISC).</code></p>
  
  <p>now you can tokenize your text with identifying different NER tags, and join "B-" and "I-" prefixes with the same NER tag. e.g. you have "harry potter and ..." as a single token, which is a MISC and starts from harry (since harry is B-MISC) and ends to azkaban (since azkaban has the last consecutive I-MISC tag). Now you can let your POS tagger act the "harry potter ..." as a single token, and it must tag it with "NN"</p>
  
  <p>Another solution for this problem is called "chunking". It works based on a set of rules and detect Noun Phrases (NPs). Here, you define a rule and create the regexp statement for it. e.g. you define all consecutive <code>NN</code>s as a single NP (harry potter): <code>&lt;NN&gt;+</code> or a <code>DT</code> followed by a string of <code>NN</code>s (the brown fox): <code>&lt;DT&gt;?&lt;NN&gt;+</code>. Now you tokenize your text based on a regexp matching method. but in your case, chunking such a long NP is almost impossible. because a rule that finds this NP as a NP candidate, would by mistake find a lot of other string of words as a NP candidate too, while they're not a NP at all. (see <a href="http://www.nltk.org/book/ch07.html" rel="nofollow noreferrer">nltk: information extraction</a>)</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-22T09:39:33.800Z
Answers.LastActivityDate: 2017-06-22T09:51:09.330Z
Answers.LastEditDate: 2017-06-22T09:51:09.330Z
Answers.UserDisplayName: Alireza
Body: <p>By means of parts of speech tagging, words of a given sentence can be assumed to be <code>noun</code>/<code>verb</code> etc, but if the sentence is for instance:</p>
  
  <pre><code>"My favourite book is harry potter and the prizoner of azkaban"
  </code></pre>
  
  <p>note that the inputs I receive would be from a chat interface so having a fixed format for the data can't be expected. Is there a way to identify <code>"harry potter and the prizoner of azkaban"</code> as a proper noun from such messages?</p>
  
  <p>Currently this query tags as:</p>
  
  <pre><code>My|PRP$ 
  favourite|JJ 
  book|NN 
  is|VBZ 
  harry|JJ 
  potter|NN 
  and|CC 
  the|DT 
  prizoner|NN 
  of|IN 
  azkaban|NN
  </code></pre>
  
  <p>I would like to know if this can be handled some way, or if there is another algorithm that can handle this?</p>
  
Tags: <natural-language-processing>
Title: Tagging parts of speech when proper noun is a composite
UserDisplayName: Amresh Venugopal
ThreadText: # Tagging parts of speech when proper noun is a composite
  
  (Asked by: username_0 on 2017-06-21)
  
  By means of parts of speech tagging, words of a given sentence can be assumed to be noun/verb etc, but if the sentence is for instance:
  "My favourite book is harry potter and the prizoner of azkaban"
  
  note that the inputs I receive would be from a chat interface so having a fixed format for the data can't be expected. Is there a way to identify "harry potter and the prizoner of azkaban" as a proper noun from such messages?
  Currently this query tags as:
  My|PRP$ 
  favourite|JJ 
  book|NN 
  is|VBZ 
  harry|JJ 
  potter|NN 
  and|CC 
  the|DT 
  prizoner|NN 
  of|IN 
  azkaban|NN
  
  I would like to know if this can be handled some way, or if there is another algorithm that can handle this?
  
  **Tags:** natural-language-processing
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-06-22. Score: 4 | Accepted answer
  
  I guess your problem is a form of NER (Named-Entity Recognition) tagging. NER tags consist of PER(person), LOC(location), ORG(organization) and MISC(miscellaneous) and O(other). with the help of a NER-tagger algorithm, you probably would have:
  
  `My(O) favorite(O) book(O) is(O) harry(B-MISC) potter(I-MISC) and(I-MISC) the(I-MISC) prisoner(I-MISC) of(I-MISC) azkaban(I-MISC).`
  
  now you can tokenize your text with identifying different NER tags, and join "B-" and "I-" prefixes with the same NER tag. e.g. you have "harry potter and ..." as a single token, which is a MISC and starts from harry (since harry is B-MISC) and ends to azkaban (since azkaban has the last consecutive I-MISC tag). Now you can let your POS tagger act the "harry potter ..." as a single token, and it must tag it with "NN"
  
  Another solution for this problem is called "chunking". It works based on a set of rules and detect Noun Phrases (NPs). Here, you define a rule and create the regexp statement for it. e.g. you define all consecutive `NN`s as a single NP (harry potter): `+` or a `DT` followed by a string of `NN`s (the brown fox): `?+`. Now you tokenize your text based on a regexp matching method. but in your case, chunking such a long NP is almost impossible. because a rule that finds this NP as a NP candidate, would by mistake find a lot of other string of words as a NP candidate too, while they're not a NP at all. (see [nltk: information extraction](http://www.nltk.org/book/ch07.html))
  
  
  ---
  
  