Answers.Body: <p>You left some critical context out. The author is referring to the following sentence:</p>
  <ul>
  <li>[PER Jane Villanueva ] of [ORG United] , a unit of [ORG United Airlines
  Holding] , said the fare applies to the [LOC Chicago ] route.</li>
  </ul>
  <p>A NER algorithm should label &quot;Jane Villanueva&quot; as a person. The BIO labels would be B-PER for 'Jane' and I-PER for 'Villanueva.' Labeling 'Villanueva' as 'O' is incorrect and produces a false positive. Not labeling 'Villanueva' as I-PER is incorrect and produces a false negative. As Jurafsky explains, &quot;any tokens outside of any span of interest are labeled O.&quot;</p>
  <p>Labeling 'Jane Villanueva' as B-LOC I-LOC would also be incorrect; therefore, both labels would be false negatives.</p>
  <p>The author assumes that the reader knows the BIO tagging labels. For a reference, see this <a href="https://huggingface.co/dslim/bert-base-NER" rel="nofollow noreferrer">page</a> from Hugging Face. Here is an excerpt:</p>
  <p><a href="https://i.sstatic.net/eROD7.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/eROD7.jpg" alt="enter image description here" /></a></p>
  <p>For those interested, the book is online <a href="https://web.stanford.edu/%7Ejurafsky/slp3/ed3book.pdf" rel="nofollow noreferrer">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-10T05:31:02.880Z
Answers.Text: I still don't understand what's the problem with NER evaluation. Is he referring to that these should actually be counted as 1 error? Furthermore, why don't we say that labelling Jane Villanueva as B-ORG I-ORG causes 4 errors, false negatives relative to B-PER and I-PER and false positives relative to B-ORG I-ORG
Answers.UserDisplayName: Essam
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-09T21:44:18.820Z
Answers.LastActivityDate: 2022-12-10T01:42:17.043Z
Answers.LastEditDate: 2022-12-10T01:42:17.043Z
Answers.UserDisplayName: Brian O'Donnell
Body: <p>I'm reading from the book Speech and Language Processing by
  Dan Jurafsky and James H. Martin and I've stuck for a while trying to understand what the authors mean by the following</p>
  <blockquote>
  <p>The fact that named entity tagging has a segmentation component which
  is not present in tasks like text categorization or part-of-speech
  tagging causes some problems with evaluation. For example, a system
  that labeled Jane but not Jane Villanueva as a person would cause
  two errors, a false positive for O and a false negative for I-PER.
  In addition, using entities as the unit of response but words as the
  unit of training means that there is a mismatch between the training
  and test conditions.</p>
  </blockquote>
  <p>In particular, I don't see what's the problem with</p>
  <blockquote>
  <p>For example, a system
  that labeled Jane but not Jane Villanueva as a person would cause
  two errors, a false positive for O and a false negative for I-PER.</p>
  </blockquote>
  <p>Was is expected to cause one error or something? What if Jane Villaneuva was labeled as B-LOC I-LOC?</p>
  <p>Thanks.</p>
  
Tags: <natural-language-processing><named-entity-recognition>
Title: Confused about NER evaluation
UserDisplayName: Essam
ThreadText: # Confused about NER evaluation
  
  (Asked by: username_0 on 2022-12-09)
  
  I'm reading from the book Speech and Language Processing by
  Dan Jurafsky and James H. Martin and I've stuck for a while trying to understand what the authors mean by the following
  
  > The fact that named entity tagging has a segmentation component which
  > is not present in tasks like text categorization or part-of-speech
  > tagging causes some problems with evaluation. For example, a system
  > that labeled Jane but not Jane Villanueva as a person would cause
  > two errors, a false positive for O and a false negative for I-PER.
  > In addition, using entities as the unit of response but words as the
  > unit of training means that there is a mismatch between the training
  > and test conditions.
  
  In particular, I don't see what's the problem with
  
  > For example, a system
  > that labeled Jane but not Jane Villanueva as a person would cause
  > two errors, a false positive for O and a false negative for I-PER.
  
  Was is expected to cause one error or something? What if Jane Villaneuva was labeled as B-LOC I-LOC?
  
  Thanks.
  
  **Tags:** natural-language-processing, named-entity-recognition
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-12-09. Score: 1
  
  You left some critical context out. The author is referring to the following sentence:
  
  * [PER Jane Villanueva ] of [ORG United] , a unit of [ORG United Airlines
    Holding] , said the fare applies to the [LOC Chicago ] route.
  
  A NER algorithm should label "Jane Villanueva" as a person. The BIO labels would be B-PER for 'Jane' and I-PER for 'Villanueva.' Labeling 'Villanueva' as 'O' is incorrect and produces a false positive. Not labeling 'Villanueva' as I-PER is incorrect and produces a false negative. As Jurafsky explains, "any tokens outside of any span of interest are labeled O."
  
  Labeling 'Jane Villanueva' as B-LOC I-LOC would also be incorrect; therefore, both labels would be false negatives.
  
  The author assumes that the reader knows the BIO tagging labels. For a reference, see this [page](https://huggingface.co/dslim/bert-base-NER) from Hugging Face. Here is an excerpt:
  
  [![enter image description here](https://i.sstatic.net/eROD7.jpg)](https://i.sstatic.net/eROD7.jpg)
  
  For those interested, the book is online [here](https://web.stanford.edu/%7Ejurafsky/slp3/ed3book.pdf).
  
  > **Comments:**
  > **username_0**: I still don't understand what's the problem with NER evaluation. Is he referring to that these should actually be counted as 1 error? Furthermore, why don't we say that labelling Jane Villanueva as B-ORG I-ORG causes 4 errors, false negatives relative to B-PER and I-PER and false positives relative to B-ORG I-ORG  
  
  ---
  
  