<p>You left some critical context out. The author is referring to the following sentence:</p>
<ul>
<li>[PER Jane Villanueva ] of [ORG United] , a unit of [ORG United Airlines
Holding] , said the fare applies to the [LOC Chicago ] route.</li>
</ul>
<p>A NER algorithm should label &quot;Jane Villanueva&quot; as a person. The BIO labels would be B-PER for 'Jane' and I-PER for 'Villanueva.' Labeling 'Villanueva' as 'O' is incorrect and produces a false positive. Not labeling 'Villanueva' as I-PER is incorrect and produces a false negative. As Jurafsky explains, &quot;any tokens outside of any span of interest are labeled O.&quot;</p>
<p>Labeling 'Jane Villanueva' as B-LOC I-LOC would also be incorrect; therefore, both labels would be false negatives.</p>
<p>The author assumes that the reader knows the BIO tagging labels. For a reference, see this <a href="https://huggingface.co/dslim/bert-base-NER" rel="nofollow noreferrer">page</a> from Hugging Face. Here is an excerpt:</p>
<p><a href="https://i.sstatic.net/eROD7.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/eROD7.jpg" alt="enter image description here" /></a></p>
<p>For those interested, the book is online <a href="https://web.stanford.edu/%7Ejurafsky/slp3/ed3book.pdf" rel="nofollow noreferrer">here</a>.</p>

CC BY-SA 4.0
2022-12-10T05:31:02.880Z
I still don't understand what's the problem with NER evaluation. Is he referring to that these should actually be counted as 1 error? Furthermore, why don't we say that labelling Jane Villanueva as B-ORG I-ORG causes 4 errors, false negatives relative to B-PER and I-PER and false positives relative to B-ORG I-ORG
Essam
CC BY-SA 4.0
2022-12-09T21:44:18.820Z
2022-12-10T01:42:17.043Z
2022-12-10T01:42:17.043Z
Brian O'Donnell
<p>I'm reading from the book Speech and Language Processing by
Dan Jurafsky and James H. Martin and I've stuck for a while trying to understand what the authors mean by the following</p>
<blockquote>
<p>The fact that named entity tagging has a segmentation component which
is not present in tasks like text categorization or part-of-speech
tagging causes some problems with evaluation. For example, a system
that labeled Jane but not Jane Villanueva as a person would cause
two errors, a false positive for O and a false negative for I-PER.
In addition, using entities as the unit of response but words as the
unit of training means that there is a mismatch between the training
and test conditions.</p>
</blockquote>
<p>In particular, I don't see what's the problem with</p>
<blockquote>
<p>For example, a system
that labeled Jane but not Jane Villanueva as a person would cause
two errors, a false positive for O and a false negative for I-PER.</p>
</blockquote>
<p>Was is expected to cause one error or something? What if Jane Villaneuva was labeled as B-LOC I-LOC?</p>
<p>Thanks.</p>

<natural-language-processing><named-entity-recognition>
Confused about NER evaluation
Essam
# Confused about NER evaluation

(Asked by: username_0 on 2022-12-09)

I'm reading from the book Speech and Language Processing by
Dan Jurafsky and James H. Martin and I've stuck for a while trying to understand what the authors mean by the following

> The fact that named entity tagging has a segmentation component which
> is not present in tasks like text categorization or part-of-speech
> tagging causes some problems with evaluation. For example, a system
> that labeled Jane but not Jane Villanueva as a person would cause
> two errors, a false positive for O and a false negative for I-PER.
> In addition, using entities as the unit of response but words as the
> unit of training means that there is a mismatch between the training
> and test conditions.

In particular, I don't see what's the problem with

> For example, a system
> that labeled Jane but not Jane Villanueva as a person would cause
> two errors, a false positive for O and a false negative for I-PER.

Was is expected to cause one error or something? What if Jane Villaneuva was labeled as B-LOC I-LOC?

Thanks.

**Tags:** natural-language-processing, named-entity-recognition


---

## 1 answers

### Answer by username_1 on 2022-12-09. Score: 1

You left some critical context out. The author is referring to the following sentence:

* [PER Jane Villanueva ] of [ORG United] , a unit of [ORG United Airlines
  Holding] , said the fare applies to the [LOC Chicago ] route.

A NER algorithm should label "Jane Villanueva" as a person. The BIO labels would be B-PER for 'Jane' and I-PER for 'Villanueva.' Labeling 'Villanueva' as 'O' is incorrect and produces a false positive. Not labeling 'Villanueva' as I-PER is incorrect and produces a false negative. As Jurafsky explains, "any tokens outside of any span of interest are labeled O."

Labeling 'Jane Villanueva' as B-LOC I-LOC would also be incorrect; therefore, both labels would be false negatives.

The author assumes that the reader knows the BIO tagging labels. For a reference, see this [page](https://huggingface.co/dslim/bert-base-NER) from Hugging Face. Here is an excerpt:

[![enter image description here](https://i.sstatic.net/eROD7.jpg)](https://i.sstatic.net/eROD7.jpg)

For those interested, the book is online [here](https://web.stanford.edu/%7Ejurafsky/slp3/ed3book.pdf).

> **Comments:**
> **username_0**: I still don't understand what's the problem with NER evaluation. Is he referring to that these should actually be counted as 1 error? Furthermore, why don't we say that labelling Jane Villanueva as B-ORG I-ORG causes 4 errors, false negatives relative to B-PER and I-PER and false positives relative to B-ORG I-ORG  

---

