Answers.Body: <p>There's nothing stopping you from training a model with whatever tags you want.</p>
  <p>Using what you describe as &quot;usual&quot; format means you would have approx half as many tags as using the IOB format. In theory this means your model will develop higher accuracy faster and with less training data. On the downside, you will need to do more work when interpreting the results in order to be confident where one named entity ends and another one begins.</p>
  <p>I made a <a href="https://gist.github.com/alanbuxton/804082dbd5f3a5e85ab1e68e5a4317cd" rel="nofollow noreferrer">notebook to do some empirical tests on this</a> which uses 17 tags in the IOB format and 9 in the &quot;usual&quot; format.</p>
  <p>TL;DR using the &quot;usual&quot; format did not produce a noticeable improvement in model quality.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-01T16:47:56.357Z
Answers.LastActivityDate: 2021-01-01T16:47:56.357Z
Answers.UserDisplayName: Alan Buxton
Body: <p>Do we have to use the IOB format on labels in the NER dataset (such as B-PERSON, I-PERSON, etc.) instead of using the usual format (PERSON, ORGANIZATION, etc.)? If so, why? How will it affect the performance of the model?</p>
  
Tags: <machine-learning><natural-language-processing><named-entity-recognition>
Title: Do we have to use the IOB format on labels in the NER dataset? If so, why?
UserDisplayName: Rachmansyah Adhi Widhianto
ThreadText: # Do we have to use the IOB format on labels in the NER dataset? If so, why?
  
  (Asked by: username_0 on 2020-07-16)
  
  Do we have to use the IOB format on labels in the NER dataset (such as B-PERSON, I-PERSON, etc.) instead of using the usual format (PERSON, ORGANIZATION, etc.)? If so, why? How will it affect the performance of the model?
  
  **Tags:** machine-learning, natural-language-processing, named-entity-recognition
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-01-01. Score: 0
  
  There's nothing stopping you from training a model with whatever tags you want.
  
  Using what you describe as "usual" format means you would have approx half as many tags as using the IOB format. In theory this means your model will develop higher accuracy faster and with less training data. On the downside, you will need to do more work when interpreting the results in order to be confident where one named entity ends and another one begins.
  
  I made a [notebook to do some empirical tests on this](https://gist.github.com/alanbuxton/804082dbd5f3a5e85ab1e68e5a4317cd) which uses 17 tags in the IOB format and 9 in the "usual" format.
  
  TL;DR using the "usual" format did not produce a noticeable improvement in model quality.
  
  
  ---
  
  