Answers.Body: <p>It depend on your workflow, language of text. Your official guide at here <a href="https://github.com/explosion/assets/blob/main/Prodigy/Prodigy_NER_flowchart_v2_0_0_light.pdf" rel="nofollow noreferrer">https://github.com/explosion/assets/blob/main/Prodigy/Prodigy_NER_flowchart_v2_0_0_light.pdf</a> . You can know how many data is enough. You can see few number <code>4000</code>, <code>25%</code>, <code>2000</code>, etc.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-03T07:50:04.637Z
Answers.LastActivityDate: 2023-08-03T07:50:04.637Z
Answers.UserDisplayName: Vy Do
Answers.Body: <p>There is no specific number of labels that is &quot;enough&quot;. For simple cases you can start with a few hundred examples, but normally you'll want several thousand.</p>
  <p>Since you have a large number of classes your problem might be a harder one, but on the other hand it could be easy if most of your text is like &quot;This merchant is called XXX&quot;.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-26T05:15:33.107Z
Answers.LastActivityDate: 2021-12-26T05:15:33.107Z
Answers.UserDisplayName: polm23
Answers.Body: <p>In my experience with NER with Spacy, and disagreeing with <a href="https://stackoverflow.com/questions/59488470/how-many-training-datasentences-are-required-for-custom-ner-using-spacy-python">this stackoverflow solution</a> and as @polm23 rightly mentioned, <strong>a several thousand samples for each entity should generate/predict entities</strong>, otherwise spacy would just recognise them based on default spacy entity types (mainly 'work-of-art')</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-26T18:13:46.787Z
Answers.Text: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-27T01:10:58.670Z
Answers.Text: This does not provide an answer to the question. Once you have sufficient [reputation](https://ai.stackexchange.com/help/whats-reputation) you will be able to [comment on any post](https://ai.stackexchange.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/19616)
Answers.UserDisplayName: Saurav Maheshkar
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-26T18:07:58.877Z
Answers.LastActivityDate: 2022-03-10T17:55:59.847Z
Answers.LastEditDate: 2022-03-10T17:55:59.847Z
Answers.UserDisplayName: Hemant Rakesh
Body: <p>I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of thousands of different merchants.</p>
  <p>Some questions that I have:</p>
  <ul>
  <li><p>How much labelling do I need to do given the number of merchants I need to extract?</p>
  </li>
  <li><p>How many different instances of the same merchant I need to label to get decent results?</p>
  </li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-26T09:32:29.387Z
Comments.Text: Could you provide an example of a transaction description? (with fake name of course). Unless the descriptions are really extensive it sounds hard to me to believe that the pre trained model from spacy perform badly and require further training.
Comments.UserDisplayName: Edoardo Guerriero
Tags: <machine-learning><deep-learning><natural-language-processing><named-entity-recognition><spacy>
Title: How much labelling is required for NER with SpaCy?
UserDisplayName: Unicorn07
ThreadText: # How much labelling is required for NER with SpaCy?
  
  (Asked by: username_0 on 2021-11-23)
  
  I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of thousands of different merchants.
  
  Some questions that I have:
  
  * How much labelling do I need to do given the number of merchants I need to extract?
  * How many different instances of the same merchant I need to label to get decent results?
  
  **Tags:** machine-learning, deep-learning, natural-language-processing, named-entity-recognition, spacy
  > **Comments:**
  > **username_1**: Could you provide an example of a transaction description? (with fake name of course). Unless the descriptions are really extensive it sounds hard to me to believe that the pre trained model from spacy perform badly and require further training.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2023-08-03. Score: 0
  
  It depend on your workflow, language of text. Your official guide at here <https://github.com/explosion/assets/blob/main/Prodigy/Prodigy_NER_flowchart_v2_0_0_light.pdf> . You can know how many data is enough. You can see few number `4000`, `25%`, `2000`, etc.
  
  
  ---
  
  ### Answer by username_3 on 2021-12-26. Score: 0
  
  There is no specific number of labels that is "enough". For simple cases you can start with a few hundred examples, but normally you'll want several thousand.
  
  Since you have a large number of classes your problem might be a harder one, but on the other hand it could be easy if most of your text is like "This merchant is called XXX".
  
  
  ---
  
  ### Answer by username_4 on 2021-12-26. Score: 0
  
  In my experience with NER with Spacy, and disagreeing with [this stackoverflow solution](https://stackoverflow.com/questions/59488470/how-many-training-datasentences-are-required-for-custom-ner-using-spacy-python) and as @polm23 rightly mentioned, **a several thousand samples for each entity should generate/predict entities**, otherwise spacy would just recognise them based on default spacy entity types (mainly 'work-of-art')
  
  > **Comments:**
  > **username_5**: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  > **username_6**: This does not provide an answer to the question. Once you have sufficient [reputation](https://ai.stackexchange.com/help/whats-reputation) you will be able to [comment on any post](https://ai.stackexchange.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/19616)  
  
  ---
  
  