Answers.Body: <p><strong>It might be worthwhile to try <a href="https://en.wikipedia.org/wiki/Tf%E2%80%93idf" rel="nofollow noreferrer">TF-IDF</a> and see if that works for you.</strong></p>
  <p>Score each term in each document proportional to how often it occurs in that document, but inversely proportional to how often it occurs across multiple documents. Then look at the the terms that have the highest scores for each document. You can use scikit-learn's <a href="https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html" rel="nofollow noreferrer">TF-IDF Vectorizer</a> to help you with this, if you are using Python.</p>
  <p>Presumably, words that are highly specific and relevant to each of your three documents will stand out, and words that relate to personal information (as well as generic legal terms and non-specific English words) will be common to multiple documents and get filtered out.</p>
  <p><strong>Note:</strong> This will get you the specific <em>words</em> that are particular to each document. If the type of &quot;content&quot; you are seeking to extract goes beyond the word level, then you might have to take a different approach. Perhaps one way is to use the words obtained from TF-IDF to highlight the places in each document where the desired content might be found.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-10T04:01:49.950Z
Answers.Text: This may be a good base to obtain an extraction-based summarization. Thanks.
Answers.UserDisplayName: fuyutsuki
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-08T18:53:11.063Z
Answers.LastActivityDate: 2021-10-08T18:53:11.063Z
Answers.UserDisplayName: Zayn
Answers.Body: <p>I think the best task for your purpose is <strong><a href="https://en.wikipedia.org/wiki/Named-entity_recognition" rel="nofollow noreferrer">name entity recognition</a></strong> (NER) rather than text summarization.</p>
  <p>The logic is the following: if the three classes of documents are truly specific, there would be specific entities for each of them, but since the documents are linked by information about a single individual, all entities related to that individual and not to the specific domain would be shared.</p>
  <p>So the most obvious shared entity in all documents, the name of the individual, could be identified and then pruned in all document, same holds for every other shared entity (can't came up with more clever examples right now).</p>
  <p>If you work with python, <a href="https://spacy.io/usage/linguistic-features#named-entities" rel="nofollow noreferrer">SpaCy</a> offer pretrained models that do a great job already also for NER, and in several languages as well. But you might consider to train your own model as well, maybe retrain on top of spacy models, cause for these type of tasks, the most information you can provide about which entities belongs to which class, the best the performances, and unfortunately, generic use models can account for many entities, but they can't associate them directly to specific domains of interest.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-10T03:50:22.967Z
Answers.Text: In my situation, N varies and the nature of the unique content is unknown. How can we make NER work in this case?
Answers.UserDisplayName: fuyutsuki
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-10T13:33:52.570Z
Answers.Text: The models in spacy are pretrained on billions of documents, so you don't need to know before hand every name or entity. And you said that one document is law related, the second is about marriage and the third about company stuff, so to me sound like the content domains are known?
Answers.UserDisplayName: Edoardo Guerriero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-08T08:41:31.280Z
Answers.LastActivityDate: 2021-10-08T08:41:31.280Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>I now have <span class="math-container">$N$</span> documents, which share common content and they have special unique content.</p>
  <p>Say I have <span class="math-container">$3$</span> legal documents related to the same person. Document <span class="math-container">$A$</span> is about land law, document <span class="math-container">$B$</span> is about company law and document <span class="math-container">$C$</span> is about marriage law.
  How can I extract the land, company and marriage content from each document respectively and skip the common personal information?</p>
  <p>It sounds like text-summarization but with a very different nature. Any idea is welcome.</p>
  <p>Edit: In my situation, <span class="math-container">$N$</span> varies and the nature of the unique content is unknown.</p>
  
Tags: <natural-language-understanding><text-summarization>
Title: Among N documents, how to summarize the most unique content in each document?
UserDisplayName: fuyutsuki
ThreadText: # Among N documents, how to summarize the most unique content in each document?
  
  (Asked by: username_0 on 2021-10-08)
  
  I now have $N$ documents, which share common content and they have special unique content.
  
  Say I have $3$ legal documents related to the same person. Document $A$ is about land law, document $B$ is about company law and document $C$ is about marriage law.
  How can I extract the land, company and marriage content from each document respectively and skip the common personal information?
  
  It sounds like text-summarization but with a very different nature. Any idea is welcome.
  
  Edit: In my situation, $N$ varies and the nature of the unique content is unknown.
  
  **Tags:** natural-language-understanding, text-summarization
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2021-10-08. Score: 0 | Accepted answer
  
  **It might be worthwhile to try [TF-IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) and see if that works for you.**
  
  Score each term in each document proportional to how often it occurs in that document, but inversely proportional to how often it occurs across multiple documents. Then look at the the terms that have the highest scores for each document. You can use scikit-learn's [TF-IDF Vectorizer](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html) to help you with this, if you are using Python.
  
  Presumably, words that are highly specific and relevant to each of your three documents will stand out, and words that relate to personal information (as well as generic legal terms and non-specific English words) will be common to multiple documents and get filtered out.
  
  **Note:** This will get you the specific *words* that are particular to each document. If the type of "content" you are seeking to extract goes beyond the word level, then you might have to take a different approach. Perhaps one way is to use the words obtained from TF-IDF to highlight the places in each document where the desired content might be found.
  
  > **Comments:**
  > **username_0**: This may be a good base to obtain an extraction-based summarization. Thanks.  
  
  ---
  
  ### Answer by username_2 on 2021-10-08. Score: 0
  
  I think the best task for your purpose is **[name entity recognition](https://en.wikipedia.org/wiki/Named-entity_recognition)** (NER) rather than text summarization.
  
  The logic is the following: if the three classes of documents are truly specific, there would be specific entities for each of them, but since the documents are linked by information about a single individual, all entities related to that individual and not to the specific domain would be shared.
  
  So the most obvious shared entity in all documents, the name of the individual, could be identified and then pruned in all document, same holds for every other shared entity (can't came up with more clever examples right now).
  
  If you work with python, [SpaCy](https://spacy.io/usage/linguistic-features#named-entities) offer pretrained models that do a great job already also for NER, and in several languages as well. But you might consider to train your own model as well, maybe retrain on top of spacy models, cause for these type of tasks, the most information you can provide about which entities belongs to which class, the best the performances, and unfortunately, generic use models can account for many entities, but they can't associate them directly to specific domains of interest.
  
  > **Comments:**
  > **username_0**: In my situation, N varies and the nature of the unique content is unknown. How can we make NER work in this case?  
  > **username_2**: The models in spacy are pretrained on billions of documents, so you don't need to know before hand every name or entity. And you said that one document is law related, the second is about marriage and the third about company stuff, so to me sound like the content domains are known?  
  
  ---
  
  