Body: <p>I am working with co-reference resolution in a large text. Is there an optimal way to split the text into small parts? Or the best correct procedure is to use the entire text?</p>
  <p>Just for reference, I am using the library <a href="https://spacy.io/universe/project/neuralcoref" rel="nofollow noreferrer">spacy-neuralcoref</a> in Python that is based on <a href="https://www.aclweb.org/anthology/D16-1245.pdf" rel="nofollow noreferrer">Deep Reinforcement Learning for Mention-Ranking Coreference Models</a> by Kevin Clark and Christopher D. Manning, EMNLP 2016.</p>
  <p>Why am I asking about splitting the text?</p>
  <p>I am applying coreference to chapters of books (roughly 30 pages of text). All the examples I have seen show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right?</p>
  <p>I didn't check all databases that people use to test coreference but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.</p>
  <p><strong>A test Example</strong>:</p>
  <blockquote>
  <p><strong>TST1-MUC3-0001</strong></p>
  <p>GUATEMALA CITY, 4 FEB 90 (ACAN-EFE) -- [TEXT] THE GUATEMALA ARMY
  DENIED TODAY THAT GUERRILLAS ATTACKED THE &quot;SANTO TOMAS&quot; PRESIDENTIAL
  FARM, LOCATED ON THE PACIFIC SIDE, WHERE PRESIDENT CEREZO HAS BEEN
  STAYING SINCE 2 FEBRUARY.</p>
  <p>A REPORT PUBLISHED BY THE &quot;CERIGUA&quot; NEWS AGENCY -- MOUTHPIECE OF
  THE GUATEMALAN NATIONAL REVOLUTIONARY UNITY (URNG) -- WHOSE MAIN
  OFFICES ARE IN MEXICO, SAYS THAT A GUERRILLA COLUMN ATTACKED THE FARM
  2 DAYS AGO.</p>
  <p>HOWEVER, ARMED FORCES SPOKESMAN COLONEL LUIS ARTURO ISAACS SAID
  THAT THE ATTACK, WHICH RESULTED IN THE DEATH OF A CIVILIAN WHO WAS
  PASSING BY AT THE TIME OF THE SKIRMISH, WAS NOT AGAINST THE FARM, AND
  THAT PRESIDENT CEREZO IS SAFE AND SOUND.</p>
  <p>HE ADDED THAT ON 3 FEBRUARY PRESIDENT CEREZO MET WITH THE
  DIPLOMATIC CORPS ACCREDITED IN GUATEMALA.</p>
  <p>THE GOVERNMENT ALSO ISSUED A COMMUNIQUE DESCRIBING THE REBEL REPORT
  AS &quot;FALSE AND INCORRECT,&quot; AND STRESSING THAT THE PRESIDENT WAS NEVER
  IN DANGER.</p>
  <p>COL ISAACS SAID THAT THE GUERRILLAS ATTACKED THE &quot;LA EMINENCIA&quot;
  FARM LOCATED NEAR THE &quot;SANTO TOMAS&quot; FARM, WHERE THEY BURNED THE
  FACILITIES AND STOLE FOOD.</p>
  <p>A MILITARY PATROL CLASHED WITH A REBEL COLUMN AND INFLICTED THREE
  CASUALTIES, WHICH WERE TAKEN AWAY BY THE GUERRILLAS WHO FLED TO THE
  MOUNTAINS, ISAACS NOTED.</p>
  <p>HE ALSO REPORTED THAT GUERRILLAS KILLED A PEASANT IN THE CITY OF
  FLORES, IN THE NORTHERN EL PETEN DEPARTMENT, AND BURNED A TANK TRUCK.</p>
  </blockquote>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-25T03:00:06.223Z
Comments.Text: I am applying coreference to chapters of books (roughtly 30 pages of text). All the examples I have seem show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right? You are saying that the best approach is to input the entire chapter?
Comments.UserDisplayName: DanielTheRocketMan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-25T03:11:11.153Z
Comments.Text: I didn't check all databases but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.
Comments.UserDisplayName: DanielTheRocketMan
Tags: <natural-language-processing>
Title: Is there an optimal way to split the text into small parts when working with co-reference resolution?
UserDisplayName: DanielTheRocketMan
ThreadText: # Is there an optimal way to split the text into small parts when working with co-reference resolution?
  
  (Asked by: username_0 on 2020-07-24)
  
  I am working with co-reference resolution in a large text. Is there an optimal way to split the text into small parts? Or the best correct procedure is to use the entire text?
  
  Just for reference, I am using the library [spacy-neuralcoref](https://spacy.io/universe/project/neuralcoref) in Python that is based on [Deep Reinforcement Learning for Mention-Ranking Coreference Models](https://www.aclweb.org/anthology/D16-1245.pdf) by Kevin Clark and Christopher D. Manning, EMNLP 2016.
  
  Why am I asking about splitting the text?
  
  I am applying coreference to chapters of books (roughly 30 pages of text). All the examples I have seen show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right?
  
  I didn't check all databases that people use to test coreference but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.
  
  **A test Example**:
  
  > **TST1-MUC3-0001**
  >
  > GUATEMALA CITY, 4 FEB 90 (ACAN-EFE) -- [TEXT] THE GUATEMALA ARMY
  > DENIED TODAY THAT GUERRILLAS ATTACKED THE "SANTO TOMAS" PRESIDENTIAL
  > FARM, LOCATED ON THE PACIFIC SIDE, WHERE PRESIDENT CEREZO HAS BEEN
  > STAYING SINCE 2 FEBRUARY.
  >
  > A REPORT PUBLISHED BY THE "CERIGUA" NEWS AGENCY -- MOUTHPIECE OF
  > THE GUATEMALAN NATIONAL REVOLUTIONARY UNITY (URNG) -- WHOSE MAIN
  > OFFICES ARE IN MEXICO, SAYS THAT A GUERRILLA COLUMN ATTACKED THE FARM
  > 2 DAYS AGO.
  >
  > HOWEVER, ARMED FORCES SPOKESMAN COLONEL LUIS ARTURO ISAACS SAID
  > THAT THE ATTACK, WHICH RESULTED IN THE DEATH OF A CIVILIAN WHO WAS
  > PASSING BY AT THE TIME OF THE SKIRMISH, WAS NOT AGAINST THE FARM, AND
  > THAT PRESIDENT CEREZO IS SAFE AND SOUND.
  >
  > HE ADDED THAT ON 3 FEBRUARY PRESIDENT CEREZO MET WITH THE
  > DIPLOMATIC CORPS ACCREDITED IN GUATEMALA.
  >
  > THE GOVERNMENT ALSO ISSUED A COMMUNIQUE DESCRIBING THE REBEL REPORT
  > AS "FALSE AND INCORRECT," AND STRESSING THAT THE PRESIDENT WAS NEVER
  > IN DANGER.
  >
  > COL ISAACS SAID THAT THE GUERRILLAS ATTACKED THE "LA EMINENCIA"
  > FARM LOCATED NEAR THE "SANTO TOMAS" FARM, WHERE THEY BURNED THE
  > FACILITIES AND STOLE FOOD.
  >
  > A MILITARY PATROL CLASHED WITH A REBEL COLUMN AND INFLICTED THREE
  > CASUALTIES, WHICH WERE TAKEN AWAY BY THE GUERRILLAS WHO FLED TO THE
  > MOUNTAINS, ISAACS NOTED.
  >
  > HE ALSO REPORTED THAT GUERRILLAS KILLED A PEASANT IN THE CITY OF
  > FLORES, IN THE NORTHERN EL PETEN DEPARTMENT, AND BURNED A TANK TRUCK.
  
  **Tags:** natural-language-processing
  > **Comments:**
  > **username_0**: I am applying coreference to chapters of books (roughtly 30 pages of text). All the examples I have seem show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right? You are saying that the best approach is to input the entire chapter?  
  > **username_0**: I didn't check all databases but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.  
  
  ---
  
  ## 0 answers
  
  