<p>I am working with co-reference resolution in a large text. Is there an optimal way to split the text into small parts? Or the best correct procedure is to use the entire text?</p>
<p>Just for reference, I am using the library <a href="https://spacy.io/universe/project/neuralcoref" rel="nofollow noreferrer">spacy-neuralcoref</a> in Python that is based on <a href="https://www.aclweb.org/anthology/D16-1245.pdf" rel="nofollow noreferrer">Deep Reinforcement Learning for Mention-Ranking Coreference Models</a> by Kevin Clark and Christopher D. Manning, EMNLP 2016.</p>
<p>Why am I asking about splitting the text?</p>
<p>I am applying coreference to chapters of books (roughly 30 pages of text). All the examples I have seen show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right?</p>
<p>I didn't check all databases that people use to test coreference but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.</p>
<p><strong>A test Example</strong>:</p>
<blockquote>
<p><strong>TST1-MUC3-0001</strong></p>
<p>GUATEMALA CITY, 4 FEB 90 (ACAN-EFE) -- [TEXT] THE GUATEMALA ARMY
DENIED TODAY THAT GUERRILLAS ATTACKED THE &quot;SANTO TOMAS&quot; PRESIDENTIAL
FARM, LOCATED ON THE PACIFIC SIDE, WHERE PRESIDENT CEREZO HAS BEEN
STAYING SINCE 2 FEBRUARY.</p>
<p>A REPORT PUBLISHED BY THE &quot;CERIGUA&quot; NEWS AGENCY -- MOUTHPIECE OF
THE GUATEMALAN NATIONAL REVOLUTIONARY UNITY (URNG) -- WHOSE MAIN
OFFICES ARE IN MEXICO, SAYS THAT A GUERRILLA COLUMN ATTACKED THE FARM
2 DAYS AGO.</p>
<p>HOWEVER, ARMED FORCES SPOKESMAN COLONEL LUIS ARTURO ISAACS SAID
THAT THE ATTACK, WHICH RESULTED IN THE DEATH OF A CIVILIAN WHO WAS
PASSING BY AT THE TIME OF THE SKIRMISH, WAS NOT AGAINST THE FARM, AND
THAT PRESIDENT CEREZO IS SAFE AND SOUND.</p>
<p>HE ADDED THAT ON 3 FEBRUARY PRESIDENT CEREZO MET WITH THE
DIPLOMATIC CORPS ACCREDITED IN GUATEMALA.</p>
<p>THE GOVERNMENT ALSO ISSUED A COMMUNIQUE DESCRIBING THE REBEL REPORT
AS &quot;FALSE AND INCORRECT,&quot; AND STRESSING THAT THE PRESIDENT WAS NEVER
IN DANGER.</p>
<p>COL ISAACS SAID THAT THE GUERRILLAS ATTACKED THE &quot;LA EMINENCIA&quot;
FARM LOCATED NEAR THE &quot;SANTO TOMAS&quot; FARM, WHERE THEY BURNED THE
FACILITIES AND STOLE FOOD.</p>
<p>A MILITARY PATROL CLASHED WITH A REBEL COLUMN AND INFLICTED THREE
CASUALTIES, WHICH WERE TAKEN AWAY BY THE GUERRILLAS WHO FLED TO THE
MOUNTAINS, ISAACS NOTED.</p>
<p>HE ALSO REPORTED THAT GUERRILLAS KILLED A PEASANT IN THE CITY OF
FLORES, IN THE NORTHERN EL PETEN DEPARTMENT, AND BURNED A TANK TRUCK.</p>
</blockquote>

CC BY-SA 4.0
2020-07-25T03:00:06.223Z
I am applying coreference to chapters of books (roughtly 30 pages of text). All the examples I have seem show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right? You are saying that the best approach is to input the entire chapter?
DanielTheRocketMan
CC BY-SA 4.0
2020-07-25T03:11:11.153Z
I didn't check all databases but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.
DanielTheRocketMan
<natural-language-processing>
Is there an optimal way to split the text into small parts when working with co-reference resolution?
DanielTheRocketMan
# Is there an optimal way to split the text into small parts when working with co-reference resolution?

(Asked by: username_0 on 2020-07-24)

I am working with co-reference resolution in a large text. Is there an optimal way to split the text into small parts? Or the best correct procedure is to use the entire text?

Just for reference, I am using the library [spacy-neuralcoref](https://spacy.io/universe/project/neuralcoref) in Python that is based on [Deep Reinforcement Learning for Mention-Ranking Coreference Models](https://www.aclweb.org/anthology/D16-1245.pdf) by Kevin Clark and Christopher D. Manning, EMNLP 2016.

Why am I asking about splitting the text?

I am applying coreference to chapters of books (roughly 30 pages of text). All the examples I have seen show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right?

I didn't check all databases that people use to test coreference but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.

**A test Example**:

> **TST1-MUC3-0001**
>
> GUATEMALA CITY, 4 FEB 90 (ACAN-EFE) -- [TEXT] THE GUATEMALA ARMY
> DENIED TODAY THAT GUERRILLAS ATTACKED THE "SANTO TOMAS" PRESIDENTIAL
> FARM, LOCATED ON THE PACIFIC SIDE, WHERE PRESIDENT CEREZO HAS BEEN
> STAYING SINCE 2 FEBRUARY.
>
> A REPORT PUBLISHED BY THE "CERIGUA" NEWS AGENCY -- MOUTHPIECE OF
> THE GUATEMALAN NATIONAL REVOLUTIONARY UNITY (URNG) -- WHOSE MAIN
> OFFICES ARE IN MEXICO, SAYS THAT A GUERRILLA COLUMN ATTACKED THE FARM
> 2 DAYS AGO.
>
> HOWEVER, ARMED FORCES SPOKESMAN COLONEL LUIS ARTURO ISAACS SAID
> THAT THE ATTACK, WHICH RESULTED IN THE DEATH OF A CIVILIAN WHO WAS
> PASSING BY AT THE TIME OF THE SKIRMISH, WAS NOT AGAINST THE FARM, AND
> THAT PRESIDENT CEREZO IS SAFE AND SOUND.
>
> HE ADDED THAT ON 3 FEBRUARY PRESIDENT CEREZO MET WITH THE
> DIPLOMATIC CORPS ACCREDITED IN GUATEMALA.
>
> THE GOVERNMENT ALSO ISSUED A COMMUNIQUE DESCRIBING THE REBEL REPORT
> AS "FALSE AND INCORRECT," AND STRESSING THAT THE PRESIDENT WAS NEVER
> IN DANGER.
>
> COL ISAACS SAID THAT THE GUERRILLAS ATTACKED THE "LA EMINENCIA"
> FARM LOCATED NEAR THE "SANTO TOMAS" FARM, WHERE THEY BURNED THE
> FACILITIES AND STOLE FOOD.
>
> A MILITARY PATROL CLASHED WITH A REBEL COLUMN AND INFLICTED THREE
> CASUALTIES, WHICH WERE TAKEN AWAY BY THE GUERRILLAS WHO FLED TO THE
> MOUNTAINS, ISAACS NOTED.
>
> HE ALSO REPORTED THAT GUERRILLAS KILLED A PEASANT IN THE CITY OF
> FLORES, IN THE NORTHERN EL PETEN DEPARTMENT, AND BURNED A TANK TRUCK.

**Tags:** natural-language-processing

> **Comments:**
> **username_0**: I am applying coreference to chapters of books (roughtly 30 pages of text). All the examples I have seem show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right? You are saying that the best approach is to input the entire chapter?  
> **username_0**: I didn't check all databases but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.  

---

## 0 answers

